A Deep Learning-Based Robot Vision Recognition Decision Control Method

Abstract

This invention discloses a deep learning-based robot visual recognition and decision-making control method, belonging to the field of control system technology. It includes constructing a thought decision tree based on input commands and real-time visual images, obtaining the linear relationship between the middle-level leaf nodes and the top leaf nodes, and using this relationship to associate current visual features with task intent. By constructing a multi-level thought decision tree structure, combined with multi-modal feature fusion and a deep learning engine, this invention achieves deep fusion and unified representation of multi-source heterogeneous data. Furthermore, by constructing a fusion search engine and semantic alignment mechanism, it effectively integrates text commands with visual images and other modal information, enabling accurate parsing of task intent and collaborative processing of environmental perception, and improving the real-time response and decision-making accuracy of service robots for complex tasks.

Description

Technical Field

[0001] This invention relates to the field of control system technology, specifically to a robot vision recognition and decision control method based on deep learning. Background Technology

[0002] With the continuous development of artificial intelligence, computer vision and human-computer interaction technologies, service robots, industrial robots and special operation robots are being deployed more and more widely in various application scenarios. In actual task execution, service robots need to perceive the environment and understand the task objectives based on natural language instructions, and plan a reasonable sequence of behaviors to complete complex operations. Therefore, a robot vision recognition decision control method based on deep learning is needed.

[0003] A search revealed Chinese invention patent CN114298244A, which discloses "a decision control method, device, and system for intelligent agent group interaction." This application constructs an initial decision control model including a top-level learning model and a bottom-level learning model, and performs top-level and bottom-level fusion training on the initial decision control model to obtain a final decision control model for decision control. This decision control method, device, and system improve the effectiveness of decision control during intelligent agent group interaction.

[0004] In addition, the Chinese invention patent with announcement number "CN108985463A" discloses "Artificial Intelligence Combat Method and Robot System Based on Knowledge Base and Deep Learning". This application solves the problem of insufficient combat case samples for effective deep learning and decision support by automatically generating combat case samples and combining them with real combat case samples. It improves the effect of deep learning and the ability to support combat decisions, and enhances the accuracy of deep learning models and the initiative and intelligence of combat robots.

[0005] However, in practical use, the aforementioned patented methods and similar patented methods have failed to establish a unified and dynamically related deep reasoning mechanism between natural language commands and real-time visual perception, which makes it easy for robots to have problems such as untimely response and rigid behavior planning when facing complex and ever-changing environments. Summary of the Invention

[0006] The purpose of this invention is to provide a robot vision recognition decision control method based on deep learning to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a robot vision recognition decision control method based on deep learning, comprising:

[0008] Based on input instructions and real-time visual images, a thought decision tree is constructed. The trunk of the thought decision tree is the main structural representation of the integrated interconnected big data and deep learning search engine. Each leaf node of the thought decision tree represents a specific decision logic node and can be divided into three functional modules: the top leaf node corresponds to the task target parsing layer, which is responsible for semantic decomposition and intent recognition; the middle leaf nodes constitute the feature vector understanding layer of real-time visual images; and the bottom leaf nodes form the action generation layer, which completes the planning and execution of behavior sequences.

[0009] Obtain the linear relationship between the middle-layer leaf nodes and the top leaf nodes to realize the association between the current visual features and the task intent, and ensure that the task semantic target can be correctly inferred from the real-time image content.

[0010] Retrieve the content of the corresponding bottom-level leaf node based on the linear relationship;

[0011] Based on the attribute characteristics of the decision objective and the content of the input instructions, corresponding decision adjustment suggestions are constructed for each bottom leaf node to dynamically optimize the behavior generation strategy and improve adaptability in complex environments.

[0012] As a further preferred embodiment of this technical solution, the method for constructing a thought decision tree includes:

[0013] Build a data acquisition library to receive heterogeneous data from multiple sources;

[0014] Build a converged search engine for retrieving and analyzing multi-source heterogeneous data;

[0015] The framework of the decision tree is based on a data collection database and a fusion search engine.

[0016] Constructing a leaf node framework for thinking and decision-making;

[0017] The leaf node framework of the thinking and decision-making system assigns content to the top-level leaf node based on input instructions;

[0018] The trunk framework divides the mid-level leaf nodes into functional categories based on the feature vectors in the real-time visual image.

[0019] As a further preferred embodiment of this technical solution, the construction method of the integrated search engine includes:

[0020] Obtain feature vector representations of multi-source heterogeneous data for standardized mapping from data space to vector space;

[0021] Establish a multimodal feature fusion framework to achieve deep interaction and semantic alignment of heterogeneous features;

[0022] Construct a co-evolutionary paradigm for semantic indexing and deep learning models, and design a dynamic update method based on incremental learning;

[0023] Implement optimized deployment of joint retrieval scheduling strategies and establish a multi-engine dynamic allocation mechanism based on reinforcement learning.

[0024] As a further preferred embodiment of this technical solution, the dynamic update method allows the data acquisition library and deep learning model parameters to be optimized synchronously. After receiving new visual or semantic input data, the existing deep learning model is adjusted based on the online learning algorithm, and the semantic index items are updated to adapt to the new task requirements.

[0025] As a further preferred embodiment of this technical solution, the multi-engine dynamic allocation mechanism includes:

[0026] Using text semantic feature vectors as the baseline template, by obtaining regions in visual feature vectors that match text semantic elements and statistically analyzing the proportion of matching feature vectors, a semantic matching degree calculation formula is established: Matching degree = Σ(number of matching feature vectors) / Σ(total number of feature vectors) × 100%;

[0027] When the matching degree exceeds a preset threshold of 50%, the semantic search engine is invoked first; otherwise, the online deep learning engine is activated for incremental learning.

[0028] As a further preferred embodiment of this technical solution, the method for obtaining the content of the corresponding bottom-level leaf nodes includes:

[0029] Obtain linear path index relationships by selecting path graphs in the thinking decision tree, with the selection criteria following the path graph from "top-level task node → middle-level perception node";

[0030] Based on the linear path index relationship, the content of the corresponding bottom leaf node is determined in the thinking decision tree. The correspondence between the top task node and the middle perception node is obtained, and the content of the bottom leaf node is determined based on the correspondence.

[0031] As a further preferred embodiment of this technical solution, the method for constructing decision adjustment suggestions includes:

[0032] The execution bottleneck score of the bottom leaf node is initialized and assigned to detect performance degradation, execution failure or response delay in the task execution process in the bottom leaf node of the thinking decision tree, thereby identifying bottlenecks or potential error sources.

[0033] The bottom leaf nodes are sorted according to their execution bottleneck scores. This sorting is based on the execution bottleneck scores of each leaf node, and the ranking results determine the priority of the optimization process. Nodes with higher scores will be optimized first.

[0034] Based on the sorting results, adjustment suggestions are constructed for the corresponding bottom-level leaf nodes.

[0035] As a further preferred embodiment of this technical solution, the method for initializing and assigning values ​​to the bottom leaf nodes includes:

[0036] Define the execution bottleneck evaluation function: ; Used to represent the bottleneck score of the nth bottom leaf node;

[0037] Used to represent the task execution error rate, it reflects the failure rate of service robots when performing tasks; specifically, it represents the ratio of the number of incorrect identifications to the total number of attempts. ;

[0038] Task completion delay represents the time interval between the service robot receiving the instruction and starting task execution. It is obtained by monitoring the timestamps of task execution and recording the time difference between instruction input and task start-up by the service robot. ;

[0039] Resource utilization rate represents the consumption of resources within the service robot during task execution. It is obtained by monitoring the resource usage of the service robot's internal hardware. ;

[0040] These are the task weight coefficient, resource constraint coefficient, and environmental adaptability coefficient, respectively.

[0041] As a further preferred embodiment of this technical solution, The number of urgent tasks indicates tasks that have a time limit, while the number of complex tasks refers to tasks that have more than one command to execute.

[0042] The available resources are calculated through real-time collection and analysis of service robot monitoring data, and the resource constraint coefficient is also considered. A decrease indicates sufficient resources, while a decrease indicates higher resource utilization. An increase indicates that resource allocation needs to be optimized when performing the task;

[0043] The number of features within an image is obtained through the service robot's vision system.

[0044] Compared with the prior art, the beneficial effects of the present invention are:

[0045] This deep learning-based robot vision recognition and decision control method constructs a multi-level thinking decision tree structure, combines multi-modal feature fusion with a deep learning engine, realizes deep fusion and unified representation of multi-source heterogeneous data, and effectively integrates text instructions with visual images and other modal information by constructing a fusion search engine and semantic alignment mechanism. This enables accurate parsing of task intent and collaborative processing of environmental perception, and improves the real-time response and decision accuracy of service robots for complex tasks.

[0046] Furthermore, by establishing a linear mapping relationship between top-level semantic nodes and mid-level perception nodes, service robots can quickly locate key areas related to task objectives based on real-time images, thereby accurately inferring semantic intent and enhancing the dynamic adaptability of behavior planning strategies.

[0047] Finally, by generating bottleneck scores and adjustment suggestions based on the execution status of the underlying behavioral nodes, the action sequence can be optimized in real time during task execution, improving execution stability under resource-constrained or scenario-changing conditions. Attached Figure Description

[0048] Figure 1 This is a flowchart of the steps of the method disclosed in this invention;

[0049] Figure 2 This is an auxiliary illustration of step S100 of the present invention;

[0050] Figure 3 This is a diagram illustrating the steps of constructing the decision tree of this invention.

[0051] Figure 4 This is a flowchart illustrating the steps of constructing the decision adjustment suggestions of this invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Before understanding the technical solution proposed in this application, it should be clear that the control method proposed in this application is mainly applied in the fields of service robots and industrial robots, and is particularly suitable for intelligent robot systems that need to combine natural language command parsing and real-time visual perception to make autonomous decisions.

[0054] Specifically, such as Figure 1As shown, the present invention provides a technical solution: a robot vision recognition decision control method based on deep learning, including: steps S100-S400.

[0055] Step S100: Construct a thought decision tree based on the input instructions and real-time visual images.

[0056] It should be noted that this is for reference only. Figure 2 As can be seen, the trunk of the thinking decision tree disclosed in step S100 of this application is the main structural representation of the integrated interconnected big data and deep learning search engine. Each leaf node of the thinking decision tree represents a specific decision logic node and can be divided into three functional modules: the top leaf node corresponds to the task target parsing layer, which is responsible for semantic decomposition and intent recognition; the middle leaf nodes constitute the feature vector understanding layer of real-time visual images; and the bottom leaf nodes form the action generation layer, which completes the planning and execution of behavior sequences.

[0057] It should be added that, in this application, the trunk of the thought decision tree serves as the processing center for decision control, integrating multi-source heterogeneous data, a real-time online deep learning engine, and a semantic search engine.

[0058] It is worth noting that, reference Figure 3 It can be seen that the method for constructing the thinking decision tree in step S100 includes: steps S101-S105.

[0059] Step S101: Construct a data collection library.

[0060] It should be noted that the data acquisition library in step S101 is used to receive multi-source heterogeneous data. Specifically, in the technical solution proposed in this application, it is mainly used to acquire multi-source heterogeneous data in the categories of visual data and instruction data. In addition, it should be noted that the data acquisition library constructed in step S101 in this application is used to provide data support for the construction of the thinking decision tree.

[0061] Step S102: Build a converged search engine.

[0062] It should be noted that the fusion search engine in step S102 includes a real-time online deep learning engine and a semantic search engine. It should be added that both real-time online deep learning engines and semantic search engines are common search tools in existing technologies. By combining the two, rapid retrieval and analysis of multi-source heterogeneous data can be achieved, providing technical support for the construction of the thinking decision tree.

[0063] It should be added that the core framework of the thinking decision tree consists of the data collection library disclosed in step S101 and the fusion search engine disclosed in step S102.

[0064] It is worth noting that the combination of real-time online deep learning engine and semantic search engine includes steps S102A-S102D.

[0065] Step S102A: Obtain the feature vector representation of multi-source heterogeneous data to achieve standardized mapping from data space to vector space.

[0066] It should be noted that, in this application, since the multi-source heterogeneous data are visual data and instruction data respectively, the feature vector representation of the multi-source heterogeneous data in step S102A is an image feature vector, and the instruction data is a text feature vector.

[0067] Step S102B: Establish a multimodal feature fusion framework.

[0068] It is important to understand that after completing the feature vector representation, in order to achieve deep fusion of semantics within the text feature vector and image feature vector, a neural network architecture adapted to multimodal data fusion needs to be constructed. In actual construction, the alignment of multimodal information in the shared semantic space is completed by using the graph neural network (GNN) in the existing technology, thereby obtaining a feature representation with enhanced contextual association. It should be further explained that step S102B not only improves the complementarity between different modalities, but also provides a guarantee for the joint retrieval of the subsequent search engine.

[0069] As a preferred embodiment, this embodiment selects step S102B for specific explanation. In this embodiment, a cross-modal attention model based on the Transformer structure in the prior art is adopted. The visual and semantic vectors generated in step S102A are used as Key-Value and Query inputs, respectively. Taking "grab the red cup on the table" as the input instruction, the visual features will locate the red object region in the image feature vector, and "grab", "red", and "cup" in the semantic vector are used as tokens to guide visual recognition to focus on the target region (red object region).

[0070] Step S102C: Construct a co-evolutionary paradigm for semantic indexing and deep learning models, and design a dynamic update method based on incremental learning.

[0071] It should be noted that, considering the dynamic changes in the task environment and the real-time requirements, this step proposes a dynamic update method that allows the data acquisition library and deep learning model parameters to be optimized synchronously. Specifically, after receiving new visual or semantic input data, the existing deep learning model is adjusted based on online learning algorithms (such as incremental learning or meta-learning), and the semantic index items are updated to adapt to the new task requirements.

[0072] As a preferred implementation scheme, step S102C is used for explanation. Specifically, during the service robot's task execution, for example, when identifying a new type of tool (pliers), the service robot cannot correctly identify it. Since the service robot cannot make the corresponding action, the action feedback after the service robot's failure is used to record the sample (pliers) as a new sample into the data collection library in step S101. The service robot's vision system is updated at night under low load using the incremental training method in the existing technology. At the same time, the matching label of "pliers" and related task actions is added to the semantic index.

[0073] Step S102D: Implement optimized deployment of joint retrieval scheduling strategy and establish a multi-engine dynamic allocation mechanism based on reinforcement learning.

[0074] It should be noted that, in order to establish an efficient decision support mechanism, a joint retrieval scheduling strategy needs to be constructed to coordinate the query call order of semantic search and deep learning engines. Specifically, this implementation plan stipulates that the engine call priority should be dynamically adjusted based on the quantitative scoring results by evaluating the matching degree of input information in visual space.

[0075] As an optimized implementation plan, this section selects step S102D for technical analysis. When the service robot receives the voice command "Please hand me the black notebook on my desk," its execution process includes: First, the service robot extracts core semantic elements ("notebook," "black," and "desk") through a semantic search engine, and infers the task intent as "retrieving and delivering" based on the verb "hand over." At the same time, the service robot's vision system matches the collected visual feature vectors with the above semantic elements according to the multimodal feature fusion mechanism in step S102B. Specifically, using the text semantic feature vector as the benchmark template, the system locates the regions in the visual feature vectors that match the text semantic elements, counts the proportion of matching feature vectors, and establishes a semantic matching degree calculation formula: Matching degree = Σ (number of matching feature vectors) / Σ (total number of feature vectors) × 100%. When the matching degree exceeds the preset threshold of 50%, the semantic search engine is called first; otherwise, the online deep learning engine is activated for incremental learning.

[0076] It should be noted that the online deep learning engine performs incremental learning by updating the service set robot's vision system during low-load nighttime using the incremental training method in step S102C, while adding matching tags for "notebook", "black", "desk" and related task actions (retrieving and delivering items) to the semantic index.

[0077] Step S103: Construct the leaf node framework for thinking and decision-making.

[0078] It should be noted that in step S103, the thinking decision tree framework uses graph neural networks (GNNs) in the prior art to model the tree structure. Leaf nodes are connected through semantic relationships, and the trunk is based on the data collection library to build linear connections between each leaf node.

[0079] As a preferred implementation scheme, step S103 is selected for explanation. Specifically, when the input instruction is "clean the window", the top leaf node of the thought decision tree is "cleaning task", the middle leaf nodes are "identify the window" and "locate the stain area", and the bottom leaf nodes include the service robot's actions such as "move the arm" and "wiping action".

[0080] Step S104: The thinking decision leaf node framework assigns content to the top-level leaf node according to the input instructions.

[0081] It should be noted that in step S104, the input instruction is semantically parsed using a natural language processing model (T5 or GPT) in the prior art, extracting action verbs, target objects, and location phrases, and setting linear relationships. Based on the linear relationships, the action verbs, target objects, and location phrases are formed into corresponding tasks, mapped to the thinking decision tree framework, and the top-level leaf nodes are given functions.

[0082] Step S105: The trunk of the thinking decision tree is functionally divided into middle-layer leaf nodes based on the feature vectors in the real-time visual image.

[0083] It should be noted that in this step, the service robot's vision acquisition system captures images of the surrounding environment and extracts image features through a deep learning model (multimodal feature fusion mechanism). These feature vectors are then input into the middle layer leaf nodes to identify and understand real-time visual information, such as object category, location, and shape.

[0084] As a preferred implementation scheme, this implementation scheme is described with step S105 as the main body. Specifically, the service robot collects the current environment image through the vision system, that is, it collects the current environment image through the integrated camera. The image data is input to the visual feature extraction module of the cross-modal Transformer in the prior art, extracts the multi-dimensional feature vector of the image (including color, texture, edge, shape, etc.), and inputs it into the middle layer leaf node. For example, when the input command is "grab the red water cup", the vision system will focus on the object area in the image with the color feature of red and the shape of "cup", and map the feature to the middle layer nodes such as "water cup area" and "red area".

[0085] Step S200: Obtain the linear relationship between the middle-level leaf nodes and the top-level leaf nodes.

[0086] It should be noted that step S200 is used to determine the executability of the input instruction by obtaining a linear relationship. The linear relationship in step S200 corresponds to the top part of the thought decision tree.

[0087] Step S300: Obtain the content of the corresponding bottom leaf node based on the linear relationship.

[0088] It should be noted that step S300 is used to quantify the input instructions and visual results into control parameters, providing a specific instruction set (grasping point and angle), thereby improving the operability of the input instructions.

[0089] Specifically, the method for obtaining the content of the corresponding bottom leaf node includes: steps S301-S302.

[0090] Step S301: Obtain the linear path index relationship.

[0091] It should be noted that by selecting a path graph in the thought decision tree, the selection criteria follow the path graph from the top-level task node to the middle-level perception node. The path graph represents the connection relationship and transmission direction between different layer leaf nodes, thereby determining the path from the top-level task node to the bottom-level execution node.

[0092] Step S302: Determine the content of the corresponding bottom leaf node in the thought decision tree based on the linear path index relationship.

[0093] It should be noted that in step S302, when determining the position of the bottom leaf node, the correspondence between the top task node and the middle perception node is first obtained. The correspondence includes spatial and temporal aspects. The artificial intelligence association model in the prior art determines the content of the bottom leaf node by analyzing the correspondence.

[0094] As a preferred implementation scheme, this implementation method selects step S302 for description. Specifically, when the content corresponding to the top-level task node is "keep the room clean before 12 o'clock" and the content of the middle-level perception node is "identify the presence of toys that have fallen on the floor in the room", the artificial intelligence association model in the existing technology will analyze the constraint condition of "before 12 o'clock" in the time correspondence and compare it with "current time". At the same time, it will analyze the overlap range between the coordinate system of "floor area" and the working range of the service robot in the spatial correspondence and obtain the content of the bottom leaf node as "speed adjustment", "position movement" and "cleaning".

[0095] Step S400: Based on the attribute characteristics of the decision objective and the input instruction content, construct corresponding decision adjustment suggestions for each bottom leaf node.

[0096] It should be noted that the decision adjustment suggestion in step S400 is used to change the content of the bottom leaf nodes.

[0097] For details, please refer to Figure 4 It can be seen that the method for constructing decision adjustment suggestions includes steps S401-S403.

[0098] Step S401: Initialize and assign values ​​to the execution bottleneck scores of the bottom leaf nodes.

[0099] It should be added that the goal of step S401 is to detect problems such as performance degradation, execution failure or response delay during task execution in the bottom leaf nodes of the thinking decision tree, thereby identifying bottlenecks or potential sources of error.

[0100] Specifically, an execution bottleneck evaluation function is defined to address how to identify and initialize the bottom leaf nodes in step S401.

[0101] Right now It should be added that, Used to represent the bottleneck score of the nth bottom leaf node. Used to represent the task execution error rate, it reflects the failure rate of service robots when performing tasks; specifically, it represents the ratio of the number of incorrect identifications to the total number of attempts. , Task completion delay specifically refers to the time delay between the robot receiving the instruction and starting task execution. This is obtained by monitoring the timestamps of task execution through the service robot and recording the time difference between instruction input and task start. , Resource utilization rate represents the consumption of resources within the service robot during task execution. It is obtained by monitoring the resource usage of the service robot's internal hardware. For example, existing resource monitoring tools (such as `top` and `htop`) can be used to collect system resource usage in real time and calculate CPU utilization and memory usage rate. , These are the task weight coefficient, resource constraint coefficient, and environment adaptability coefficient, respectively. It should be added that... It should be noted that the number of urgent tasks refers to tasks with a time limit, while the number of complex tasks refers to tasks with more than one command to execute. It should be noted that "available resources" are calculated through real-time collection and analysis of service robot monitoring data. For example, when CPU utilization is low, the resource limitation coefficient is... A decrease indicates sufficient resources, while a decrease indicates higher resource utilization. An increase indicates that resource allocation needs to be optimized when performing the task. The number of features in the image is obtained through the service robot's vision system. The target area is represented by the multimodal feature fusion mechanism disclosed in step S102B. The total number of features in the image is captured and counted in real time by the service robot's vision recognition system. The more features there are, the more complex the environment is, which puts forward higher requirements on the service robot's vision recognition and processing capabilities. Therefore, the environmental adaptability coefficient will increase accordingly.

[0102] As a preferred implementation scheme, this scheme elaborates on the bottleneck evaluation function. Specifically, when performing the "moving a water cup" task, the service robot misidentifies the water cup's position, resulting in three failed attempts, for a total of five attempts. Therefore... =0.6, the task delay time is 4s, therefore =4, the service robot's (CPU usage is 70%), therefore =0.7, the total number of tasks is 5, the task content is the same, therefore the number of urgent tasks and the number of complex tasks are both 1, so the task weight coefficient is 0.2, the resource constraint coefficient is 0.3 (CPU usage is 70%), the environment adaptability coefficient is 0.2, the total number of features in the image content is 5, and the target feature is only one water cup, therefore... =0.2×0.6 +0.3×4 + 0.2×0.7 = 0.12 + 1.2 + 0.14 = 1.46, therefore, the bottleneck score of the bottom leaf node under this implementation scheme is 1.46.

[0103] It is worth noting that in step S401, by recording and analyzing the failure or inefficiency of task execution in the bottom leaf nodes, bottleneck factors affecting the effectiveness of decision-making are identified, such as identification failure, execution error, and environmental interference. In addition, it should be added that there is a linkage between step S401 and step S102C.

[0104] As a preferred implementation scheme, this scheme describes the linkage between steps S401 and S102C. For example, when the service robot receives a task instruction and begins execution, the robot monitors the task status in real time and calculates the bottleneck score (Bn). Furthermore, assuming that in the grasping task, the service robot finds that there are many grasping failures, resulting in an increased error rate (Er), thus generating a high bottleneck score, the evaluation result triggers the semantic index and deep learning model collaborative update mechanism in step S102C. The service robot will input the images of grasping failures and related data (such as pliers images) as samples into the data collection library. At the same time, the semantic search engine and deep learning model perform incremental training and updates based on the new data, improving the recognition ability of the item (such as pliers). Therefore, between steps S401 and S102C, the execution bottleneck is first evaluated in step S401 to discover the execution bottleneck in real time, and then the model update mechanism in step S102C dynamically adjusts the robot's visual recognition and semantic understanding capabilities to solve the root cause of the bottleneck, realizing a bottleneck feedback mechanism, that is, optimizing the robot's behavior through online learning, thereby executing tasks more efficiently and accurately.

[0105] Step S402: Sort the bottom leaf nodes according to the execution bottleneck score.

[0106] It should be noted that in step S402, each leaf node is sorted according to its execution bottleneck score. The sorting result of the execution bottleneck score determines the priority of the optimization process. Nodes with higher scores will be optimized first. Step S402 is used to provide a clear priority order for subsequent optimizations to ensure that the execution bottleneck is resolved.

[0107] Step S403: Construct adjustment suggestions for the corresponding bottom leaf nodes based on the sorting results.

[0108] It should be noted that in step S403, based on the sorting results of step S402, a specific optimization decision scheme is generated for each leaf node. Since the content represented by the bottleneck evaluation function includes three parameters: task execution error rate, task completion delay, and resource utilization rate, the adjustment suggestions need to set three types of optimization strategies: for nodes with high error rate, activate the incremental learning mechanism in step S102C to supplement the target object feature template; for nodes with high latency, send an instruction to the service robot to shorten the motion trajectory; and for nodes with high resource utilization, reduce the impact by disabling unnecessary functions within the service robot.

[0109] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended embodiments and their equivalents.

Claims

1. A robot vision recognition, decision-making, and control method based on deep learning, characterized in that, include: Based on input instructions and real-time visual images, a thought decision tree is constructed. The trunk of the thought decision tree is the main structural representation of the integrated interconnected big data and deep learning search engine. Each leaf node of the thought decision tree represents a specific decision logic node and can be divided into three functional modules: the top leaf node corresponds to the task target parsing layer, which is responsible for semantic decomposition and intent recognition; the middle leaf nodes constitute the feature vector understanding layer of real-time visual images; and the bottom leaf nodes form the action generation layer, which completes the planning and execution of behavior sequences. Obtain the linear relationship between the middle-layer leaf nodes and the top leaf nodes to realize the association between the current visual features and the task intent, and ensure that the task semantic target can be correctly inferred from the real-time image content. Retrieve the content of the corresponding bottom-level leaf node based on the linear relationship; Based on the attribute characteristics of the decision objective and the content of the input instructions, corresponding decision adjustment suggestions are constructed for each bottom leaf node to dynamically optimize the behavior generation strategy and improve adaptability in complex environments.

2. The robot vision recognition, decision-making, and control method based on deep learning according to claim 1, characterized in that: Methods for constructing decision trees include: Build a data acquisition library to receive heterogeneous data from multiple sources; Build a converged search engine for retrieving and analyzing multi-source heterogeneous data; The framework of the decision tree is based on a data collection database and a fusion search engine. Constructing a leaf node framework for thinking and decision-making; The leaf node framework of the thinking and decision-making system assigns content to the top-level leaf node based on input instructions; The trunk framework divides the mid-level leaf nodes into functional categories based on the feature vectors in the real-time visual image.

3. The robot vision recognition, decision-making, and control method based on deep learning according to claim 2, characterized in that: Methods for building converged search engines include: Obtain feature vector representations of multi-source heterogeneous data for standardized mapping from data space to vector space; Establish a multimodal feature fusion framework to achieve deep interaction and semantic alignment of heterogeneous features; Construct a co-evolutionary paradigm for semantic indexing and deep learning models, and design a dynamic update method based on incremental learning; Implement optimized deployment of joint retrieval scheduling strategies and establish a multi-engine dynamic allocation mechanism based on reinforcement learning.

4. The robot vision recognition and decision control method based on deep learning according to claim 3, characterized in that: The dynamic update method allows the data acquisition library and deep learning model parameters to be optimized synchronously. After receiving new visual or semantic input data, the existing deep learning model is adjusted based on the online learning algorithm, and the semantic index items are updated to adapt to the new task requirements.

5. The robot vision recognition and decision control method based on deep learning according to claim 3, characterized in that: The multi-engine dynamic allocation mechanism includes: Using text semantic feature vectors as the baseline template, by obtaining regions in visual feature vectors that match text semantic elements and statistically analyzing the proportion of matching feature vectors, a semantic matching degree calculation formula is established: Matching degree = Σ(number of matching feature vectors) / Σ(total number of feature vectors) × 100%; When the matching degree exceeds a preset threshold of 50%, the semantic search engine is invoked first; otherwise, the online deep learning engine is activated for incremental learning.

6. The robot vision recognition, decision-making, and control method based on deep learning according to claim 1, characterized in that: Methods for retrieving the content of the corresponding bottom-level leaf node include: Obtain linear path index relationships by selecting path graphs in the thinking decision tree, with the selection criteria following the path graph from "top-level task node → middle-level perception node"; Based on the linear path index relationship, the content of the corresponding bottom leaf node is determined in the thinking decision tree. The correspondence between the top task node and the middle perception node is obtained, and the content of the bottom leaf node is determined based on the correspondence.

7. The robot vision recognition, decision-making, and control method based on deep learning according to claim 1, characterized in that: The methods for constructing decision adjustment recommendations include: The execution bottleneck score of the bottom leaf node is initialized and assigned to detect performance degradation, execution failure or response delay in the task execution process in the bottom leaf node of the thinking decision tree, thereby identifying bottlenecks or potential error sources. The bottom leaf nodes are sorted according to their execution bottleneck scores. This sorting is based on the execution bottleneck scores of each leaf node, and the ranking results determine the priority of the optimization process. Nodes with higher scores will be optimized first. Based on the sorting results, adjustment suggestions are constructed for the corresponding bottom-level leaf nodes.

8. The robot vision recognition, decision-making, and control method based on deep learning according to claim 7, characterized in that: The methods for initializing and assigning values ​​to the bottom-level leaf nodes include: Define the execution bottleneck evaluation function: ; Used to represent the bottleneck score of the nth bottom leaf node; Used to represent the task execution error rate, it reflects the failure rate of service robots when performing tasks; specifically, it represents the ratio of the number of incorrect identifications to the total number of attempts. ; Task completion delay represents the time interval between the service robot receiving the instruction and starting task execution. It is obtained by monitoring the timestamps of task execution and recording the time difference between instruction input and task start-up by the service robot. ; Resource utilization rate represents the consumption of resources within the service robot during task execution. It is obtained by monitoring the resource usage of the service robot's internal hardware. ; These are the task weight coefficient, resource constraint coefficient, and environmental adaptability coefficient, respectively.

9. A robot vision recognition, decision-making, and control method based on deep learning according to claim 8, characterized in that: The number of urgent tasks indicates tasks that have a time limit, while the number of complex tasks refers to tasks that have more than one command to execute. The available resources are calculated through real-time collection and analysis of service robot monitoring data, and the resource constraint coefficient is also considered. A decrease indicates sufficient resources, while a decrease indicates higher resource utilization. An increase indicates that resource allocation needs to be optimized when performing the task; The number of features within an image is obtained through the service robot's vision system.

Images