Service robot module matching method based on fault deduction and related device

Abstract

The application is suitable for the technical field of service robots, and provides a service robot module matching method based on fault deduction and related devices. The method comprises the following steps: performing semantic matching on task description information of a service robot and module function description information of multiple robot modules to obtain multiple robot module combinations; performing fault deduction on each robot module combination according to the behavior correlation between the robot modules in each robot module combination to obtain an execution prediction result corresponding to each robot module combination respectively; comparing the execution prediction results corresponding to each robot module combination respectively, and taking a robot module combination to which an optimal execution prediction result belongs as a component structure of the service robot. The method can reduce the problems of module cooperation abnormality and task execution abnormality of the finally selected robot module combination in an actual task process.

Description

Technical Field

[0001] This invention relates to the field of service robot technology, and in particular to a service robot module matching method and related apparatus based on fault inference. Background Technology

[0002] In the field of service robot technology, there is a need to match and combine different robot modules to achieve corresponding task functions. One relevant service robot module matching method achieves module combination by matching task requirements with module functions. However, this method has the drawbacks of only performing static matching of module functions and lacking fault propagation correlation analysis between modules. This can lead to problems such as abnormal module coordination and task execution during actual tasks. Summary of the Invention

[0003] This invention provides a service robot module matching method, apparatus, equipment, medium, and program product based on fault inference, to solve the problem that selected robot module combinations are prone to module coordination abnormalities and task execution abnormalities during actual tasks.

[0004] In a first aspect, embodiments of this application provide a service robot module matching method based on fault inference, including: Semantic matching is performed between the task description information of the service robot and the module function description information of multiple robot modules to obtain a combination of multiple robot modules; Based on the behavioral correlation between each robot module in each robot module combination, fault inference is performed for each robot module combination to obtain the execution prediction results corresponding to each robot module combination. The execution prediction results corresponding to each robot module combination are compared, and the robot module combination to which the best execution prediction result belongs is taken as the composition structure of the service robot.

[0005] Secondly, embodiments of this application provide a service robot module matching device based on fault inference, comprising: The matching module is used to semantically match the task description information of the service robot with the module function description information of multiple robot modules to obtain a combination of multiple robot modules. The prediction module is used to perform fault inference for each robot module combination based on the behavioral correlation between each robot module combination, and obtain the execution prediction result corresponding to each robot module combination. A selected module is used to compare the execution prediction results corresponding to each robot module combination, and the robot module combination to which the best execution prediction result belongs is used as the composition structure of the service robot.

[0006] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the above-described service robot module matching method based on fault deduction.

[0007] Fourthly, embodiments of this application provide a readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described service robot module matching method based on fault deduction.

[0008] Fifthly, embodiments of this application provide a computer program product, which includes a computer program that, when executed by a processor, enables the implementation of the steps of the above-described service robot module matching method based on fault deduction.

[0009] In one solution provided by the aforementioned service robot module matching method, apparatus, device, medium, and program product based on fault inference, firstly, based on semantic matching between task description information and module function description information, multiple robot module combinations suitable for the current task content and requirements are determined; secondly, fault inference is performed based on the behavioral correlation between robot modules in the same robot module combination, thereby adaptively determining the execution prediction results of different robot module combinations in the corresponding task process under fault occurrence scenarios; thirdly, the execution prediction results corresponding to different robot module combinations are compared, thereby efficiently determining the robot module combination to which the optimal execution prediction result belongs, as the composition structure of the service robot; based on this, in the entire technical solution, on the basis of obtaining multiple robot module combinations based on semantic matching, fault inference is further performed based on the behavioral correlation between robot modules to select the final robot module combination, thereby reducing the problems of module coordination anomalies and task execution anomalies in the actual task process of the finally selected robot module combination.

[0010] In other words, after matching the task requirements with the module functions, the matching robot module combination is not directly used as the final structure. Instead, the behavioral relationships between the robot modules during the task are further combined to analyze the task execution process of different robot module combinations under the influence of faults. Thus, among multiple robot module combinations that can perform the same task, the more stable robot module combination in the corresponding task execution process under the influence of faults is further selected, so that the final determined robot module combination takes into account both the task execution requirements and the influence of faults. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of a service robot module matching system based on fault inference in one embodiment of the present invention; Figure 2 This is a flowchart illustrating a service robot module matching method based on fault inference in one embodiment of the present invention; Figure 3 This is a schematic diagram of a graph search structure in a robot module knowledge base according to an embodiment of the present invention; Figure 4 yes Figure 2 A schematic diagram of the implementation process of step S10; Figure 5 yes Figure 2 A schematic diagram of the implementation process of step S20; Figure 6 This is a schematic diagram of a fault impact path in one embodiment of the present invention; Figure 7 yes Figure 1 A structural diagram of a mid-terminal device; Figure 8 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. It should also be understood that, as used in this specification and the appended claims, the term "and / or" refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0015] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0016] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0017] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0018] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0019] In the field of service robot technology, the matching and combination of different robot modules is involved to achieve corresponding task functions. Related service robot module matching methods typically determine the appropriate robot module combinations by performing semantic matching on the correspondence between task requirements and module functions.

[0020] However, this approach primarily performs static matching based on the functional descriptions of each robot module, lacking correlation analysis of the behavioral dependencies among multiple robot modules during actual tasks. Therefore, when any robot module malfunctions, exhibits abnormal actions, or displays abnormal data, the impact can easily propagate to other robot modules along the data interaction, resource consumption, and action response processes between modules. Because existing methods lack the ability to extrapolate and analyze this abnormal propagation process, the generated robot module combination, while meeting functional requirements, is prone to problems such as module coordination interruptions, task execution conflicts, and abnormal task output during actual task execution.

[0021] To address the aforementioned issues, this application proposes a service robot module matching method, apparatus, device, medium, and program product based on fault inference. By semantically matching the task description information of the service robot with the module function description information of multiple robot modules, multiple robot module combinations are obtained. Based on the behavioral correlations between the robot modules in each combination, fault inference is performed on each robot module combination to obtain the execution prediction result corresponding to each combination. The execution prediction results corresponding to each robot module combination are compared, and the robot module combination with the optimal execution prediction result is taken as the composition structure of the service robot. Based on this, the problems of module coordination anomalies and task execution anomalies in the actual task process of the finally selected robot module combination can be reduced.

[0022] The service robot module matching method based on fault inference provided in this invention can be applied to, for example... Figure 1 The service robot module matching system based on fault inference shown includes a service robot module matching device based on fault inference, as well as a server and terminal equipment used by the device. The terminal equipment communicates with the server through a network or bus.

[0023] In practical applications, the server or terminal device receives task description information for the service robot from the user and sends it to the device. The device performs semantic matching between the task description information of the service robot and the module function description information of multiple robot modules to obtain multiple robot module combinations. Then, based on the behavioral correlation between the robot modules in each robot module combination, the device performs fault inference for each robot module combination to obtain the execution prediction result corresponding to each robot module combination. The device then compares the execution prediction results corresponding to each robot module combination and takes the robot module combination with the best execution prediction result as the composition structure of the service robot. Finally, the device returns the composition structure of the service robot to the server or terminal device.

[0024] The device can be a terminal device, which refers to a device that corresponds to the server and provides local services to customers. This terminal device includes, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices.

[0025] The device can also be a server, which can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0026] In one embodiment, such as Figure 2 As shown, a service robot module matching method based on fault inference is provided, which is then applied to... Figure 1 Taking the server in the example, the following steps are included: S10: Semantically match the task description information of the service robot with the module function description information of multiple robot modules to obtain a combination of multiple robot modules.

[0027] The task description information of the service robot indicates the task content and requirements that the service robot needs to perform, including the task name, task process, task execution object, and task execution order.

[0028] Among them, the robot module represents an independent functional unit in the service robot, such as a mobile module, a voice interaction module, a grasping module, or an image recognition module; the module function description information of the robot module indicates the functional processing content and functional execution scope of the corresponding robot module, which includes information such as the function name, functional processing content, and functional input and output content of the corresponding robot module.

[0029] For example, the task description information of the service robot and the module function description information of multiple robot modules are semantically parsed to determine the task semantic content in the task description information and the function semantic content in the function description information of each module. Semantic matching is then performed based on the correspondence between the task semantic content and the function semantic content to determine multiple robot module combinations that can independently complete the corresponding task content and task requirements. That is, the same task description information can correspond to multiple robot module combinations that can independently execute the corresponding task, and there are corresponding functional cooperation contents between the robot modules in each robot module combination during the task execution process.

[0030] Optionally, the multiple robot modules are stored in a preset robot module knowledge base. The robot modules in the preset robot module knowledge base are associated and organized through corresponding module function description information to form a corresponding graph search structure. In the graph search structure, each robot module is a corresponding graph node. Under the same functional applicable environment, different graph nodes form corresponding node connection relationships based on the functional processing content and functional execution actions in the corresponding module function description information.

[0031] The robot module knowledge base is a data management library used to store multiple robot modules and their corresponding functional descriptions. It can be constructed using relational databases, graph databases, or vector databases. Figure 3 This diagram illustrates a graph search structure in a robot module knowledge base. In this knowledge base, the recognition module, the movement module, and the voice interaction module are stored as corresponding graph nodes. The module function description information for the recognition module includes target recognition content and recognition actions; the module function description information for the movement module includes path movement content and turning actions; and the module function description information for the voice interaction module includes broadcast content and vocalization actions. If all the above robot modules correspond to the functional application environment of an indoor corridor area, and the target recognition content in the recognition module can provide input basis for the path movement content in the movement module, while the path movement content in the movement module can provide positional basis for the broadcast content in the voice interaction module, then corresponding node connection relationships are formed between the corresponding graph nodes, i.e., module association relationships within the same scene, thus forming the corresponding graph search structure.

[0032] S20: Based on the behavioral correlation between each robot module in each robot module combination, perform fault inference for each robot module combination to obtain the execution prediction results corresponding to each robot module combination.

[0033] For example, for each robot module combination, the module function description information corresponding to each robot module in the corresponding robot module combination is read, and the behavioral correlation between each robot module is determined according to the execution content of different robot modules in the corresponding task process. Specifically, the behavioral correlation is used to represent the mutual influence between different robot modules in the task execution process, which includes the data interaction, resource consumption and action response between different robot modules.

[0034] Since the robot modules in the same robot module assembly do not operate independently, but rather coordinately according to the corresponding task process, when any robot module fails, the corresponding fault will be transmitted to other robot modules along the behavioral correlations between different robot modules. Based on this, according to the corresponding behavioral correlations between robot modules, the fault transmission process after any robot module fails is deduced to determine the interruption, delay, and abnormal situation caused by the corresponding fault to the task execution process of other robot modules, thereby obtaining the execution prediction result that characterizes the task execution status when the corresponding robot module assembly fails in the corresponding task process.

[0035] S30: Compare the execution prediction results corresponding to each robot module combination, and take the robot module combination with the best execution prediction result as the composition structure of the service robot.

[0036] For example, the execution prediction results corresponding to different robot module combinations are compared to determine the execution differences between the different robot module combinations during task execution. Then, based on the execution differences between the different execution prediction results, the optimal execution prediction result is determined, which represents the best execution prediction result for the corresponding task execution among multiple robot module combinations. Finally, the robot module combination corresponding to the optimal execution prediction result is determined as the composition structure of the service robot under the current task content and requirements, thereby enabling the service robot to complete the task execution process according to the corresponding robot module combination.

[0037] For example, for the same delivery task, the first robot module combination includes a mobility module, an obstacle avoidance module, and a voice interaction module, while the second robot module combination includes a mobility module, a vision recognition module, and a voice interaction module. During the corresponding task, if the obstacle avoidance module in the first robot module combination malfunctions, the malfunction only affects the path adjustment process; however, if the vision recognition module in the second robot module combination malfunctions, the malfunction affects both target recognition and path determination. Based on this, by comparing the execution prediction results corresponding to the two robot module combinations, it is determined that the task with the first robot module combination has fewer abnormal situations, i.e., the task execution is optimal, and the corresponding execution prediction result is determined as the optimal execution prediction result.

[0038] In this embodiment, in step S10, based on the semantic matching between task description information and module function description information, multiple robot module combinations suitable for the current task content and requirements are determined. In step S20, fault inference is performed based on the behavioral correlation between robot modules in the same robot module combination, thereby adaptively determining the execution prediction results of different robot module combinations in the corresponding task process under fault occurrence scenarios. In step S30, the execution prediction results corresponding to different robot module combinations are compared, thereby efficiently determining the robot module combination to which the optimal execution prediction result belongs, as the composition structure of the service robot. Based on this, in the entire technical solution, on the basis of obtaining multiple robot module combinations based on semantic matching, fault inference is further performed based on the behavioral correlation between robot modules to select the final robot module combination, thereby reducing the problems of module collaboration anomalies and task execution anomalies in the actual task process of the finally selected robot module combination.

[0039] In one embodiment, such as Figure 4 As shown, step S10 involves semantically matching the task description information of the service robot with the module function description information of multiple robot modules to obtain a combination of multiple robot modules. This specifically includes the following steps: S11: Perform semantic parsing on the task description information to obtain task semantic features that include task objectives, task environment, and task behavior.

[0040] Among them, the task objective refers to the task processing content and task completion content of the service robot, such as item delivery, target recognition or voice interaction; the task environment refers to the task scenario and environmental conditions during task execution, such as indoor environment, corridor area or counter area; the task behavior refers to the actions and processes performed by the service robot during task execution, such as moving, grasping, recognizing or broadcasting.

[0041] S12: Perform semantic parsing on the module function description information of each robot module to obtain the functional semantic content of each robot module, which includes the function processing content, the applicable environment of the function, and the function execution action.

[0042] Among them, the function processing content represents the task processing logic and processing results corresponding to the robot module, such as determining the location of the item, recognizing the target status, or generating broadcast content; the function applicable environment represents the task environment applicable to the robot module, such as indoor environment, corridor area, or counter area; the function execution action represents the action execution method corresponding to the robot module, such as moving, turning, grasping, or making sounds.

[0043] S13: Match the task semantic content with the functional semantic content corresponding to each robot module, and combine the matched robot modules according to the task semantic content to obtain multiple robot module combinations, where each robot module combination independently corresponds to the task semantic content.

[0044] For example, semantic matching is performed based on the correspondence between the task objective, task environment, and task behavior in the task semantic content and the functional processing content, functional applicable environment, and functional execution actions in the functional semantic content to identify robot modules that have a semantic correspondence with the task semantic content. Then, based on the corresponding functional processing content and functional execution actions between different robot modules, multiple matched robot modules are combined to form corresponding robot module combinations. Since different robot modules correspond to different combination methods, the same task semantic content can correspond to multiple robot module combinations, and each robot module combination independently corresponds to its corresponding task semantic content. That is, the multiple robot modules in a corresponding robot module combination can jointly complete the task objective, task environment, and task behavior in the corresponding task semantic content.

[0045] For example, for the semantic content of "indoor item delivery," the task objective is item delivery, the task environment is an indoor corridor area, and the task behaviors include movement, recognition, and broadcasting. Based on this, a movement module with path movement capability, a recognition module with target recognition capability, and a voice interaction module with voice broadcasting capability can be combined to form a first robot module combination; simultaneously, a movement module with path movement capability, an obstacle avoidance module with obstacle avoidance capability, and a voice interaction module with voice broadcasting capability can be combined to form a second robot module combination, thus obtaining multiple robot module combinations each corresponding to the same task semantic content.

[0046] In this embodiment, in step S11, task semantic content, including task objectives, task environment, and task behavior, is parsed from the task description information, thereby converting the task content in the task description information into matchable semantic content. In step S12, functional semantic content, including functional processing content, functional applicable environment, and functional execution actions, is parsed from the module function description information of each robot module, thereby converting the functional scope of each robot module into matchable semantic content. In step S13, matching and combination are performed based on the correspondence between task semantic content and functional semantic content, thereby obtaining multiple robot module combinations that independently correspond to task semantic content. Based on this, in the entire technical solution, multiple robot module combinations capable of performing the same task can be formed around the semantic correspondence between task content and module function.

[0047] In one embodiment, such as Figure 5 As shown, in step S20, based on the behavioral correlation between the robot modules in each robot module combination, fault inference is performed on each robot module combination to obtain the execution prediction result corresponding to each robot module combination. Specifically, this includes the following steps: S21: Based on the module function description information of each robot module in each robot module combination, determine the behavioral dependencies of each robot module in the process of executing the task corresponding to the task description information.

[0048] For example, since the robot modules in the same robot module assembly do not execute independently, but rather coordinate their execution around the task objectives, task environment, and task behaviors described in the same task description information, there are corresponding behavioral relationships between different robot modules. Based on this, the module function description information corresponding to each robot module in the same robot module assembly is read, and the function processing content, function execution actions, and data input / output content in each module's function description information are extracted. Then, based on the correspondence between the function processing content, data input / output content, and function execution actions of different robot modules, the corresponding behavioral dependencies between each robot module are determined. Here, behavioral dependencies represent the mutual dependence between different robot modules during task execution. The corresponding behavioral dependencies between different robot modules are then correlated and organized to determine the fault propagation direction and fault propagation order between different robot modules.

[0049] Optionally, behavioral dependencies include resource dependencies, data interaction dependencies, and action response dependencies, wherein: resource dependencies represent the association of different robot modules occupying the same operating resources; data interaction dependencies represent the correspondence between data input and output between different robot modules; and action response dependencies represent the response association between the execution action of any robot module and the execution action of another robot module.

[0050] For example, in a robot module combination corresponding to an indoor delivery task: when the mobile module and the recognition module simultaneously call the same positioning or communication resource, it indicates that there is a resource dependency relationship between the corresponding mobile module and the recognition module; when the recognition module sends the target recognition result to the mobile module, and the mobile module adjusts its movement path according to the target recognition result, it indicates that there is a data interaction relationship between the corresponding recognition module and the mobile module; and when the obstacle avoidance module detects an obstacle ahead and triggers a deceleration action, and the mobile module adjusts its movement action or stops its movement action accordingly, it indicates that there is an action response relationship between the obstacle avoidance module and the mobile module.

[0051] S22: Based on behavioral dependencies, determine the failure impact path of any robot module on other robot modules in the same robot module combination when a failure occurs.

[0052] For example, when any robot module malfunctions, if there is a resource dependency relationship between other robot modules and the corresponding robot module, it is determined that the corresponding malfunction affects other robot modules through the corresponding operating resources; if there is a data interaction relationship between other robot modules and the corresponding robot module, it is determined that the corresponding malfunction affects other robot modules through the data input and output process; if there is an action response relationship between other robot modules and the corresponding robot module, it is determined that the corresponding malfunction affects other robot modules through changes in the execution of actions.

[0053] The fault originates from the robot module that initially malfunctions, and other affected robot modules are identified based on their corresponding behavioral dependencies. Subsequently, based on the behavioral dependencies between the affected robot modules and other robot modules, subsequent affected robot modules are identified level by level, thus forming a fault propagation process. This creates a fault impact path, starting from the initially malfunctioning robot module and determined by the direction and order of fault propagation among different robot modules. Furthermore, within the same robot module combination, different robot modules are used as fault origins to determine their respective fault impact paths, resulting in multiple fault impact paths for the same robot module combination.

[0054] In other words, different robot modules within the same robot module assembly correspond to different functional processing contents, functional execution actions, and behavioral dependencies. Therefore, when different robot modules fail, the direction, scope, and affected robot modules of the fault transmission within the robot module assembly will differ. For example, a fault in one robot module may only affect the data input / output process, while a fault in another robot module may affect both resource usage and action response processes. Based on this, it is necessary to use different robot modules as the fault starting point and determine the fault impact paths corresponding to each robot module, thereby obtaining the fault impact of the same robot module assembly under different fault conditions.

[0055] For example, Figure 6A schematic diagram of a fault impact path is shown, in which module A (environmental perception module) is taken as the fault starting point. Based on the corresponding behavioral dependencies between different robot modules, the transmission process of the fault between module B (obstacle detection module), module C (obstacle avoidance decision module), module D (voice broadcasting module), and module E (data recording module) is represented step by step. Specifically, module A and module B are connected through data interaction, with module A providing environmental data (such as obstacle information, location, etc.) to module B. Therefore, when module A initially malfunctions during the environmental information perception and acquisition process, abnormal environmental data acquisition or data loss may occur. The corresponding fault will further affect the obstacle detection and recognition process in module B, thus forming a fault impact path segment from module A to module B.

[0056] Furthermore, module B and module C are connected through action response relationships, with module B providing obstacle information (such as the location, distance, and type of obstacles) to module C. Therefore, when module B is affected in the obstacle detection and recognition process, there may be problems with missed or false detection of obstacles. The corresponding fault will further affect the obstacle avoidance decision-making and strategy selection process in module C, thus forming a fault impact path segment from module B to module C.

[0057] Furthermore, module C and module D are connected through action response relationships, with module C providing obstacle avoidance suggestions (such as avoidance direction and speed) to module D. Therefore, when module C is affected in the obstacle avoidance decision-making and strategy selection process, obstacle avoidance decision errors or generation delays may occur. The corresponding faults will further affect the voice broadcasting and prompting process in module D, thus forming a fault impact path segment from module C to module D.

[0058] Furthermore, module D and module E are connected through data interaction, with module D providing broadcast content data (such as prompts, timestamps, etc.) to module E. Therefore, when module D is affected in the voice broadcast and prompt process, there may be problems with the broadcast content being incorrect or missing. The corresponding fault will further affect the operation data recording and storage process in module E, thus forming a fault impact path segment from module D to module E.

[0059] Furthermore, module C and module E are connected through data interaction, with module C providing module E with operational status data (such as speed, position, obstacle avoidance status, etc.). Therefore, when module C is affected in the obstacle avoidance decision-making and strategy selection process, the corresponding fault will further affect the operational data recording and storage process in module E, thus forming a fault impact path segment from module C to module E.

[0060] Based on this, the above-mentioned fault impact path segments are integrated according to the module arrangement order, thereby forming a fault impact path that takes module A as the fault starting point and represents the transmission process of the fault between modules B, C, D, and E level by level.

[0061] S23: Based on multiple fault impact paths corresponding to the same robot module combination, jointly determine the execution prediction result of the corresponding robot module combination in the corresponding task process.

[0062] For example, within the same robot module combination, the fault origin, affected robot modules, and corresponding transmission order in each fault impact path are extracted to determine the fault impact of each fault impact path when it is established individually. The fault origin corresponds to the robot module that initially experiences the fault, the affected robot modules correspond to those whose behavior dependencies are affected, and the transmission order corresponds to the order of influence from the fault origin to the affected robot modules within the same fault impact path. For instance, if the recognition module malfunctions, it first affects the path processing of the motion module and then the broadcast content of the voice interaction module. Therefore, the transmission order within this fault impact path corresponds to the recognition module, the motion module, and the voice interaction module in that order.

[0063] Based on this, multiple fault impact paths are analyzed independently to determine the affected scope and content of each fault impact path. The affected scope refers to the robot modules affected in the corresponding fault impact path, and the affected content refers to the affected functional processing content and functional execution actions in the corresponding robot modules. For example, if a fault impact path corresponds to the mobile module and the voice interaction module, the corresponding affected content includes the path planning content and forward turning actions in the mobile module, and the voice broadcast content and broadcast output actions in the voice interaction module.

[0064] Furthermore, while maintaining the independence of each fault impact path, the same affected robot modules and the same affected content in multiple fault impact paths are summarized, and the overall task execution of the robot module combination in the corresponding task process is determined based on the summarized affected scope and affected content, which serves as the execution prediction result; wherein, the execution prediction result represents the predicted task execution of the corresponding robot module combination after considering the affected scope and affected content when multiple fault impact paths are respectively valid.

[0065] Optionally, the process of determining the above-mentioned fault impact path can be described as follows: based on the behavioral dependency relationship, taking the target robot module as the fault starting point, determining the corresponding affected robot module and the corresponding affected content in the same robot module combination, and then determining the next affected robot module and the corresponding affected content level by level to form the corresponding fault impact path; taking each robot module as the fault starting point in the same robot module combination, obtaining the fault impact path corresponding to each fault starting point.

[0066] Furthermore, in determining the fault impact path, the path can be merged or separated based on the consistency of influence between different affected contents. For example, when the recognition module malfunctions, it simultaneously affects the path planning content in the motion module and the broadcast content in the voice interaction module, and all corresponding effects originate from the same target recognition content anomaly. In this case, the corresponding impact process is determined as the same fault impact path. However, when the motion module malfunctions, some impacts correspond to abnormal motion actions, while others correspond to abnormal path planning content, affecting different robot modules respectively. In this case, the corresponding impact process is split into different fault impact paths. Based on this, the transmission structure in the fault impact path can be further adjusted according to the correspondence between different affected contents.

[0067] In this embodiment, in step S21, behavioral dependencies are determined based on the module function description information of each robot module, thereby clarifying the basis for mutual influence of each robot module in the corresponding task process; in step S22, the fault impact path corresponding to the failure of any robot module is determined based on the corresponding behavioral dependencies, thereby clarifying the impact transmission process of the fault in the same robot module combination; in step S23, the execution prediction result is jointly determined based on multiple fault impact paths corresponding to the same robot module combination, thereby predicting the task execution status of the robot module combination in the corresponding task process; based on this, in the entire technical solution, after module function matching, execution prediction can be further combined with behavioral dependencies and fault impact paths, so that the determination process of robot module combination is no longer limited to the static function matching stage.

[0068] In one embodiment, step S23, which involves jointly determining the execution prediction result of the corresponding robot module combination in the corresponding task process based on multiple fault impact paths corresponding to the same robot module combination, specifically includes the following steps: S230: In multiple fault impact paths corresponding to the same robot module combination, the inference weights corresponding to each fault impact path are obtained according to the degree of correlation between each fault impact path, so as to jointly determine the execution prediction results of the corresponding robot module combination in the corresponding task process.

[0069] For example, the degree of correlation between different fault impact paths is determined based on the different affected ranges and affected contents corresponding to different fault impact paths; wherein, when the same affected robot module or the same affected contents exist in different fault impact paths, it is determined that the corresponding fault impact paths have a high degree of correlation; while when different fault impact paths correspond to different affected robot modules and affected contents, it is determined that the corresponding fault impact paths have a low degree of correlation.

[0070] For example, when the affected area of ​​the first fault impact path includes the mobile module and the voice interaction module, and the corresponding affected content includes the path planning content and forward movement content in the mobile module, as well as the broadcast content and broadcast movement content in the voice interaction module; and the affected area of ​​the second fault impact path includes the mobile module, the recognition module, and the voice interaction module, and the corresponding affected content includes the path planning content and turning movement content in the mobile module, as well as the broadcast content and broadcast movement content in the voice interaction module, then since both fault impact paths include the mobile module and the voice interaction module, and both correspond to path planning content, forward movement content, broadcast content, and broadcast movement content, the correlation degree between the two fault impact paths is determined to be 0.85 based on the proportion of the same affected area and the same affected content to the total affected area and the total affected content.

[0071] For example, when the affected area of ​​the third fault impact path includes the grasping module and the affected content includes grasping control content and grasping action content, and the affected area of ​​the fourth fault impact path includes the obstacle avoidance module and the recognition module and the affected content includes obstacle recognition content, target recognition content and turning action content, then since there is no common affected area or common affected content between the two fault impact paths, the correlation between the two fault impact paths is determined to be 0.

[0072] Then, based on the degree of correlation between different fault impact paths, the inference weight corresponding to each fault impact path is determined. The inference weight characterizes the degree of influence of the corresponding fault impact path in the process of determining the prediction result. Specifically, if any fault impact path has many identical affected robot modules and identical affected content with other fault impact paths, then that fault impact path is determined to have a high inference weight; if any fault impact path has few identical affected robot modules and identical affected content with other fault impact paths, then that fault impact path is determined to have a low inference weight. For example, if the correlation degree between the first fault impact path and the second and third fault impact paths is 0.85 and 0.75 respectively, then based on their average correlation degree of 0.80, the inference weight of the first fault impact path is determined to be 0.80; if the correlation degree between the fourth fault impact path and the second and third fault impact paths is 0.20 and 0.30 respectively, then based on their average correlation degree of 0.25, the inference weight of the fourth fault impact path is determined to be 0.25.

[0073] Further, step S230, namely, jointly determining the execution prediction result of the corresponding robot module combination in the corresponding task process, includes one of the following steps: S231: Based on the inference weights corresponding to the characteristics of each path, the number of affected robot modules, the complexity of the affected content, and the path transmission length corresponding to each fault-affected path are weighted and accumulated to obtain the corresponding execution prediction results.

[0074] For example, the number of affected robot modules represents the affected scope in the corresponding fault impact path, the complexity of the affected content represents the types and quantities of content in the corresponding affected content, and the path transmission length represents the number of transmission levels of the corresponding fault between different robot modules. Based on this, according to the inference weights corresponding to each fault impact path, the number of affected robot modules, the complexity of the affected content, and the path transmission length are weighted and accumulated, where fault impact paths with higher inference weights correspond to larger impact values.

[0075] For example, when the inference weight corresponding to the first fault impact path is 0.80, the number of affected robot modules is 4, the complexity of the affected content is 8, and the path transmission length is 5 levels, the corresponding impact value is 13.6 according to "0.80×(4+8+5)"; while when the inference weight corresponding to the second fault impact path is 0.25, the number of affected robot modules is 2, the complexity of the affected content is 3, and the path transmission length is 2 levels, the corresponding impact value is 1.75 according to "0.25×(2+3+2)". Based on this, the impact value is calculated pairwise for each fault impact path, and then all the impact values ​​are accumulated to obtain the comprehensive impact value. The execution prediction result is determined based on this value to characterize the overall task execution of the corresponding robot module combination after being affected by a fault in the corresponding task. The larger the comprehensive impact value, the larger the affected range, the more complex the affected content, and the more fault transmission levels in the corresponding robot module combination, and the higher the degree of task execution abnormality of the corresponding robot module combination in the task process.

[0076] S232: Based on the inference weights corresponding to the characteristics of each path, overlap determination is performed on the same affected robot modules and the same affected content in different fault-affected paths, and the corresponding execution prediction results are obtained based on the results of the overlap determination.

[0077] For example, the affected robot modules in different fault impact paths are compared, and if the same affected robot module exists, the affected content in the corresponding robot module is further compared to determine whether the same affected robot module corresponds to the same affected content. If multiple fault impact paths correspond to the same affected robot module and correspond to the same or similar affected content, then the overlap of the affected robot module and the corresponding affected content is determined according to the inference weight of the corresponding fault impact path.

[0078] For example, both the first and second fault impact paths correspond to path planning and turning actions in the movement module, with corresponding inference weights of 0.80 and 0.65, respectively. The affected content of the first fault impact path includes path planning, turning actions, path movement content, and movement actions, while the affected content of the second fault impact path includes path planning, turning actions, obstacle avoidance content, and deceleration actions. Therefore, the number of identical affected content items in the two fault impact paths is 4, and the total number of affected content items is 8. Based on "4 / 8", the corresponding overlap ratio is determined to be 0.50, and the first overlap impact value of 0.3625 is obtained by "0.50×(0.80+0.65) / 2". Based on this, each fault impact path is calculated pairwise to obtain the corresponding overlapping impact value. Then, all overlapping impact values ​​are accumulated to obtain the comprehensive overlapping impact value, and the execution prediction result is determined accordingly. This value is used to characterize the overall task execution of the corresponding robot module combination after being affected by a fault during the corresponding task. The larger the comprehensive overlapping impact value, the higher the degree of concentrated impact of multiple fault impact paths on the same affected robot module and the same affected content, and the higher the probability of the corresponding robot module combination being affected during the corresponding task.

[0079] In this embodiment, in step S230, the inference weights corresponding to each fault impact path are determined according to the degree of correlation between different fault impact paths, thereby distinguishing the degree of influence of different fault impact paths in the process of determining the execution prediction results; in step S231, the number of affected robot modules, the complexity of affected content, and the path transmission length are weighted and accumulated according to the inference weights, thereby obtaining the comprehensive impact of the corresponding robot module combination in the task process; in step S232, the overlap of the same affected robot modules and the same affected content in different fault impact paths is determined according to the inference weights, thereby determining the repeated impact between different fault impact paths; based on this, in the entire technical solution, the overall task execution of the robot module combination in the task process can be adaptively analyzed by using the inference weights corresponding to different fault impact paths, so as to determine the corresponding execution prediction results and improve the effectiveness and diversity of result determination.

[0080] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0081] In one embodiment, a service robot module matching device based on fault inference is provided, which corresponds one-to-one with the service robot module matching method based on fault inference in the above embodiments. For example... Figure 7As shown, the service robot module matching device based on fault deduction includes a matching module 701, a prediction module 702, and a selection module 703. Detailed descriptions of each functional module are as follows: The matching module 701 is used to perform semantic matching between the task description information of the service robot and the module function description information of multiple robot modules to obtain a combination of multiple robot modules. The prediction module 702 is used to perform fault inference on each robot module combination based on the behavioral correlation between each robot module in each robot module combination, and obtain the execution prediction result corresponding to each robot module combination. Module 703 is selected to compare the execution prediction results corresponding to each robot module combination, and the robot module combination to which the best execution prediction result belongs is used as the composition structure of the service robot.

[0082] In one embodiment, the matching module 701 is further configured to: perform semantic parsing on the task description information to obtain task semantic content including task objectives, task environment, and task behavior; perform semantic parsing on the module function description information of each robot module to obtain functional semantic content corresponding to each robot module, including functional processing content, functional applicable environment, and functional execution actions; match the task semantic content with the functional semantic content corresponding to each robot module, and combine the matched robot modules according to the task semantic content to obtain multiple robot module combinations, wherein each robot module combination independently corresponds to the task semantic content.

[0083] In one embodiment, the prediction module 702 is further configured to: determine the behavioral dependencies of each robot module in the task process corresponding to the task description information based on the module function description information of each robot module in each robot module combination; determine the fault impact path of any robot module failure on other robot modules in the same robot module combination based on the behavioral dependencies; and jointly determine the execution prediction result of the corresponding robot module combination in the corresponding task process based on the multiple fault impact paths corresponding to the same robot module combination.

[0084] In one embodiment, the prediction module 702 is further configured to: based on the behavioral dependency relationship, take the target robot module as the fault starting point, determine the corresponding affected robot module and the corresponding affected content in the same robot module combination, and then determine the next affected robot module and the corresponding affected content step by step to form a corresponding fault impact path; and take each robot module as the fault starting point in the same robot module combination to obtain the fault impact path corresponding to each fault starting point.

[0085] In one embodiment, the behavioral dependency relationship includes resource dependency relationship, data interaction relationship and action response relationship, wherein: resource dependency relationship represents the association of different robot modules occupying the same operating resource; data interaction relationship represents the correspondence between data input and output between different robot modules; action response relationship represents the response association generated by the change of the execution action of any robot module to the execution action of another robot module.

[0086] In one embodiment, the prediction module 702 is further configured to: obtain the inference weights corresponding to each fault influence path based on the degree of correlation between each fault influence path in multiple fault influence paths corresponding to the same robot module combination, so as to jointly determine the execution prediction result of the corresponding robot module combination in the corresponding task process.

[0087] In one embodiment, the prediction module 702 is further configured to: weight and accumulate the number of affected robot modules, the complexity of the affected content, and the path transmission length corresponding to each fault-affected path according to the inference weights corresponding to each path feature, and obtain the corresponding execution prediction result; and determine the overlap of the same affected robot modules and the same affected content in different fault-affected paths according to the inference weights corresponding to each path feature, and obtain the corresponding execution prediction result based on the result of the overlap determination.

[0088] In one embodiment, multiple robot modules are stored in a preset robot module knowledge base. The robot modules in the preset robot module knowledge base are associated and organized through corresponding module function description information to form a corresponding graph search structure.

[0089] In one embodiment, in the graph search structure, each robot module serves as a corresponding graph node. Under the same functional applicable environment, different graph nodes form corresponding node connection relationships based on the function execution content and function execution actions in the corresponding module's functional description information.

[0090] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0091] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0092] This application also provides a computer device, such as... Figure 8 As shown, the computer device includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor. When the processor executes the computer program, it implements the steps in any of the above method embodiments, or when the processor executes the computer program, it implements the functions of each module / unit in the above device embodiments.

[0093] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the computer device.

[0094] Those skilled in the art will understand that Figure 8 The computer device described is merely an example and does not constitute a limitation on the computer device. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device may also include input / output devices, network access devices, buses, etc.

[0095] The aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0096] The memory can be an internal storage unit of the computer device, such as a hard drive or RAM. The memory can also be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory can include both internal and external storage units of the computer device.

[0097] This application also provides a readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0098] This application provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0099] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0100] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0101] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0102] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0103] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0104] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A service robot module matching method based on fault inference, characterized in that, include: Semantic matching is performed between the task description information of the service robot and the module function description information of multiple robot modules to obtain a combination of multiple robot modules; Based on the behavioral correlation between each robot module in each robot module combination, fault inference is performed for each robot module combination to obtain the execution prediction results corresponding to each robot module combination. The execution prediction results corresponding to each robot module combination are compared, and the robot module combination to which the best execution prediction result belongs is taken as the composition structure of the service robot.

2. The method as described in claim 1, characterized in that, The process involves semantically matching the task description information of the service robot with the module function description information of multiple robot modules to obtain a combination of multiple robot modules, including: Semantic parsing is performed on the task description information to obtain task semantic content that includes task objectives, task environment, and task behavior; Semantic parsing is performed on the module function description information of each robot module to obtain the functional semantic content of each robot module, which includes the function processing content, the applicable environment of the function, and the function execution action. The task semantic content is matched with the functional semantic content corresponding to each robot module, and the matched robot modules are combined according to the task semantic content to obtain multiple robot module combinations, wherein each robot module combination independently corresponds to the task semantic content.

3. The method as described in claim 1, characterized in that, The step involves performing fault deduction on each robot module combination based on the behavioral correlations between the robot modules within each combination, to obtain the execution prediction results corresponding to each robot module combination, including: Based on the module function description information of each robot module in each robot module combination, determine the behavioral dependencies of each robot module in the process of executing the task corresponding to the task description information; Based on the behavioral dependencies, determine the fault impact path of any robot module failure on other robot modules in the same robot module combination; Based on multiple fault impact paths corresponding to the same robot module combination, the execution prediction results of the corresponding robot module combination in the corresponding task process are jointly determined.

4. The method as described in claim 3, characterized in that, The step of determining the fault impact path of any robot module malfunctioning on other robot modules in the same robot module assembly based on the behavioral dependency relationship includes: Based on the behavioral dependencies, the target robot module is taken as the fault starting point. The corresponding affected robot module and its corresponding affected content are determined in the same robot module combination. Then, the next affected robot module and its corresponding affected content are determined step by step to form the corresponding fault impact path. In the same robot module combination, each robot module is taken as the fault starting point to obtain the fault impact path corresponding to each fault starting point.

5. The method as described in claim 3 or 4, characterized in that, The behavioral dependencies include resource dependencies, data interaction dependencies, and action response dependencies, wherein: The resource dependency relationship represents the association of different robot modules occupying the same operating resource; The data interaction relationship represents the data input and output correspondence between different robot modules; The action response relationship represents the response association between a change in the action performed by one robot module and the action performed by another robot module.

6. The method as described in claim 3, characterized in that, The step of jointly determining the execution prediction result of the corresponding robot module combination in the corresponding task process based on multiple fault impact paths corresponding to the same robot module combination includes: In multiple fault impact paths corresponding to the same robot module combination, the inference weights corresponding to each fault impact path are obtained based on the degree of correlation between each fault impact path, so as to jointly determine the execution prediction results of the corresponding robot module combination in the corresponding task process.

7. The method as described in claim 6, characterized in that, The joint determination of the execution prediction result of the corresponding robot module combination in the corresponding task process includes one of the following steps: Based on the inference weights corresponding to the characteristics of each path, the number of affected robot modules, the complexity of the affected content, and the path transmission length of each fault-affected path are weighted and accumulated to obtain the corresponding execution prediction results. Based on the inference weights corresponding to the characteristics of each path, the overlap of the same affected robot modules and the same affected content in different fault-affected paths is determined, and the corresponding execution prediction results are obtained based on the results of the overlap determination.

8. The method as described in claim 1, characterized in that, The multiple robot modules are stored in a preset robot module knowledge base. The robot modules in the preset robot module knowledge base are associated and organized through corresponding module function description information to form a corresponding graph search structure.

9. The method as described in claim 8, characterized in that, In the graph search structure, each robot module serves as a corresponding graph node. Under the same functional applicable environment, different graph nodes form corresponding node connection relationships based on the function execution content and function execution actions in the corresponding module's functional description information.

10. A service robot module matching device based on fault inference, characterized in that, include: The matching module is used to semantically match the task description information of the service robot with the module function description information of multiple robot modules to obtain a combination of multiple robot modules. The prediction module is used to perform fault inference for each robot module combination based on the behavioral correlation between each robot module combination, and obtain the execution prediction result corresponding to each robot module combination. A selected module is used to compare the execution prediction results corresponding to each robot module combination, and the robot module combination to which the best execution prediction result belongs is used as the composition structure of the service robot.

11. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the service robot module matching method based on fault inference as described in any one of claims 1 to 9.

12. A readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the service robot module matching method based on fault inference as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, enables the implementation of the steps of the service robot module matching method based on fault inference as described in any one of claims 1 to 9.

Images