Flexible manufacturing process planning and executing method applied to humanoid robot and related equipment

Abstract

The invention discloses a flexible manufacturing process planning and execution method applied to a humanoid robot and related equipment, and the method comprises the steps: generating a standardized task description according to the visual element information, operator voice instruction information, work order text information and field state information of a flexible manufacturing production scene, searching in the process knowledge map to obtain an optimal standard process path, and generating a motion track of the humanoid robot; tool updating and station reconstruction are carried out according to the standardized task; and the humanoid robot is driven to execute technological operation according to the motion trail, and the working state of the humanoid robot is adjusted according to technological parameters and quality inspection parameters in the execution process. According to the embodiment of the invention, the method achieves the dynamic process planning and execution of the humanoid robot in a flexible manufacturing scene based on the multi-modal perception and process knowledge map technology, reduces the manual intervention, and improves the production efficiency and quality. The method can be widely applied to the technical field of automatic manufacturing.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automation manufacturing technology, and in particular to a flexible manufacturing process planning and execution method applied to humanoid robots and related equipment. BACKGROUND

[0002] With the gradual development of manufacturing industry towards flexibility and intelligence, production lines need to maintain high efficiency and stability in the mode of multi-variety, small batch and rapid switching. Although traditional industrial robots have the advantages of high speed and high repeatability, their programming and application environment are often relatively fixed, and it is difficult to quickly adapt to frequent changes in product models, tooling fixtures and processes. Humanoid robots have broad application prospects in flexible manufacturing due to their human-like motion structure and multi-modal perception capabilities. By combining voice, vision, touch and process data, humanoid robots can not only imitate workers to assemble, carry and detect, but also can be linked with manufacturing execution systems (MES) and production line control systems (PLC), thereby assuming more complex tasks in the process flow and providing support for flexible production and intelligent upgrading of workshops.

[0003] The existing humanoid robots or traditional collaborative robots still have some defects in the application of flexible manufacturing: first, manufacturing tasks usually have diversified and non-standardized characteristics, and robots have difficulty in correctly understanding complex task instructions such as work order changes or natural instructions of operators; second, traditional manufacturing process planning and execution methods often rely on fixed production processes and manual configuration, and when external conditions such as production environment, task demand or equipment state change, a large amount of manual re-planning of process flow and robot action path is required, which affects manufacturing efficiency; third, the existing system lacks dynamic scheduling capability, and when the part specifications change, the tooling is replaced or the environment fluctuates, the robot action often cannot be adjusted in real time, which is easy to cause deviation and rework.

[0004] In summary, the technical problems in the related art need to be improved. SUMMARY

[0005] The main purpose of the embodiments of the present application is to propose a flexible manufacturing process planning and execution method applied to humanoid robots and related equipment, based on multi-modal perception and process knowledge graph technology, dynamic process planning and execution of humanoid robots in flexible manufacturing scenarios are realized, manual intervention is reduced, and production efficiency and production quality are improved.

[0006] To achieve the above purpose, one aspect of the embodiments of the present application proposes a flexible manufacturing process planning and execution method applied to humanoid robots, the method comprising: generating a standardized task description according to visual element information, operator voice instruction information, work order text information and on-site state information of the flexible manufacturing production scene; retrieve an optimal standard process path from a process knowledge graph according to the standardized task description, and generate a motion trajectory of the humanoid robot according to the standard process path; control the humanoid robot to update a tool and reconstruct a station according to the standardized task description; drive the humanoid robot to perform a process operation according to the motion trajectory, acquire a process parameter and a quality inspection parameter in a process of performing the process operation, and adjust a working state of the humanoid robot according to the process parameter and the quality inspection parameter.

[0007] In some embodiments, the visual element information includes a workpiece type, a tooling mark, and a station state, the work order text information includes a part model, a batch number, and a delivery time requirement, and the scene state information includes process context information and environmental sensing parameters, the process context information includes station occupancy information, tooling positioning information, and tool life information, the standardized task description is generated according to the visual element information, the operator voice instruction information, the work order text information, and the scene state information of the flexible manufacturing production scene, and includes: acquire image information of the flexible manufacturing production scene through a visual recognition unit, and extract the visual element information according to the image information; acquire the operator voice instruction information through a voice recognition unit; acquire a work order text, analyze the work order text through a text analysis unit, and obtain the work order text information; supplement the visual element information, the operator voice instruction information, the work order text information, and the scene state information through a minimum follow-up mechanism; time synchronize and semantically fuse the supplemented visual element information, the operator voice instruction information, the work order text information, and the scene state information, and obtain the standardized task description.

[0008] In some embodiments, the optimal standard process path is retrieved from the process knowledge graph according to the standardized task description, and includes: acquire constraint condition information, retrieve a plurality of executable process paths from the process knowledge graph according to the standardized task description and the constraint condition information; evaluate each of the executable process paths according to a preset production index evaluation algorithm, and obtain the optimal standard process path.

[0009] In some embodiments, the motion trajectory of the humanoid robot is generated according to the standard process path, and includes: convert the standard process path into a directed acyclic graph; mapping each node in the directed acyclic graph into a plurality of operation instructions in a humanoid robot skill primitive library; generating a collaborative trajectory for the double arms, the torso and the base of the humanoid robot according to each operation instruction through a preset path planning algorithm, and performing feasibility verification on the collaborative trajectory through digital twin simulation; the collaborative trajectory that passes the feasibility verification is used as the motion trajectory.

[0010] In some embodiments, the controlling the humanoid robot to update tools and reconfigure workstations according to the standardized task description comprises: determining tool use requirements and workstation configuration requirements according to the standardized task description; determining a tool replacement plan according to the tool use requirements, and controlling the humanoid robot to take a target tool from a tool library according to the tool replacement plan; controlling the humanoid robot to move tooling tables and jigs and update workstation coordinate systems according to the workstation configuration requirements.

[0011] In some embodiments, the process parameters include mechanical signals, pose signals, temperature signals and acoustic signals, and the quality inspection parameters include assembly gaps, hole position accuracy, torque curves and weld appearance, and the adjusting the working state of the humanoid robot according to the process parameters and the quality inspection parameters comprises: comparing the process parameters with a preset process template to obtain process deviation indicators, and executing corresponding process control strategies according to the process deviation indicators; determining quality deviation indicators according to the quality inspection parameters, and executing corresponding quality control strategies according to the quality deviation indicators; performing safety detection on the working state of the humanoid robot to obtain safety detection results, and executing corresponding safety control strategies according to the safety detection results.

[0012] In some embodiments, the flexible manufacturing process planning and execution method further comprises: obtaining execution logs and quality inspection data during execution; updating the process knowledge graph according to the execution logs and the quality inspection data.

[0013] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application proposes a flexible manufacturing process planning and execution device applied to a humanoid robot, the device comprising: a task description generation module configured to generate a standardized task description according to visual element information, operator voice instruction information, work order text information and on-site state information of a flexible manufacturing production scene; The trajectory generation module is used to retrieve the optimal standard process path from the process knowledge graph based on the standardized task description, and generate the motion trajectory of the humanoid robot based on the standard process path. The tool update and workstation reconstruction module is used to control the humanoid robot to perform tool updates and workstation reconstruction according to the standardized task description. The execution and adjustment module is used to drive the humanoid robot to perform process operations according to the motion trajectory, obtain process parameters and quality inspection parameters during the execution process, and adjust the working state of the humanoid robot according to the process parameters and the quality inspection parameters.

[0014] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the methods described above.

[0015] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0016] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the methods described above.

[0017] The embodiments of this application include at least the following beneficial effects: This application provides a flexible manufacturing process planning and execution method and related equipment applied to humanoid robots. This solution generates a unified standardized task description by integrating visual element information, operator voice commands, work order text information, and on-site status information, realizing semantic fusion of multi-source heterogeneous information, improving the parsing efficiency of complex manufacturing tasks, and providing a foundation for subsequent automated process planning; based on the standardized task description, the optimal standard process path is retrieved in the process knowledge graph, and the corresponding humanoid robot motion trajectory is generated, ensuring that the process planning result is feasible and optimal; based on the standardized task description, the humanoid robot is automatically controlled to perform tool updates and workstation reconstruction, which can quickly adjust production conditions according to the current task, improve the adaptability of the manufacturing system when task requirements change, thereby improving the automation level and production efficiency of flexible manufacturing, and reducing manual intervention and maintenance costs; during process execution, by collecting process parameters and quality inspection parameters and performing dynamic comparison, the working status of the humanoid robot is adjusted in real time, which can effectively eliminate process deviations and quality fluctuations, and improve production quality and consistency. Attached Figure Description

[0018] Figure 1This is a flowchart illustrating the steps of a flexible manufacturing process planning and execution method for humanoid robots, as provided in an embodiment of this application. Figure 2 This is a schematic diagram of a flexible manufacturing process planning and execution device for humanoid robots provided in an embodiment of this application; Figure 3 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit the scope of this application.

[0021] The concept of the present invention will now be explained in conjunction with the background art.

[0022] The application of existing humanoid robots or traditional collaborative robots in flexible manufacturing still has several shortcomings: First, although robots can perceive through sensors, they lack a unified task parsing mechanism for multimodal fusion, making it difficult to correctly understand work order changes or natural instructions from operators; Second, process switching still relies on manual teaching or pre-programming, resulting in insufficient flexibility and production cycle delays; Third, existing systems lack dynamic programming capabilities, and robot movements often cannot be adjusted in real time when part specifications change, tooling is replaced, or the environment fluctuates, easily causing deviations and rework; Fourth, the lack of self-optimization and learning mechanisms prevents robots from accumulating experience and forming personalized process knowledge during execution.

[0023] Therefore, this application provides a flexible manufacturing process planning and execution method and related equipment for humanoid robots. By integrating technologies such as multimodal perception, process knowledge graph and dynamic process orchestration, the humanoid robot can achieve rapid process switching, adaptive execution and continuous optimization in a flexible manufacturing environment, thereby improving production efficiency, reducing human intervention, and ensuring safety and traceability.

[0024] This application provides a flexible manufacturing process planning and execution method for humanoid robots, relating to the field of information technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited thereto. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing a flexible manufacturing process planning and execution method for humanoid robots, but is not limited to the above forms.

[0025] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0026] Figure 1 This is an optional flowchart of a flexible manufacturing process planning and execution method for humanoid robots provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S101 to S104.

[0027] S101. Generate a standardized task description based on the visual element information, operator voice command information, work order text information, and on-site status information of the flexible manufacturing production scenario. In some embodiments, visual element information includes workpiece type, tooling markings, and workstation status; work order text information includes part model, batch number, and delivery time requirement; and on-site status information includes process context information and environmental sensor parameters. The process context information includes workstation occupancy information, tooling positioning information, and tool life information. A standardized task description is generated based on the visual element information, operator voice command information, work order text information, and on-site status information of the flexible manufacturing production scenario, including: S1011. Obtain image information of the flexible manufacturing production scene through the visual recognition unit, and extract visual element information based on the image information; S1012. Obtain operator voice command information through the voice recognition unit; S1013. Obtain the work order text. Parse the work order text using the text parsing unit to obtain the work order text information. S1014. Supplement visual element information, operator voice command information, work order text information, and on-site status information through the minimum questioning mechanism. S1015. Time synchronization and semantic fusion are performed on the supplemented visual element information, operator voice command information, work order text information, and on-site status information to obtain a standardized task description.

[0028] Specifically, in this embodiment, image information of the production scene can be acquired through visual recognition units such as industrial cameras, and then visual element information such as workpiece type, tooling markings, and workstation status can be extracted through a target detection model; the speech recognition unit uses speech-to-text and natural language understanding algorithms to convert operator commands into structured instructions; the text parsing unit can be implemented through a large language model or pattern matching, and after reading the MES work order, the text parsing unit extracts work order text information such as part model, batch number, and delivery time requirements; the on-site status information includes process context information such as workstation occupancy, tooling placement, and tool life, as well as environmental sensing parameters such as temperature and humidity. In this embodiment, all input data is timestamped and uniformly written into a message queue, and then semantically fused by the task parser to output a standardized task description including "target part - process requirements - tooling resources - constraints". To address the frequent ambiguities and incompleteness in flexible scenarios, when the resolution confidence or parameter completeness is insufficient, the robot uses voice prompts or screen displays to minimize follow-up questions (such as confirming only key slots: tool specifications, inspection sampling rate, or switching windows) to supplement executable information without interrupting the cycle.

[0029] S102. Based on the standardized task description, retrieve the optimal standard process path from the process knowledge graph, and generate the motion trajectory of the humanoid robot based on the standard process path. In some embodiments, retrieving the optimal standard process path from the process knowledge graph based on the standardized task description includes: S1021. Obtain constraint information. Based on the standardized task description and constraint information, retrieve several executable process paths from the process knowledge graph. S1022. Evaluate each executable process path according to the preset production index evaluation algorithm to obtain the optimal standard process path.

[0030] In some embodiments, generating the motion trajectory of the humanoid robot according to a standard process path includes: S1023. Convert the standard process path into a directed acyclic graph; S1024. Map each node in the directed acyclic graph to several operation instructions in the humanoid robot skill primitive library. S1025. Using a preset path planning algorithm, a collaborative trajectory is generated for the humanoid robot's arms, torso, and base based on each operation command, and the feasibility of the collaborative trajectory is verified through digital twin simulation. S1026. The cooperative trajectory that has passed the feasibility verification shall be used as the motion trajectory.

[0031] Specifically, in steps S1021-S1022, the process knowledge graph is implemented using a graph database. Its nodes represent parts, processes, tooling, cutting tools, parameter templates, and quality characteristics, while edges represent "pre- / post-dependencies," "substitution relationships," and "constraint bindings." Upon receiving a new task description, the inference engine retrieves relevant paths from the graph and calls the constraint compiler to transform constraint information such as SOP clauses, EHS safety specifications, and PLC interlock logic into a computable set of constraints. During candidate path generation, the inference engine automatically filters out branches that do not meet the conditions, outputting only feasible solutions that satisfy the process, quality, and safety requirements. If multiple feasible paths exist in the knowledge graph, they are sorted according to production indicators such as capacity, energy consumption, and safety margin, with the optimal comprehensive solution being output first as the standard process path.

[0032] In steps S1023-S1026, the process path obtained from the aforementioned reasoning is transformed into a directed acyclic graph (DAG), where each node contains the required processes, tooling resources, execution parameters, and quality inspection point information. Nodes are further mapped to operational instructions in the humanoid robot's skill primitive library, such as grasping, handling, alignment, assembly, welding, or inspection. The orchestrator invokes a path planning algorithm to generate cooperative trajectories for the robot's dual arms, mobile chassis, and end effector, and determines the parallel or serial execution mode based on process constraints. Before execution, a lightweight digital twin simulation is run to verify the feasibility of key actions and generate stopping points and rollback points as recovery mechanisms in abnormal situations. During execution, if the detected deviation of torque, position, or tooling state from the expected value exceeds a threshold, the orchestrator will invoke a local replanning unit to instantly adjust the motion trajectory or process parameters, ensuring the task continues without requiring overall reprogramming.

[0033] It can be recognized that this embodiment enables humanoid robots to quickly understand work order changes and workpiece replacement requirements through multimodal task parsing and process knowledge graph reasoning, eliminating the need for cumbersome manual teaching or reprogramming. It can complete the entire process from part identification and process matching to motion choreography within minutes, significantly shortening process changeover time and significantly improving production line response speed in flexible manufacturing environments.

[0034] S103. Control the humanoid robot to update tools and reconfigure workstations according to the standardized task description; In some embodiments, controlling a humanoid robot to perform tool updates and workstation reconfiguration according to a standardized task description includes: S1031. Determine tool usage requirements and workstation configuration requirements based on standardized task descriptions; S1032. Determine the tool replacement plan based on tool usage requirements, and control the humanoid robot to retrieve the target tool from the tool library according to the tool replacement plan; S1033. Control the humanoid robot to move the tooling table and fixture according to the workstation configuration requirements and update the workstation coordinate system.

[0035] Specifically, in this embodiment, each time a process flow is generated based on the task description, a tool replacement plan is automatically generated according to the node requirements. Each tool and fixture has a unique identifier, lifespan curve, maintenance record, and calibration parameters. When a tool's lifespan is nearing its end, abnormal wear is detected, or the current task requires the use of other tools, the robot can autonomously retrieve spare parts from the tool library and calibrate the tool's installation position and posture using both vision and force sensing. When a work order requires switching parts or transferring production line bottlenecks, the spatial layout of workstations and materials, as well as other workstation configuration requirements, also need to be changed accordingly. The robot can move lightweight tooling tables, rearrange fixtures, and update the coordinate system based on the digital map of the workshop and the current workstation configuration requirements, forming new reachable and safe zones to ensure that the production line can be reconfigured within the minimum downtime window.

[0036] It can be recognized that this embodiment, through integrated toolchain management and workstation reconfiguration, supports robots in performing multiple processes such as drilling, welding, inspection, and assembly within a single task. The robot can autonomously complete tool picking, placement, and calibration, and pre-schedule spare parts based on tool life and wear. This function significantly expands the functional boundaries of humanoid robots, enabling them to perform multiple tasks and reducing manual intervention and maintenance costs.

[0037] S104. Drive the humanoid robot to perform process operations according to the motion trajectory, obtain process parameters and quality inspection parameters during the execution process, and adjust the working state of the humanoid robot according to the process parameters and quality inspection parameters.

[0038] In some embodiments, process parameters include mechanical signals, pose signals, temperature signals, and acoustic signals; quality inspection parameters include assembly clearance, hole position accuracy, torque curve, and weld morphology. Adjusting the working state of the humanoid robot based on the process parameters and quality inspection parameters includes: S1041. Compare the process parameters with the preset process template to obtain the process deviation index, and execute the corresponding process control strategy according to the process deviation index. S1042. Determine the quality deviation index based on the quality inspection parameters, and implement the corresponding quality control strategy based on the quality deviation index.

[0039] S1043. Perform safety checks on the working status of the humanoid robot, obtain the safety check results, and execute corresponding safety control strategies based on the safety check results.

[0040] Specifically, in step S104, when the humanoid robot performs contact-type actions, the controller adjusts the end effector position and torque in real time based on an impedance / compliance control algorithm. By collecting process parameters such as force, pose, temperature, and acoustic signals during the execution process and comparing them with the ideal curve in the process parameter template, if the deviation exceeds the limit, a parameter adaptive mechanism is triggered to automatically adjust the feed rate, torque, or welding current. At the same time, this embodiment also sets up a safety fence mechanism to monitor PLC interlocks, energy isolation, emergency stop buttons, and safety light curtains in real time. Once a violation is detected, the action stops immediately, the robot retreats to the nearest safe posture, and a safety event card containing the cause, evidence, and timestamp is generated and uploaded to the MES and auditing system.

[0041] In step S1042, this embodiment uses a binocular camera, a laser profilometer, and force-displacement curve analysis to perform online detection of quality inspection parameters such as assembly gap, hole position accuracy, torque curve, and weld morphology. The detection results are fed back to the process flow controller in real time. If a deviation is found but is still within the allowable range of the process, parameter fine-tuning is triggered; if the quality red line is exceeded, an alternative path is automatically entered or a shutdown prompt is issued.

[0042] In step S1043, this embodiment compiles the SOP terms, interlock logic, cooperative space limits, and energy isolation status into an unbypassable safety fence, which serves as a necessary condition for issuing actions. During execution, the fence is monitored in real time at the edge as a high-priority task: if an emergency stop, grating occlusion, out-of-bounds posture, or unclosed energy interlock is detected, the action is immediately interrupted and the user reverts to the nearest safe stopping point. At the same time, a safety event card consisting of "cause-evidence-timestamp-state snapshot" is generated to ensure the compliance and traceability of the operation.

[0043] It can be recognized that this embodiment achieves online parameter adjustment by combining force, vision, and acoustic signals. When tolerance deviations, tool wear, or environmental changes are detected, the robot can automatically replan its actions or fine-tune process parameters, thereby ensuring the stability and continuity of the production process. This mechanism reduces rework and scrap rates, and improves the consistency and robustness of manufacturing quality. At the same time, quality inspection is moved forward to the execution stage, using visual, mechanical, and acoustic features to achieve embedded quality inspection, and process parameters can be fine-tuned in real time when deviations are detected. Furthermore, a safety fence mechanism is used to verify the restrictions such as SOPs, EHS clauses, and PLC interlocks in real time, ensuring that all predictions and actions are executed within the safety boundary. Once a risk is triggered, the action is immediately interrupted and a safety event card with evidence is generated, thereby achieving safety control and accountability traceability.

[0044] In some embodiments, the flexible manufacturing process planning and execution method further includes: S105. Obtain execution logs and quality inspection data during the execution process; S106. Update the process knowledge graph based on the execution log and quality inspection data.

[0045] Specifically, in this embodiment, in response to the problem that traditional methods lack self-optimization and learning mechanisms, and that robots cannot accumulate experience and form personalized process knowledge during execution, this embodiment establishes a learning optimization mechanism by collecting execution logs and quality inspection data during the execution process, and expands the process knowledge graph online.

[0046] The present invention will be further described below with reference to a specific embodiment: The experimental environment was set up in a flexible electronics manufacturing workshop. This workshop exhibits typical characteristics of multi-variety, small-batch production, with workpieces consisting of control circuit modules of varying models. The main processes included PCB insertion, screw fixing, solder joint inspection, and casing assembly. A dual-arm humanoid robot was selected, equipped with a perception system including binocular industrial cameras, an end-effector camera, a six-axis torque sensor, an array microphone, and temperature and humidity sensors; an execution system consisting of dual-arm end-effector grippers, an automatic tool changer (supporting bits, suction cups, and torque wrenches), and a mobile chassis; and an information system comprising an edge computing unit (8-core CPU, 32GB memory, and a GPU inference card), connected to the MES / PLC via OPCUA and REST interfaces, and a software system.

[0047] When the MES issues a new work order requiring the production to be switched to "Circuit Module B" with the process of "PCB assembly + upper shell fixing + online solder joint inspection", the robot first uses visual recognition to confirm the workpiece pallet model and matches it with the work order information. Simultaneously, it receives the operator's verbal instruction "Switch to B batch, pay attention to solder joint inspection". The robot performs time synchronization and semantic fusion of the text work order, visual recognition results, voice instructions and on-site status information to form a standardized task description containing "part = B-type module, process = three steps, quality requirement = add inspection steps".

[0048] Subsequently, the standard process path for module B is retrieved from the process knowledge graph. Based on the node content of the standard process path, it is confirmed that "fixed fixture A, screwdriver M2, and camera C1 inspection" are required. A tool change plan and workstation configuration requirements are generated. If the inference engine finds that fixture A is currently occupied, an alternative path will be called, spare fixture B will be selected, and the parameter template will be updated. At the same time, when producing the standard process path, the constraint solver checks the constraint information such as SOP clauses and confirms that electrical testing must be performed before entering the assembly step. Therefore, the corresponding additional nodes are automatically inserted into the path. After the above process nodes are converted into DAG, information such as "node 1 = PCB assembly, node 2 = screw fixing, node 3 = electrical testing, node 4 = solder joint inspection, node 5 = upper shell assembly" can be obtained. The robot skill primitive library is called to map the above nodes to action primitives such as grasping, insertion, tightening, and measurement. A collaborative trajectory is planned for the two arms. After the simulation is quickly completed, the motion trajectory containing five safe stopping points and two rollback paths is output.

[0049] After generating the motion trajectory, based on the aforementioned tool change plan and workstation configuration requirements, the system detects that the task requires switching the screwdriver specification. The robot autonomously retrieves an M2 bit from the tool library and uses both vision and force sensing to confirm the installation accuracy. Subsequently, the robot moves the lightweight tooling table, rearranges the workstation, and ensures accessibility. The system updates the digital map and broadcasts the new layout coordinates.

[0050] Finally, during the execution phase, when assembling the PCB, the force sensor detects that the insertion torque is higher than the template value, triggering adaptive parameter adjustments, reducing speed, and fine-tuning the posture. During screw fixing, if the torque curve exceeds the threshold, the robot automatically switches to a backup torque template to ensure the screw is not damaged. In the electrical testing phase, the robot uses camera C1 to compare the brightness and current value of the solder joints in real time and feeds the results back to the MES. After solder joint inspection, the robot uses a binocular camera and a laser profilometer to confirm the solder joint forming quality. If a defective solder joint is found, the system records the anomaly in the log and marks the work order number, workstation environmental parameters, and tool status. After the experiment, the logs are uploaded to the cloud for statistical process control analysis and written into a knowledge graph.

[0051] Please see Figure 2 This application also provides a flexible manufacturing process planning and execution device for humanoid robots, which can implement the above-mentioned method. The device includes: The task description generation module is used to generate standardized task descriptions based on visual element information, operator voice command information, work order text information, and on-site status information in flexible manufacturing production scenarios. The trajectory generation module is used to retrieve the optimal standard process path from the process knowledge graph based on the standardized task description, and generate the motion trajectory of the humanoid robot based on the standard process path. The tool update and workstation reconfiguration module is used to control the humanoid robot to perform tool updates and workstation reconfiguration based on standardized task descriptions. The execution and adjustment module is used to drive the humanoid robot to perform process operations according to the motion trajectory, obtain process parameters and quality inspection parameters during the execution process, and adjust the working state of the humanoid robot according to the process parameters and quality inspection parameters.

[0052] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0053] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0054] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0055] Please see Figure 3 , Figure 3 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the methods described in the embodiments of this application. The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0056] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0057] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0058] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0059] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0060] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0061] This application provides a flexible manufacturing process planning and execution method and related equipment for humanoid robots. The method integrates visual element information, operator voice commands, work order text information, and on-site status information to generate a unified, standardized task description. This achieves semantic fusion of multi-source heterogeneous information, improves the parsing efficiency of complex manufacturing tasks, and provides a foundation for subsequent automated process planning. Based on the standardized task description, the optimal standard process path is retrieved from the process knowledge graph, and the corresponding humanoid robot motion trajectory is generated, ensuring the feasibility and optimality of the process planning results. The humanoid robot is automatically controlled to update tools and reconfigure workstations according to the standardized task description, enabling rapid adjustment of production conditions based on the current task. This improves the adaptability of the manufacturing system to changes in task requirements, thereby increasing the automation level and production efficiency of flexible manufacturing and reducing manual intervention and maintenance costs. During process execution, by collecting and dynamically comparing process parameters and quality inspection parameters, the working status of the humanoid robot is adjusted in real time, effectively eliminating process deviations and quality fluctuations, and improving production quality and consistency.

[0062] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0063] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0064] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0065] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0066] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0067] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0068] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0069] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0070] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0071] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0072] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A flexible manufacturing process planning and execution method applied to humanoid robots, characterized in that, include: Standardized task descriptions are generated based on visual element information, operator voice command information, work order text information, and on-site status information in the flexible manufacturing production scenario. The optimal standard process path is retrieved from the process knowledge graph based on the standardized task description, and the motion trajectory of the humanoid robot is generated based on the standard process path. The humanoid robot is controlled to perform tool updates and workstation reconfiguration according to the standardized task description. The humanoid robot is driven to perform process operations according to the motion trajectory, process parameters and quality inspection parameters are obtained during the execution process, and the working state of the humanoid robot is adjusted according to the process parameters and quality inspection parameters.

2. The flexible manufacturing process planning and execution method for humanoid robots according to claim 1, characterized in that, The visual element information includes workpiece type, tooling markings, and workstation status; the work order text information includes part model, batch number, and delivery time requirement; the on-site status information includes process context information and environmental sensor parameters; the process context information includes workstation occupancy information, tooling positioning information, and tool life information; and the generation of a standardized task description based on the visual element information, operator voice command information, work order text information, and on-site status information of the flexible manufacturing production scenario includes: The visual recognition unit acquires image information of the flexible manufacturing production scene, and extracts visual element information based on the image information. The operator's voice command information is obtained through a voice recognition unit; The work order text is obtained, and the work order text is parsed by the text parsing unit to obtain the work order text information; The visual element information, operator voice command information, work order text information, and on-site status information are supplemented through a minimal questioning mechanism. The supplemented visual element information, operator voice command information, work order text information, and on-site status information are synchronized in time and semantically fused to obtain the standardized task description.

3. The flexible manufacturing process planning and execution method for humanoid robots according to claim 1, characterized in that, The step of retrieving the optimal standard process path from the process knowledge graph based on the standardized task description includes: Obtain constraint information, and retrieve several executable process paths from the process knowledge graph based on the standardized task description and the constraint information; The executable process paths are evaluated according to a preset production index evaluation algorithm to obtain the optimal standard process path.

4. The flexible manufacturing process planning and execution method for humanoid robots according to claim 1, characterized in that, The process of generating the motion trajectory of the humanoid robot according to the standard process path includes: The standard process path is transformed into a directed acyclic graph; Each node in the directed acyclic graph is mapped to a number of operation instructions in the humanoid robot skill primitive library; Using a preset path planning algorithm, a collaborative trajectory is generated for the humanoid robot's arms, torso, and base according to each of the operation instructions, and the feasibility of the collaborative trajectory is verified through digital twin simulation. The cooperative trajectory that has passed feasibility verification will be used as the motion trajectory.

5. The flexible manufacturing process planning and execution method for humanoid robots according to claim 1, characterized in that, The step of controlling the humanoid robot to perform tool updates and workstation reconfiguration according to the standardized task description includes: Based on the standardized task description, determine the tool usage requirements and workstation configuration requirements; Based on the tool usage requirements, a tool replacement plan is determined, and based on the tool replacement plan, the humanoid robot is controlled to retrieve the target tool from the tool library; According to the workstation configuration requirements, the humanoid robot is controlled to move the tooling table and fixture and update the workstation coordinate system.

6. The flexible manufacturing process planning and execution method for humanoid robots according to claim 1, characterized in that, The process parameters include mechanical signals, pose signals, temperature signals, and acoustic signals; the quality inspection parameters include assembly gaps, hole position accuracy, torque curves, and weld joint morphology; adjusting the working state of the humanoid robot based on the process parameters and the quality inspection parameters includes: The process parameters are compared with a preset process template to obtain a process deviation index, and a corresponding process control strategy is executed based on the process deviation index. The quality deviation index is determined based on the quality inspection parameters, and the corresponding quality control strategy is executed based on the quality deviation index. The working state of the humanoid robot is subjected to safety detection, the safety detection result is obtained, and the corresponding safety control strategy is executed based on the safety detection result.

7. The flexible manufacturing process planning and execution method for humanoid robots according to claim 1, characterized in that, The flexible manufacturing process planning and execution method also includes: Obtain execution logs and quality inspection data during the execution process; The process knowledge graph is updated based on the execution log and the quality inspection data.

8. A flexible manufacturing process planning and execution device for humanoid robots, characterized in that, The device includes: The task description generation module is used to generate standardized task descriptions based on visual element information, operator voice command information, work order text information, and on-site status information in flexible manufacturing production scenarios. The trajectory generation module is used to retrieve the optimal standard process path from the process knowledge graph based on the standardized task description, and generate the motion trajectory of the humanoid robot based on the standard process path. The tool update and workstation reconstruction module is used to control the humanoid robot to perform tool updates and workstation reconstruction according to the standardized task description. The execution and adjustment module is used to drive the humanoid robot to perform process operations according to the motion trajectory, obtain process parameters and quality inspection parameters during the execution process, and adjust the working state of the humanoid robot according to the process parameters and the quality inspection parameters.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images