Robot attitude control method and system based on intelligent identification

Abstract

The invention relates to the field of robot attitude control, in particular to a robot attitude control method and system based on intelligent recognition. The method comprises the following steps: continuously acquiring a real-time sandbox scene video stream based on a high-definition camera; and carrying out time sequence frame segmentation on the real-time sandbox scene video stream, and carrying out global contrast enhancement processing to obtain a global contrast optimization image sequence. Carrying out dynamic object identification on the global contrast optimization image sequence, and carrying out continuous frame optical flow motion tracking to generate a dynamic object optical flow change track; optimizing the image sequence according to the global contrast to extract scene ground and scene obstacle data; performing dynamic trajectory rendering on the scene ground and scene obstacle data based on the dynamic object optical flow change trajectory so as to construct a dynamic scene twinborn model; robot position posture parameters and user operation instructions are recognized. According to the invention, timely attitude adjustment response and accurate movement position control are realized.

Description

Technical Field

[0001] The present invention relates to the field of robot posture control, and in particular to a robot posture control method and system based on intelligent recognition. Background Art

[0002] With the continuous improvement of the degree of industrial automation, robots have been widely used in production, warehousing, logistics and other fields. As an innovative entertainment application, sandbox scene robots based on intelligent recognition have attracted widespread attention. The robot is placed in a virtual large-scale sandbox scene. By interacting with the real sandbox, it realizes autonomous mobile control in a complex environment. At the same time, the sandbox scene also integrates tower defense game elements. The robot needs to complete autonomous pathfinding, defense and other tasks under the interference of various obstacles and enemies, and experience a new entertainment interaction mode. However, to achieve autonomous and efficient movement of robots in such a complex sandbox scene, the traditional attitude control method based on fixed-point navigation can no longer meet the requirements. There are often problems such as untimely attitude adjustment response and inaccurate moving position. The intelligent sandbox scene robot needs to have the ability of comprehensive perception of the environment, intelligent recognition and adaptive decision-making, so as to cope with the interference of various unknown obstacles, hostile robots and other complex factors. Therefore, it is urgent to develop a sandbox scene robot attitude control method based on intelligent recognition to meet the needs of innovative applications of modern sandbox combat robots. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a robot posture control method and system based on intelligent recognition to solve at least one of the above technical problems.

[0004] To achieve the above object, the present invention provides a robot posture control method based on intelligent recognition, wherein the robot is embedded with a high-definition camera, comprising the following steps: Step S1: continuously acquiring a real-time sandbox scene video stream based on a high-definition camera; performing global contrast enhancement processing on the real-time sandbox scene video stream to obtain a global contrast optimized image sequence; Step S2: performing continuous frame optical flow motion tracking on the global contrast optimized image sequence to generate the optical flow change trajectory of the dynamic object; Step S3: construct a dynamic scene twin model based on the optical flow change trajectory of dynamic objects; Step S4: identifying the robot position and posture parameters and the user operation instructions; performing dynamic avoidance path planning for the user operation instructions, thereby generating a target movement avoidance path; Step S5: Perform dynamic posture execution control based on the robot position and posture parameters to construct a posture smooth control sequence; Step S6: Based on the target movement avoidance path and the attitude smoothing control sequence, the attitude tilt reverse compensation calculation is performed on the dynamic scene twin model to construct an intelligent attitude control strategy.

[0005] The present invention obtains real-time video stream through a high-definition camera to provide real-time data support for subsequent processing, performs global contrast enhancement processing on the video stream, improves image quality, and makes subsequent processing more accurate. Through optical flow motion tracking, it is possible to accurately identify and track dynamic objects. The generated optical flow change trajectory understands the movement of objects in the scene and provides important information for subsequent path planning. The scene ground and obstacle data are extracted from the image sequence to provide basic information for robot navigation. Through dynamic trajectory rendering, a dynamic scene twin model is constructed to simulate the real scene and improve the robot's perception ability. The user's operation instructions are analyzed, the target points are identified, and path planning is performed to achieve the robot's target movement and avoid obstacles. The dynamic avoidance path is generated so that the robot can complete the task safely and efficiently. Based on the robot's position and posture parameters, the execution control of the dynamic posture is realized to make the robot's movement smoother and more accurate. A posture smoothing control sequence is constructed to improve the stability and controllability of the robot's movement. Based on the target movement avoidance path and the posture smoothing control sequence, the robot movement simulation calculation and posture tilt reverse compensation calculation are performed to construct an intelligent posture control strategy to improve the robot's movement efficiency and safety.

[0006] Preferably, step S1 comprises the following steps: Step S11: continuously acquiring a real-time sandbox scene video stream based on a high-definition camera; Step S12: performing time-sequential frame segmentation on the real-time sandbox scene video stream to extract a sandbox scene image frame sequence; Step S13: performing low-pass filtering and noise reduction on the sandbox scene image frame sequence frame by frame to obtain a filtered and noise-reduced image frame sequence; Step S14: performing global contrast enhancement processing on the filtered denoised image frame sequence, thereby obtaining a global contrast optimized image sequence.

[0007] The present invention uses a high-definition camera to continuously acquire a real-time sandbox scene video stream, and the system obtains high-quality video input, thereby providing clear visual information for subsequent processing. By performing time-series frame segmentation on the real-time sandbox scene video stream, the video stream is segmented into a separate image frame sequence, providing separate frame data for subsequent processing, and facilitating independent processing of each frame. Frame-by-frame low-pass filtering and denoising effectively reduce the noise in the image, improve image quality, and make subsequent processing more accurate and reliable. By performing global contrast enhancement processing on the filtered and denoised image frame sequence, the contrast of the image is enhanced, the visual effect of the image is improved, and the image is clearer and easier to analyze, and the subsequent robot posture control is performed.

[0008] Preferably, the specific steps of step S14 are: Extracting a first frame image based on a filtered denoised image frame sequence; Performing average brightness calculation on the first frame of image to generate an average brightness value of the image; Performing brightness distribution analysis on the first frame of image to generate image brightness distribution data; Perform brightness area division processing based on image brightness distribution data to extract dark area image and bright area image of the current frame image; Performing brightness compensation enhancement on the dark region image based on the average brightness value of the image to generate a brightness enhanced dark region image; Performing global contrast deviation identification on the first frame image based on the brightness enhanced dark image to generate a contrast deviation value; Based on the contrast deviation value, the contrast of the brightness enhanced dark area image and the bright area image is optimized and adjusted, thereby constructing a global contrast optimized image; Each frame of image is extracted based on the sandbox scene image frame sequence, and the above contrast optimization adjustment process is iterated on each frame of image to obtain a global contrast optimized image sequence.

[0009] The present invention calculates the average brightness to understand the overall brightness level of the image, and provides a benchmark for subsequent brightness processing. The average brightness value can be used to normalize the brightness processing process to ensure that the processing result is within a suitable brightness range. The brightness distribution is analyzed to understand the brightness distribution of the image, and the main brightness areas and features are identified to provide a basis for subsequent processing. According to the brightness distribution data, the brightness of the image is adjusted in a targeted manner, the dark or bright details are enhanced, and the image quality is improved. The image is divided into dark and bright areas to process the details in different brightness ranges respectively, so that the brightness adjustment is more accurate, the details in the dark area are highlighted to improve the visual effect of the image, and the image is more three-dimensional and clear. The dark area image is enhanced by brightness compensation, so that the brightness of the overall image is more balanced, the visual effect of the image is enhanced, the brightness of the dark area is enhanced to highlight the detail information, and the quality and recognition performance of the image are improved. The contrast deviation value is identified to help evaluate the contrast of the image and determine whether further optimization and adjustment are needed. The contrast deviation value can be used as a basis for adjusting the contrast of the image to ensure that the visual effect of the image is more balanced and clear. The overall contrast of the image is improved by contrast optimization and adjustment, so that the image has more impact and clarity. The contrast is optimized to highlight the details and features in the image, and the visual appeal and recognition accuracy of the image are improved.

[0010] Preferably, step S2 specifically comprises the following steps: Step S21: performing dynamic object recognition on the global contrast optimized image sequence and marking the dynamic objects; Step S22: segmenting the dynamic object into image frames, thereby obtaining dynamic object image frames; Step S23: Calculate the object space position of the dynamic object image frame to generate the dynamic object space position coordinates; Step S24: calculating the object motion change rate of consecutive frames for the dynamic object spatial position coordinates to generate the object position coordinate change rate; Step S25: performing continuous frame optical flow motion tracking on the global contrast optimized image sequence based on the object position coordinate change rate to generate an optical flow change trajectory of the dynamic object.

[0011] The present invention performs dynamic object recognition on a global contrast optimized image sequence and marks the dynamic objects, so that the robot can recognize the moving objects in the scene. The robot that recognizes the dynamic objects predicts the behavior and path of the objects, thereby improving the accuracy of intelligent decision-making. The position and boundary of the dynamic objects are accurately marked by image frame segmentation, providing accurate input data for subsequent position calculation and motion tracking. The segmented image frame extracts the key information of the dynamic objects for further analysis and processing. The spatial position coordinates of the dynamic objects are calculated to help the robot accurately understand the position information of the objects in the scene, and support subsequent motion analysis and control. The generated spatial position coordinates provide the robot with the accurate position of the object, and support the robot to perform path planning and posture control. The change rate of the object position coordinates is calculated to understand the motion trajectory and speed of the object, and provide a reference for the robot to track and predict the motion of the object. The calculation of the change rate helps the robot to quickly respond to the motion changes of the object, thereby improving the flexibility of intelligent decision-making. Through optical flow motion tracking, the motion trajectory and speed of the object are tracked more accurately, and the accuracy and stability of posture control are improved. The optical flow change trajectory helps the robot predict the future motion direction of the object, and supports intelligent decision-making and obstacle avoidance functions.

[0012] Preferably, step S3 specifically comprises the following steps: Step S31: performing frame-by-frame scene visual semantic analysis on the global contrast optimized image sequence to extract scene ground and scene obstacle data; Step S32: Calculate the height and flatness of the scene ground to obtain the height and flatness data of the scene ground; Step S33: constructing a three-dimensional point cloud ground scene based on the scene ground height and flatness data and scene obstacle data to obtain a static three-dimensional scene model; Step S34: Use the optical flow change trajectory of the dynamic object to dynamically render the point cloud plane scene model, thereby constructing a dynamic scene twin model.

[0013] The present invention extracts ground and obstacle data in the scene through visual semantic analysis frame by frame, so that the robot can better understand the surrounding environment. The extracted scene data provides basic information for subsequent ground and obstacle processing, supports the robot's path planning and obstacle avoidance functions, calculates the height and flatness data of the ground to understand the characteristics of the scene terrain, and provides an important reference for the construction of three-dimensional scenes. The height and flatness data can be used for the robot's navigation decisions to help the robot choose a suitable path and driving mode. A three-dimensional point cloud ground scene model is constructed based on the ground flatness data and obstacle information. The robot establishes an overall understanding of the environment. The static three-dimensional scene model provides spatial structure information of the environment, supports the robot's positioning, navigation and obstacle avoidance capabilities, and uses the dynamic object optical flow change trajectory to perform dynamic trajectory rendering on the point cloud plane scene model to integrate dynamic objects into the scene, making the scene more realistic and vivid. The dynamic scene twin model provides motion information of dynamic objects, enhances the robot's environmental perception ability, intelligent decision-making and obstacle avoidance behavior.

[0014] Preferably, the specific steps of step S34 are: Calculate the obstacle stereoscopic shape of the scene obstacle data to obtain obstacle shape parameters; Identify the spatial layout and position distribution of scene obstacle data and extract obstacle position distribution data; Based on the obstacle position distribution data, the scene layout structure analysis is performed on the global contrast optimization image sequence to generate the scene layout structure data; Based on the scene ground height and flatness data, the ground high and low points are sampled and fitted to generate the ground terrain feature grid; The three-dimensional point cloud ground scene is constructed based on the scene layout structure data based on the ground terrain feature grid to obtain a static three-dimensional scene model.

[0015] The present invention calculates the three-dimensional morphological parameters of obstacles, deeply understands the shape and structure of obstacles, and the robot avoids and crosses obstacles. The three-dimensional morphological parameters provide detailed information of obstacles, provide a basis for the robot's path planning and obstacle avoidance decision-making, identify the spatial layout and position distribution data of obstacles, and the robot accurately perceives the surrounding environment, improves the accuracy of obstacle avoidance and navigation, and the extracted position distribution data is used to construct a spatial structure model of the scene to provide support for the robot's scene understanding and planning. The scene layout structure analysis is performed based on the obstacle position distribution data to understand the overall layout and structure of the scene, and provide guidance for the robot's path planning and action. The generated scene layout structure data enhances the robot's perception of the environment and improves the accuracy of intelligent decision-making. Through ground high and low point sampling and fitting, a ground terrain feature grid is generated to capture the detailed features of the terrain, providing accurate terrain information for scene construction. The terrain feature grid helps the robot plan a path that adapts to the terrain and improves the robot's travel efficiency under different terrains. A three-dimensional point cloud ground scene model is constructed based on the ground terrain feature grid to enable the robot to have an overall cognition and perception of the scene. The static three-dimensional scene model provides the robot with comprehensive environmental information and supports the robot's navigation, obstacle avoidance and decision-making capabilities.

[0016] Preferably, the specific steps of step S4 are: Step S41: Identify robot position and posture parameters and user operation instructions; Step S42: Analyze the user's operation instruction for the user's set target point to obtain the user's set target point; Step S43: Calculate the target position of the target point set by the user to generate the moving position coordinates of the target point; Step S44: performing obstacle collision avoidance analysis on the moving position coordinates of the target point based on the dynamic scene twin model, and generating obstacle collision avoidance logic; Step S45: Perform dynamic avoidance path planning based on obstacle collision avoidance logic to generate a target movement avoidance path.

[0017] The present invention understands the current position and orientation of the robot in real time by identifying the robot position and posture parameter system, and provides basic data for subsequent path planning. It can understand the user's intention by identifying user operation instructions, thereby providing guidance for the robot's movement and behavior. The target point analysis system for user operation instructions accurately understands the target point set by the user, improves the accuracy of the system's response to user needs, and obtains the target point set by the user to provide a clear target direction and position for the robot's movement. The target point moving position coordinates are generated by target position calculation to improve the accuracy and precision of the robot's movement. The generated target point position coordinates provide basic data for subsequent path planning and mobile operations. Obstacle collision avoidance analysis is performed based on the dynamic scene twin model, which can help the robot avoid collisions and ensure safe movement. The generated collision avoidance logic provides the robot with environmental perception capabilities, allowing the robot to better adapt to complex environments. Dynamic avoidance path planning is performed based on obstacle collision avoidance logic to optimize the robot's moving path, improve efficiency and safety, and the generated target moving avoidance path enables the robot to autonomously plan the path, avoid obstacles, and achieve accurate arrival at the target point.

[0018] Preferably, the specific steps of step S5 are: Step S51: Perform robot dynamic behavior demand analysis on user operation instructions to extract dynamic behavior demand features; Step S52: performing dynamic constraint analysis on the robot position and posture parameters to generate posture dynamic constraint data; Step S53: performing temporal action sequence analysis on the dynamic behavior demand characteristics, thereby obtaining action demand sequences at multiple time points; Step S54: performing dynamic posture execution control on the action requirement sequences at multiple time points based on the posture dynamics constraint data, thereby generating a dynamic posture control sequence; Step S55: performing posture change jump smoothing on the dynamic posture control sequence to construct a posture smoothing control sequence.

[0019] The present invention analyzes user operation instructions, and the system can provide personalized feedback and services according to the user's dynamic behavior needs, enhance user experience, extract dynamic behavior demand feature system to accurately understand the user's action requirements, provide guidance for the robot's dynamic control, generate posture dynamic constraint data to help the robot maintain stable action execution, and avoid exceeding the robot's physical ability range. Dynamic constraint analysis provides restriction conditions for the robot's dynamic control, helping the robot to perform posture control in a dynamic environment. The time-series action sequence analysis robot reasonably arranges action execution according to the time sequence to improve the fluency and coherence of dynamic behavior. Extracting action demand sequences at multiple time points helps the system predict the user's dynamic behavior and provide advance preparation for the robot's dynamic control. Dynamic posture execution control adjusts the posture control strategy according to the dynamic constraint data, so that the robot can quickly respond to the user's action needs. Dynamic posture control according to constraint data can optimize the robot's dynamic behavior and improve execution efficiency and accuracy. Smooth posture change jumps reduce posture jumps in the robot's action execution, making the action transition more natural and smooth. Constructing a posture smooth control sequence improves the coordination between robot actions and enhances the coherence and beauty of the overall action.

[0020] Preferably, the specific steps of step S6 are: Step S61: performing robot movement simulation calculation on the dynamic scene twin model based on the target movement avoidance path and the posture smoothing control sequence to obtain robot movement simulation response data; Step S62: Predicting the movement posture inclination of the robot movement simulation response data based on the scene ground height and flatness data to generate a posture inclination prediction parameter; Step S63: performing posture tilt reverse compensation calculation based on the posture tilt prediction parameters to construct posture compensation control parameters; Step S64: Calculate the optimal power cost of the robot movement simulation response data to obtain the optimal power cost path; Step S65: Perform iterative scene movement optimization based on the posture compensation control parameters and the optimal power cost path to build an intelligent posture control strategy.

[0021] Through simulation calculation, the system of the present invention can predict the movement response of the robot in the actual scene, help optimize the posture control strategy, and the simulation calculation can verify the effectiveness of the target movement avoidance path and the posture control sequence, thereby improving the success rate of the robot in executing the mobile task. Through posture tilt prediction, the system predicts the posture tilt of the robot under different ground conditions and adjusts the posture control strategy in advance. The posture tilt prediction improves the stability and safety of the robot in complex terrain and reduces the risk caused by tilt. The posture tilt reverse compensation calculation can help the robot maintain a stable posture during movement and reduce the interference of tilt on the robot operation. The posture compensation regulation parameters improve the robot's movement balance and make it more flexible and reliable in complex scenes. The optimal power cost calculation reduces the energy consumption of the robot during movement and improves the energy utilization efficiency. The optimal power cost path is calculated to optimize the cost of the robot's movement and improve the economic benefits of the system. The iterative scene movement optimization can dynamically adjust the posture compensation parameters and movement path according to the real-time situation, thereby improving the adaptability and flexibility of the robot. An intelligent posture control strategy is constructed to optimize the robot's movement performance in complex scenes and improve the intelligence level of the system.

[0022] In this specification, a robot posture control system based on intelligent recognition is provided, which is used to execute the robot posture control method based on intelligent recognition as described above, comprising: The image processing module continuously acquires the real-time sandbox scene video stream based on the high-definition camera; performs global contrast enhancement processing on the real-time sandbox scene video stream to obtain a global contrast optimized image sequence; The object change trajectory module is used to perform continuous frame optical flow motion tracking on the global contrast optimized image sequence to generate the optical flow change trajectory of dynamic objects; The twin model module builds a dynamic scene twin model based on the optical flow change trajectory of dynamic objects; The path planning module identifies the robot's position and posture parameters and user operation instructions; it performs dynamic avoidance path planning based on the user's operation instructions, thereby generating a target movement avoidance path; The behavior requirement module performs dynamic posture execution control based on the robot's position and posture parameters to build a smooth posture control sequence; The intelligent attitude control module performs attitude tilt inverse compensation calculation on the dynamic scene twin model based on the target movement avoidance path and attitude smoothing control sequence, and constructs an intelligent attitude control strategy.

[0023] The present invention can realize real-time monitoring and perception of the scene by continuously acquiring real-time sandbox scene video streams, improve image quality and contrast through time-series frame segmentation and global contrast enhancement processing, more accurately identify objects and details in the scene, realize recognition and tracking of dynamic objects, effectively monitor the position and motion changes of objects in the scene, generate object change trajectories through optical flow motion tracking to analyze the motion behavior and trend of objects, extract scene information including ground and obstacle data based on the optimized image sequence, provide scene reference for subsequent path planning and behavior control, and build a dynamic scene twin model through dynamic trajectory rendering, so that the system can more accurately simulate dynamic changes in the scene and identify user operation instructions. , generate target movement avoidance paths, help the robot plan safe and effective movement paths, perform dynamic avoidance path planning based on user instructions and scene information, improve the robot's navigation ability in complex environments, perform dynamic posture execution control according to user operation instructions and robot position and posture parameters, so that the robot can meet the user's specific behavioral needs, build a posture smoothing control sequence, and improve the stability and efficiency of the robot's movement. Perform robot movement simulation calculations based on target movement avoidance paths and posture control sequences to help the system predict the robot's performance in dynamic scenes, and improve the robot's posture control effect and motion stability in complex environments through posture tilt inverse compensation calculations and intelligent posture control strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of the steps of a robot posture control method based on intelligent recognition according to the present invention; Figure 2 Detailed implementation flow chart of step S1; Figure 3 Detailed implementation flow chart of step S2; Figure 4 Detailed implementation flow chart of step S3. DETAILED DESCRIPTION

[0025] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0026] The present application example provides a robot posture control method and system based on intelligent recognition. The execution subject of the robot posture control method and system based on intelligent recognition includes but is not limited to: mechanical equipment, data processing platform, cloud server node, network upload device, etc. equipped with the system can be regarded as the general computing node of the present application, and the data processing platform includes but is not limited to: at least one of an audio image management system, an information management system, and a cloud data management system.

[0027] See also Figures 1 to 4The present invention provides a robot posture control method based on intelligent recognition, and the robot posture control method based on intelligent recognition comprises the following steps: Step S1: continuously acquiring a real-time sandbox scene video stream based on a high-definition camera; performing global contrast enhancement processing on the real-time sandbox scene video stream to obtain a global contrast optimized image sequence; Step S2: performing continuous frame optical flow motion tracking on the global contrast optimized image sequence to generate the optical flow change trajectory of the dynamic object; Step S3: construct a dynamic scene twin model based on the optical flow change trajectory of dynamic objects; Step S4: identifying the robot position and posture parameters and the user operation instructions; performing dynamic avoidance path planning for the user operation instructions, thereby generating a target movement avoidance path; Step S5: Perform dynamic posture execution control based on the robot position and posture parameters to construct a posture smooth control sequence; Step S6: Based on the target movement avoidance path and the attitude smoothing control sequence, the attitude tilt reverse compensation calculation is performed on the dynamic scene twin model to construct an intelligent attitude control strategy.

[0028] The present invention obtains real-time video stream through a high-definition camera to provide real-time data support for subsequent processing, performs global contrast enhancement processing on the video stream, improves image quality, and makes subsequent processing more accurate. Through optical flow motion tracking, it is possible to accurately identify and track dynamic objects. The generated optical flow change trajectory understands the movement of objects in the scene and provides important information for subsequent path planning. The scene ground and obstacle data are extracted from the image sequence to provide basic information for robot navigation. Through dynamic trajectory rendering, a dynamic scene twin model is constructed to simulate the real scene and improve the robot's perception ability. The user's operation instructions are analyzed, the target points are identified, and path planning is performed to achieve the robot's target movement and avoid obstacles. The dynamic avoidance path is generated so that the robot can complete the task safely and efficiently. Based on the robot's position and posture parameters, the execution control of the dynamic posture is realized to make the robot's movement smoother and more accurate. A posture smoothing control sequence is constructed to improve the stability and controllability of the robot's movement. Based on the target movement avoidance path and the posture smoothing control sequence, the robot movement simulation calculation and posture tilt reverse compensation calculation are performed to construct an intelligent posture control strategy to improve the robot's movement efficiency and safety.

[0029] In the embodiment of the present invention, refer to Figure 1 , is a schematic flow chart of a robot posture control method based on intelligent recognition according to the present invention. In this example, the steps of the method include: Step S1: continuously acquiring a real-time sandbox scene video stream based on a high-definition camera; performing global contrast enhancement processing on the real-time sandbox scene video stream to obtain a global contrast optimized image sequence; In this embodiment, a suitable high-definition camera is selected to ensure that it can provide a high-resolution video stream (for example, 1080p or higher), and the camera has a good dynamic range and low-light performance to adapt to different lighting conditions. A programming language (such as Python, C++, etc.) and a corresponding library (such as OpenCV, GStreamer) are used to open the camera and start acquiring a real-time video stream. In the video stream, a time interval is set (for example, 30 frames are extracted per second), and frames are extracted within the interval for processing. Each extracted frame is saved to form a frame sequence. A suitable contrast enhancement algorithm is selected, such as histogram equalization, CLAHE (contrast limited adaptive histogram equalization), etc. Histogram equalization improves the global contrast of the image, while CLAHE better handles local contrast. The selected contrast enhancement algorithm is applied to each frame, and the processed frame is saved as a global contrast optimized image sequence. The global contrast optimized image sequence is saved in a format that can be used for subsequent processing (such as a NumPy array, a video file, or an image sequence).

[0030] Step S2: performing continuous frame optical flow motion tracking on the global contrast optimized image sequence to generate the optical flow change trajectory of the dynamic object; In this embodiment, the globally contrast optimized image sequence is preprocessed, including denoising and enhancement, to improve the accuracy of object recognition, Gaussian blur is used to reduce image noise, and edge detection (such as Canny edge detection) is applied to highlight the edge of the object, and a suitable dynamic object detection algorithm is selected, such as YOLO (You Only Look Once) or SSD (Single Shot Detector). These algorithms can quickly identify dynamic objects in the image, and a deep learning model is used to perform object detection on each frame to identify the dynamic objects in the scene and their bounding boxes. The dynamic object data detected in each frame (such as object category, position and size) are integrated into a data structure to prepare for subsequent optical flow tracking, and an optical flow calculation method is selected, such as the Lucas-Kanade optical flow method. This method is suitable for motion estimation in a small area and can track the motion of objects between consecutive frames. Optical flow calculation is performed on consecutive frame images to obtain Take the motion vector of each dynamic object between frames, and use Shi-Tomasi corner detection or FAST (Features from Accelerated Segment Test) algorithm to extract feature points in the identified dynamic object area. These feature points will be used to calculate the optical flow to ensure that the movement of the object can be accurately tracked. By calculating the optical flow of the selected feature points, the motion trajectory of each feature point between frames is obtained. Use RANSAC (random sampling consensus algorithm) to remove outliers to ensure the stability of the motion trajectory. According to the calculated optical flow information, the motion trajectory of the feature points is converted into the optical flow change trajectory of the dynamic object. By connecting the position information of the object in consecutive frames, the motion path of the object is drawn, and the generated dynamic object optical flow change trajectory is visualized on the image. Different colors and line styles are used to represent the motion trajectories of different objects. The speed and acceleration information of the object are analyzed for subsequent dynamic scene modeling and behavior prediction.

[0031] Step S3: construct a dynamic scene twin model based on the optical flow change trajectory of dynamic objects; In this embodiment, the image sequence with global contrast optimization is preprocessed, including denoising, smoothing and edge detection, to improve the effect of subsequent processing, using methods such as Gaussian blur or median filtering to reduce image noise, using image segmentation technology (such as threshold segmentation, region growing or deep learning model) to extract the ground and obstacles in the scene, processing each frame of the image to generate a binary image, identifying the ground and obstacle areas, optimizing the detection results through morphological operations (such as dilation and erosion) to ensure the integrity of the extracted area, integrating the ground and obstacle data extracted from each frame into a data structure to form a complete scene representation, recording the position, shape and size information of each obstacle, and using an optical flow method (such as the Lucas-Kanade method or the Horn-Schunck method) to calculate the image sequence The motion trajectory of dynamic objects in the scene is extracted by analyzing the pixel motion between consecutive frames, and the speed and direction information of the dynamic objects is extracted. According to the calculated optical flow information, a trajectory line or path is generated for each dynamic object, and the motion path of the dynamic object is drawn on the image. Color coding or transparency changes are used to indicate speed or acceleration. According to the extracted ground and obstacle data and the motion trajectory of the dynamic objects, a three-dimensional model of the dynamic scene is generated. 3D modeling tools or software (such as Blender, Unity, etc.) are used to combine the geometry of the ground and obstacles with the dynamic trajectory to construct the realism of the scene, and ensure that the dynamic scene twin model can be updated in real time to reflect the new position and motion trajectory of the dynamic objects. By continuously acquiring new video frames and optical flow information, the model is dynamically adjusted to ensure that it is always synchronized with the real scene.

[0032] Step S4: identifying the robot position and posture parameters and the user operation instructions; performing dynamic avoidance path planning for the user operation instructions, thereby generating a target movement avoidance path; In this embodiment, sensors (such as IMU, GPS, laser radar, etc.) are used to obtain the position, speed and posture information of the robot in real time. The accuracy and reliability of the data are improved by sensor fusion technology (such as Kalman filtering). The position coordinates (such as x, y, z) and posture angles (such as pitch angle, roll angle, yaw angle) of the robot are extracted. These parameters are stored in system variables for subsequent path planning and control. Design a user interface (such as a mobile phone application, a remote control or a voice recognition system) to allow users to enter operation instructions. Ensure that the interface is friendly and the command input is intuitive, and the user enters the command by touch, sliding or voice. Receive the operation instructions entered by the user and parse them to determine the user's intention, such as "go to target point A" or "avoid obstacles". Convert the parsed instructions into a format that the system can recognize and store them in variables for subsequent processing. According to the user's operation instructions, calculate the coordinates of the target point that the user wants to go to. If the instruction is a relative position (for example, "5 meters forward"), perform coordinate conversion according to the current posture parameters to calculate the absolute coordinates of the target point. Store the calculated target point coordinates in the system variables and prepare for path planning. Obtain obstacle data in the scene in real time, and detect obstacles through lidar, cameras or other sensors. Integrate the detected obstacle position and shape information into a data structure for subsequent path planning. Select an appropriate path planning algorithm, such as A* algorithm, Dijkstra algorithm or RRT (fast random tree) algorithm, to calculate the path from the current position to the target point. Consider the position of obstacles and ensure that the generated path is safe and feasible. During the path planning process, consider the movement of dynamic obstacles and update the path in real time to avoid potential collision risks. If the obstacle or target point changes, recalculate the path and adjust it dynamically. Optimize the calculated path to ensure that the path is smooth and reasonable, avoiding sharp turns or unnecessary pauses. Use interpolation methods (such as spline interpolation) to generate smooth path points. Store the generated target movement avoidance path in the system for the robot to use when moving. Ensure that the path data structure contains all necessary information, such as path point coordinates, speed, and steering angle.

[0033] Step S5: Perform dynamic posture execution control based on the robot position and posture parameters to construct a posture smooth control sequence; In this embodiment, the user's operation instructions, such as "forward", "backward", "turn", etc., are received, and natural language processing technology (such as keyword extraction and intent recognition) is used to parse the instructions, identify key parameters in the instructions, such as direction, speed and duration, and convert them into executable dynamic behavior requirements. Based on the parsing results, the characteristics of the dynamic behavior requirements are extracted to form a data structure to record information such as target speed, steering angle and acceleration, so as to ensure that these characteristics can accurately reflect the user's intentions and facilitate subsequent control. The current position information and posture parameters of the robot are obtained from sensors (such as IMU, GPS, etc.), including position coordinates (x, y, z) and posture angles (pitch angle, roll angle, yaw angle), and the accuracy of the data is enhanced by a sensor fusion method to ensure real-time updating of posture information. A suitable control algorithm (such as PID control, fuzzy control or model predictive control) is selected to implement dynamic control. Dynamic posture execution control, determine the control target, such as adjusting the robot posture to the target direction and achieving smooth transition, calculate the control instructions (such as adjusting the motor speed and steering angle) according to the extracted dynamic behavior requirements and current posture parameters, implement the control instructions, make the robot gradually adjust the posture, ensure smooth response without sudden changes, smooth the control instructions, use smoothing algorithms (such as low-pass filters, Bezier curves or spline interpolation) to remove fast-changing parts, ensure the stability of the posture change process, evaluate the posture changes at each time point, ensure that when executing dynamic behaviors, the robot's movement complies with physical constraints, avoid excessive acceleration and emergency stops, and generate a posture smoothing control sequence, which contains a series of control instructions at time points, indicating the robot's target posture and motion state at different time points, and storing the generated control sequence in the system for application in subsequent movement processes.

[0034] Step S6: Based on the target movement avoidance path and the attitude smoothing control sequence, the attitude tilt reverse compensation calculation is performed on the dynamic scene twin model to construct an intelligent attitude control strategy.

[0035] In this embodiment, it is ensured that the twin model of the dynamic scene has been constructed. The model should include obstacles, ground features and dynamic objects in the environment. The accuracy and real-time update capability of the scene model are ensured through digital technology (such as 3D modeling software). The target movement avoidance path generated in the previous step is used as input, and the motion trajectory of the robot in the scene is determined in combination with the attitude smoothing control sequence. The path and control sequence are integrated to form a complete movement instruction. The movement of the robot in the dynamic scene is simulated using a physical engine (such as Gazebo and Unity in ROS). The changes in the robot's position and attitude are calculated frame by frame, and physical factors such as gravity, friction and inertia are considered. At each time step, the attitude tilt of the robot in the dynamic scene is analyzed, especially when it is on uneven ground or encounters dynamic obstacles. The tilt angle and direction are recorded for subsequent compensation. A suitable inverse compensation method is selected, such as PID control or linear regression model, to calculate the required attitude adjustment. The robot is designed to calculate the amount of posture compensation, convert the posture tilt angle into a control instruction to compensate for the tilt deviation of the robot during movement. According to the calculated posture compensation, the robot's joint angle or drive setting is adjusted to ensure that it remains in a stable posture. The compensation instructions are updated in real time to ensure the robot's stability in a dynamic environment. Based on the previous motion simulation and posture compensation results, an intelligent posture control strategy is designed to ensure that the robot can flexibly respond to changes in a dynamic environment. The control logic is designed to respond to different scenarios and unexpected situations through a state machine or fuzzy logic controller. The designed intelligent posture control strategy is tested in a simulated environment to evaluate its performance in different situations. The strategy is optimized based on the test results, and the control parameters are adjusted to ensure that the strategy can gracefully handle various dynamic changes. A real-time feedback mechanism is established to continuously monitor the robot's motion state and posture, and the control strategy is adjusted in time to respond to emergencies. Real-time information is obtained through sensor data (such as IMU, force sensor) to optimize control decisions.

[0036] In this embodiment, refer to Figure 2 , is a flowchart of detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include: Step S11: continuously acquiring a real-time sandbox scene video stream based on a high-definition camera; Step S12: performing time-sequential frame segmentation on the real-time sandbox scene video stream to extract a sandbox scene image frame sequence; Step S13: performing low-pass filtering and noise reduction on the sandbox scene image frame sequence frame by frame to obtain a filtered and noise-reduced image frame sequence; Step S14: performing global contrast enhancement processing on the filtered denoised image frame sequence, thereby obtaining a global contrast optimized image sequence.

[0037] In this embodiment, the robot features of the present invention are as follows: the head has a binocular camera, and the first-person perspective of the robot can be remotely displayed through a VR head display; the head has 2 degrees of freedom and can turn up and down, left and right, and the upper body has 12 degrees of freedom to control the free movement of the upper limbs. Each servo can provide a speed of 60 degrees / s and a torque of 100kg. It can synchronize various human movements with the exoskeleton. The waist has 1 degree of freedom, and the legs have 6 degrees of freedom to squat, stand up, lean left, and lean right. The two motors in the feet can move forward, backward, rotate left, and rotate right. The maximum speed can reach 7M / s. There are 21 impact-resistant high-speed and high-torque servos throughout the body, which can flexibly synchronize human movements and perform kung fu martial arts movements. Through the robot's binocular camera, the first-person perspective of the robot can be remotely displayed through a VR head display to ensure that its perspective can cover the entire sandbox scene. Ensure that the ambient light is uniform, avoid strong shadows or reflections, and use natural light or soft artificial light in the best case. Use an appropriate software interface (such as OpenCV, ROS, etc.) to connect to the camera and start capturing real-time video streams. Ensure the stability of the data stream to avoid delays or frame drops in real-time transmission. Set up a loop structure to continuously read each frame in the video stream until manually stopped or a predetermined condition is met. Set the frequency of extracting frames as required (for example, extract 30 frames per second) and ensure that the timestamp of each frame image is recorded. When reading the video stream, store each frame successfully read in a list or array. Ensure that the saved images are in a standard format (such as JPEG or PNG) for subsequent processing. Name each frame image with a timestamp or sequence number to facilitate subsequent search and processing. Use low-pass filtering techniques such as Gaussian blur or mean filtering to effectively remove high-frequency noise from the image. Choose an appropriate kernel size (such as 5x5 or 7x7). The larger the kernel, the more obvious the noise reduction effect, but it will cause loss of details. Apply low-pass filtering to each stored frame one by one to generate a sequence of filtered images. Visualize the processed images to check the noise reduction effect and ensure that the image quality meets the requirements. Use histogram equalization or CLAHE (adaptive histogram equalization) method to enhance the contrast of the image. For CLAHE, the grid size (e.g. 8x8) and contrast limit need to be set to avoid over-enhancement. Contrast enhancement is applied to each frame of the filtered image to generate an enhanced image sequence. Comparing the images before and after enhancement, it is ensured that the details and layering are significantly improved, and the overall visual effect is significantly improved.

[0038] In this embodiment, the specific steps of step S14 are: Extracting a first frame image based on a filtered denoised image frame sequence; Performing average brightness calculation on the first frame of image to generate an average brightness value of the image; Performing brightness distribution analysis on the first frame of image to generate image brightness distribution data; Perform brightness area division processing based on image brightness distribution data to extract dark area image and bright area image of the current frame image; Performing brightness compensation enhancement on the dark region image based on the average brightness value of the image to generate a brightness enhanced dark region image; Performing global contrast deviation identification on the first frame image based on the brightness enhanced dark image to generate a contrast deviation value; Based on the contrast deviation value, the contrast of the brightness enhanced dark area image and the bright area image is optimized and adjusted, thereby constructing a global contrast optimized image; Each frame of image is extracted based on the sandbox scene image frame sequence, and the above contrast optimization adjustment process is iterated on each frame of image to obtain a global contrast optimized image sequence.

[0039] In this embodiment, a frame of image is randomly selected or designated from the filtered and denoised image frame sequence for subsequent processing to ensure that the selected frame can represent the overall scene characteristics. The selected image is converted into a grayscale image to make it easier to calculate the brightness. A programming tool (such as Python's NumPy library) is used to traverse the extracted image to calculate its average brightness value. The calculated average brightness value is stored in a variable for subsequent analysis. The distribution of the brightness value of each pixel in the image is calculated to generate a brightness histogram. Each bar of the histogram represents the number of pixels in a certain brightness range. The peak value, mean value and standard deviation in the brightness histogram are counted to understand the overall distribution of brightness. According to the brightness distribution data, the image is divided into a dark area, a bright area and an intermediate area. Generally, the dark area is a pixel with a brightness lower than a certain threshold, and the bright area is a pixel with a brightness higher than another threshold. The dark area image and the bright area image of the current frame image are extracted and stored in different variables respectively. A suitable brightness compensation algorithm is selected. For example, linear enhancement or gamma correction is used to perform brightness compensation enhancement on the dark area image according to the average brightness value to generate a dark image with enhanced brightness. The contrast deviation of the image is calculated using a formula. The deviation is identified by comparing the contrast of the enhanced dark area image with the original image. The contrast value is calculated, such as using a standard deviation or a histogram range to quantify the contrast. The calculated contrast deviation value is stored in a variable. Based on the contrast deviation value, the brightness enhanced dark image and the bright area image are adjusted. The contrast is stretched or compressed to achieve the best contrast of the overall image. The processed dark and bright areas are combined to construct a globally contrast optimized image. The above contrast optimization and adjustment processing steps are repeated for each frame in the sandbox scene image frame sequence after filtering and denoising to ensure that each frame undergoes the same brightness calculation, area division, brightness enhancement and contrast optimization. Finally, a new image sequence is obtained, in which each frame has undergone global contrast optimization processing to improve the overall visual effect.

[0040] Apply a Gaussian filter to the image sequence using a 5x5 convolution kernel.

[0041] import cv2 import numpy as np num_frames = 100 filtered_images = [] for i in range(num_frames): img = cv2.imread(f'frames / frame_{i}.png') filtered_img = cv2.GaussianBlur(img, (5, 5), 0) filtered_images.append(filtered_img) Select frame 1 for processing.

[0042] selected_frame = filtered_images[1]# Frame 1 Convert the image to grayscale and calculate the average brightness.

[0043] gray_frame = cv2.cvtColor(selected_frame, cv2.COLOR_BGR2GRAY) average_brightness = np.mean(gray_frame) print(f'Average Brightness: {average_brightness:.2f}') Use a histogram to calculate the brightness distribution of an image.

[0044] histogram, bins = np.histogram(gray_frame.flatten(), bins=256, range=[0, 256]) brightness_distribution = histogram.tolist() brightness_distribution = [0, 0, 1, 2, 5, ..., 200, 150, 50] # 256 values Use a threshold of 128 for cutting to extract the dark and bright areas.

[0045] dark_area_image = np.where(gray_frame<128, gray_frame, 0).astype(np.uint8) bright_area_image = np.where(gray_frame>= 128, gray_frame, 0).astype(np.uint8) Brighten the dark areas by 30.

[0046] enhanced_dark_image = np.clip(dark_area_image + 30, 0, 255).astype(np.uint8) Use standard deviation as a measure of contrast.

[0047] contrast_deviation = np.std(enhanced_dark_image) print(f'Contrast Deviation: {contrast_deviation:.2f}') Adjust the contrast of the image according to the contrast bias, assuming the adjustment factor is 1.5.

[0048] contrast_adjusted_image = cv2.convertScaleAbs(selected_frame, alpha=1.5, beta=0) The above contrast optimization adjustment process is applied to the entire image sequence.

[0049] optimized_images = [] for frame in filtered_images: gray_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) avg_brightness = np.mean(gray_frame) histogram, _ = np.histogram(gray_frame.flatten(), bins=256, range=[0,256]) dark_area_image = np.where(gray_frame<128, gray_frame, 0).astype(np.uint8) enhanced_dark_image = np.clip(dark_area_image + 30, 0, 255) contrast_deviation = np.std(enhanced_dark_image) optimized_frame = cv2.convertScaleAbs(frame, alpha=1.5, beta=0) optimized_images.append(optimized_frame).

[0050] In this embodiment, refer to Figure 3 , is a flowchart of detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include: Step S21: performing dynamic object recognition on the global contrast optimized image sequence and marking the dynamic objects; Step S22: segmenting the dynamic object into image frames, thereby obtaining dynamic object image frames; Step S23: Calculate the object space position of the dynamic object image frame to generate the dynamic object space position coordinates; Step S24: calculating the object motion change rate of consecutive frames for the dynamic object spatial position coordinates to generate the object position coordinate change rate; Step S25: performing continuous frame optical flow motion tracking on the global contrast optimized image sequence based on the object position coordinate change rate to generate an optical flow change trajectory of the dynamic object.

[0051] In this embodiment, a suitable dynamic object recognition algorithm is selected, such as background subtraction, optical flow method or deep learning model (such as YOLO, SSD, etc.), to ensure that the algorithm can process video sequences in real time and accurately identify dynamic objects, pre-process each frame of the image, such as scaling, normalization, etc., to meet the input requirements of the selected algorithm, perform dynamic object detection, mark the dynamic objects in the image, usually by drawing borders or outlines, save the dynamic object marking information recognized in each frame, provide data support for subsequent processing, and use image segmentation technology (such as Contour, BoundingBox, etc.) to extract the image frame of the dynamic object , calculate the bounding box of the dynamic object through the algorithm, get the smallest rectangular box to surround the object, record the image frame coordinate information of each dynamic object (such as the upper left corner and lower right corner coordinates) for subsequent position calculation, calculate the spatial position of the dynamic object in the image according to the coordinate information of the image frame, calculate the spatial position coordinates of the dynamic object (such as x, y, z coordinates) according to the coordinates of each image frame, calculate the center position of the object through the upper left corner and lower right corner coordinates of the bounding box, store the calculated spatial position coordinates of the dynamic object in the data frame, ensure that each coordinate is connected to the corresponding image frame number, and calculate the object position change rate between consecutive frames, change rate = , is the time interval, a is , b is , traverse the object space position coordinates of each pair of consecutive frames, calculate the change rate, store the generated object position coordinate change rate in the data frame, ensure that each change rate is connected to the corresponding frame number, select a suitable optical flow calculation method, such as the Lucas-Kanade method or the Horn-Schunck method, to track the movement of dynamic objects, perform optical flow calculation on consecutive frames, obtain the optical flow vector of dynamic objects by analyzing the pixel changes between adjacent frames, record the optical flow change trajectory of each dynamic object, usually use arrows or line segments to indicate the direction of movement, and draw the optical flow trajectory on the original image sequence.

[0052] In this embodiment, refer to Figure 4 , is a flowchart of detailed implementation steps of step S3. In this embodiment, the detailed implementation steps of step S3 include: Step S31: performing frame-by-frame scene visual semantic analysis on the global contrast optimized image sequence to extract scene ground and scene obstacle data; Step S32: Calculate the height and flatness of the scene ground to obtain the height and flatness data of the scene ground; Step S33: constructing a three-dimensional point cloud ground scene based on the scene ground height and flatness data and scene obstacle data to obtain a static three-dimensional scene model; Step S34: Use the optical flow change trajectory of the dynamic object to dynamically render the point cloud plane scene model, thereby constructing a dynamic scene twin model.

[0053] In this embodiment, a suitable visual semantic parsing algorithm is selected, such as a deep learning model (such as FCN, U-Net, DeepLab, etc.) to process the image sequence, and it is ensured that the model has been pre-trained on the relevant data set so as to accurately identify the ground and obstacles. Each frame of the image is pre-processed (such as normalization and scaling) for global contrast optimization, and a semantic segmentation model is used to infer each frame to extract the pixel areas of the ground and obstacles. The ground and obstacle data of each frame are saved as a binary mask image, and the positions of the ground and obstacles are marked in the mask to provide a basis for subsequent calculations. The ground contour is obtained using depth information (if any) or image processing technology (such as edge detection), and the height changes of the ground surface are calculated, usually using surface normals or height differences for analysis, and the ground height and flatness data are used to generate a three-dimensional point cloud, usually using laser scanning, stereo vision or The depth data obtained by the depth camera is combined with the ground and obstacle data to form a complete three-dimensional point cloud model. The generated point cloud is processed using a point cloud processing algorithm (such as the filtering and reconstruction algorithm in the PCL library) to remove noise and enhance the structure to generate a three-dimensional surface model. The point cloud is usually converted into a visual surface using a triangulated meshing method, and the extracted dynamic trajectory information is mapped to a static three-dimensional point cloud scene model. The motion path of the dynamic object is visualized on the model, and the dynamic trajectory is drawn using visualization tools (such as OpenGL, Unity, etc.). The movement of the object is usually represented in the form of a line segment or arrow. The static three-dimensional scene model is combined with the motion trajectory of the dynamic object to generate a dynamic scene twin model that can reflect the object movement and scene changes in real time, ensuring that the model can be dynamically rendered from different perspectives to provide an immersive user experience.

[0054] In this embodiment, the specific steps of step S34 are: Calculate the obstacle stereoscopic shape of the scene obstacle data to obtain obstacle shape parameters; Identify the spatial layout and position distribution of scene obstacle data and extract obstacle position distribution data; Based on the obstacle position distribution data, the scene layout structure analysis is performed on the global contrast optimization image sequence to generate the scene layout structure data; Based on the scene ground height and flatness data, the ground high and low points are sampled and fitted to generate the ground terrain feature grid; The three-dimensional point cloud ground scene is constructed based on the scene layout structure data based on the ground terrain feature grid to obtain a static three-dimensional scene model.

[0055] In this embodiment, the morphological parameters of the obstacle to be calculated, such as volume, surface area, aspect ratio, etc., are determined, the morphological parameters are calculated using a geometric model or a point cloud-based algorithm, the outer envelope of the obstacle is calculated to obtain the volume and surface area, the main morphological features of the obstacle are extracted, and each morphological parameter is calculated and recorded according to the extracted obstacle data, and the calculated obstacle morphological parameters are stored in a data frame to ensure that each parameter corresponds to the identification information of the obstacle, the coordinate information of the obstacle (usually extracted from the image frame) is used to identify the position of each obstacle in the scene, the coordinates of the center point of each obstacle are calculated, and its position in the scene is recorded, the obstacles are clustered (such as K-means clustering or DBSCAN), the spatial distribution pattern between the obstacles is identified, and obstacle position distribution data is generated, which usually includes the coordinates of the obstacles and their clustering information, the extracted obstacle position distribution data is saved in a structured data format (such as JSON or CSV) for subsequent analysis, the extracted obstacle position distribution data is used to analyze the spatial structure in the scene, the relative position relationship of the obstacles (such as distance, relative direction), and their impact on the scene layout is evaluated, and the scene layout structure data is created, including the obstacles. Relative position, quantity, type and other information are represented by graphical representation (such as graph theory) to form a structured scene layout diagram. The generated scene layout structure data is saved as a database or file for subsequent use and analysis. High and low points are extracted from the obtained high and low flatness data. High points and low points are usually selected for fitting. Key points on the ground are selected by uniform sampling or adaptive sampling methods. Interpolation algorithms (such as Kriging interpolation, spline interpolation or polynomial fitting) are used to fit the sampled high and low points to generate a ground terrain model and generate a feature grid. Ground features are usually represented in the form of a triangular grid or grid. The generated ground terrain feature grid is saved in a standard format (such as OBJ or PLY) for subsequent processing and visualization. The generated ground terrain feature grid and obstacle location distribution data are used to construct a 3D point cloud. The ground grid is combined with the obstacle data to form a complete 3D point cloud scene. The generated point cloud is processed using a point cloud processing algorithm (such as the filtering and reconstruction algorithm in the PCL library) to remove noise and mesh the generated point cloud to generate the final static 3D scene model. The constructed static 3D scene model is saved in a standard 3D file format (such as OBJ, FBX, etc.) for subsequent visualization and analysis.

[0056] In this embodiment, step S4 includes the following steps: Step S41: Identify robot position and posture parameters and user operation instructions; Step S42: Analyze the user's operation instruction for the user's set target point to obtain the user's set target point; Step S43: Calculate the target position of the target point set by the user to generate the moving position coordinates of the target point; Step S44: performing obstacle collision avoidance analysis on the moving position coordinates of the target point based on the dynamic scene twin model, and generating obstacle collision avoidance logic; Step S45: Perform dynamic avoidance path planning based on obstacle collision avoidance logic to generate a target movement avoidance path.

[0057] In this embodiment, the current position and posture (including position coordinates and orientation) of the robot are obtained through sensors (such as IMU, GPS, laser radar, etc.), and the sensor data are fused to improve the accuracy of the position and posture. Common methods include Kalman filtering, receiving user operation instructions through a user interface (such as a mobile application, a control panel or voice recognition), parsing the instructions into executable commands, such as "forward", "turn", etc., saving the recognized robot position and posture parameters and user operation instructions to system variables for subsequent processing, extracting target point information from the user operation instruction, identifying the specific position (for example: target coordinates) that the user wants the robot to reach, verifying whether the extracted target point is within the valid range, ensuring the legitimacy of the target point (for example, not exceeding the operating area), calculating the moving position coordinates of the target point according to the current robot position information and the target point set by the user, and calculating the moving position coordinates of the target point according to the current robot position information and the target point set by the user, the target position = current robot position + target point offset, using programming tools (such as Python ) to implement this calculation, and consider the robot motion model (such as differential drive or omnidirectional wheels) for coordinate transformation, store the calculated target point moving position coordinates in a data frame, use the dynamic scene twin model to identify obstacles on the target point moving path, extract the position and shape data of the obstacles, and perform intersection judgment through the obstacle position distribution data extracted in the previous step. Perform collision detection on each intermediate point to check whether it intersects with the obstacle, and use geometric methods (such as bounding box method or point-polygon collision detection) to judge the collision situation. If a collision is detected, generate avoidance logic, such as adjusting the path, changing the moving direction, or selecting an alternative path. Store the avoidance logic as a rule or state machine for subsequent path planning. According to the generated obstacle collision avoidance logic, adjust the target point moving path to ensure that it does not collide with any obstacles. Use a path planning algorithm (such as A* algorithm, Dijkstra algorithm) to recalculate the valid path, and generate a new target moving avoidance path for the adjusted path points, including the adjusted intermediate point coordinates, to ensure that the path is smooth and feasible, and avoid sharp turns or obstacles.

[0058] In this embodiment, the specific steps of step S5 are: Step S51: Perform robot dynamic behavior demand analysis on user operation instructions to extract dynamic behavior demand features; Step S52: performing dynamic constraint analysis on the robot position and posture parameters to generate posture dynamic constraint data; Step S53: performing temporal action sequence analysis on the dynamic behavior demand characteristics, thereby obtaining action demand sequences at multiple time points; Step S54: performing dynamic posture execution control on the action requirement sequences at multiple time points based on the posture dynamics constraint data, thereby generating a dynamic posture control sequence; Step S55: performing posture change jump smoothing on the dynamic posture control sequence to construct a posture smoothing control sequence.

[0059] In this embodiment, the user operation instructions are analyzed to identify the dynamic behaviors that the robot needs to perform. For example, the instructions include moving, rotating, grasping, etc. The instructions are analyzed using natural language processing (NLP) technology to extract the required characteristics of the dynamic behaviors, such as the target position, movement mode and speed, etc. The extracted dynamic behavior required characteristics are stored in a data frame to ensure that the behavior type, target position and other related parameters are included. According to the kinematic and dynamic characteristics of the robot, a corresponding dynamic model is established. Considering factors such as the mass, inertia, friction, etc. of the robot, the posture restriction conditions of the robot, such as the maximum tilt angle, movement speed and acceleration, etc., are calculated to ensure the stability of the robot during operation. The generated posture dynamic constraint data is stored in a data structure for subsequent analysis and control. According to the physical characteristics of the robot (such as mass, inertia, friction, etc.), a dynamic model is established. The Newton-Euler method or Lagrangian method is usually used for modeling. The robot's motion equations, including linear and angular motions, are defined. The posture constraints of the robot when performing dynamic behaviors, such as the maximum speed, acceleration and steering restrictions, are analyzed. The posture change range is determined to ensure that the physical extremes are not exceeded when the instructions are executed. Limits, select appropriate timing analysis methods (such as dynamic time warping, hidden Markov model, etc.) to process the extracted behavior demand features, map the dynamic behavior demand features to the time axis, generate action demand sequences at multiple time points, record the corresponding behavior features at each time point, such as speed and direction at time t1, acceleration at time t2, etc., select appropriate control algorithms (such as PID control, fuzzy control or reinforcement learning) to generate dynamic posture control sequences, calculate the control instructions at each time point based on the previously generated action demand sequence and posture dynamic constraint data, generate corresponding dynamic posture control sequences based on the output of the control algorithm, record the posture change instructions (such as speed, angle, etc.) at each time point, ensure that the generated control sequence can meet the dynamic behavior requirements and conform to the dynamic constraints, select appropriate smoothing algorithms (such as Bezier curves, moving averages, spline curves, etc.) to process the dynamic posture control sequences, ensure that the selected algorithms can effectively remove mutations and noise in the control sequences, apply smoothing algorithms to the dynamic posture control sequences, generate posture smoothing control sequences, ensure that the smoothed control sequences can smoothly transition to each posture change, and avoid sudden acceleration or deceleration.

[0060] Use sensors (such as IMU, LiDAR) to obtain the robot's current position (x, y, z) and attitude (roll, pitch, yaw).

[0061] robot_position = (5.0, 3.0, 0.0)# (x, y, z) robot_orientation = (0.0, 0.0, 1.57)# (roll, pitch, yaw) The user's operation instruction, such as "move to target point A", is recognized through a user interface (UI) or controller input.

[0062] user_command = "move to target A" Identify the target point information in the user's command.

[0063] Assume that the target point specified by the user is A and its position is (10.0, 5.0, 0.0).

[0064] target_point = (10.0, 5.0, 0.0) # Coordinates of target point A Calculate the moving direction and distance of the target point relative to the current robot position.

[0065] Using the simple Euclidean distance formula: import numpy as np def calculate_distance(start, end): return np.linalg.norm(np.array(end) - np.array(start)) distance_to_target = calculate_distance(robot_position, target_point) print(f"Distance to target: {distance_to_target:.2f} units") Use lidar or camera to identify obstacles in dynamic scenes. Assume that the detected obstacle position is (8.0, 4.0, 0.0).

[0066] obstacles = [(8.0, 4.0, 0.0)] # Assume that obstacles are detected Determine whether there are obstacles on the target path. Assume that the set safety distance is 1.5 units.

[0067] safe_distance = 1.5 collision_detected = any(calculate_distance(robot_position, obs)<safe_distance for obs in obstacles). Calculate the avoidance path, and simple avoidance strategies such as bypassing obstacles can be used.

[0068] Assume the calculated avoidance path is (10.0, 6.0, 0.0).

[0069] def plan_avoidance_path(target, obstacle): avoidance_target = (target[0], obstacle[1]+ safe_distance) # Bypass the obstacle return avoidance_target avoidance_path = plan_avoidance_path(target_point, obstacles[0]).

[0070] In this embodiment, the specific steps of step S6 are as follows: Step S61: Based on the target moving avoidance path and the attitude smoothing control sequence, perform robot movement simulation calculations on the dynamic scene twin model to obtain robot movement simulation response data; Step S62: Based on the scene ground height and flatness data, perform moving attitude tilt prediction on the robot movement simulation response data to generate attitude tilt prediction parameters; Step S63: Based on the attitude tilt prediction parameters, perform attitude tilt reverse compensation calculations to construct attitude compensation control parameters; Step S64: Perform optimal power cost calculations on the robot movement simulation response data to obtain the optimal power cost path; Step S65: Based on the attitude compensation control parameters and the optimal power cost path, perform iterative scene movement optimization to construct an intelligent attitude control strategy.

[0071] In this embodiment, it is ensured that the dynamic scene twin model has been constructed, including obstacles, ground features and other important elements, and that the target movement avoidance path and posture smooth control sequence are available. The kinematic model and the dynamic model are used to simulate the movement of the robot. A numerical simulation method (such as the Euler method or the Runge-Kutta method) is used to simulate the movement of the robot in the dynamic scene. The target movement avoidance path and the posture smooth control sequence are used as input, and the position and posture of the robot are gradually updated. The robot state is recorded at each time step, including information such as position, speed, acceleration and posture change, to form robot movement simulation response data. Based on the scene ground height and flatness data, the inclination and unevenness of the ground are analyzed, and the influence of ground features on the robot posture is identified. Based on the robot movement simulation response data, a posture tilt prediction model is established. Linear regression or machine learning methods are usually used to predict the posture tilt. The current posture, speed and ground features of the robot are considered to calculate the posture tilt prediction parameters. According to the posture tilt prediction parameters, the posture compensation required by the robot during movement is calculated. Inverse dynamics or Control theory methods (such as PID control) are used to compensate for posture tilt to ensure that the robot maintains balance. The calculated compensation amount is converted into posture compensation control parameters, such as adjusting the robot joint angle or power distribution, to ensure that the compensation parameters can be updated in real time in a dynamic environment. The power cost model of the robot's motion is defined, including factors such as energy consumption and acceleration change. The dynamic equation is usually used to describe the energy consumption of the robot in different states. The mobile simulation response data is analyzed using optimal control theory or dynamic programming algorithms (such as Dijkstra algorithm or A* algorithm) to calculate the optimal power cost path to ensure that the path can minimize the power cost while ensuring efficiency. Suitable optimization algorithms (such as genetic algorithms, particle swarm optimization, etc.) are selected for iterative optimization. Combined with posture compensation control parameters and optimal power cost path, the robot's movement path and posture control strategy in dynamic scenes are optimized through multiple iterations to ensure that it moves stably and efficiently in a dynamic environment. An intelligent posture control strategy is constructed based on the iterative results to ensure that the robot can adaptively adjust its posture in different environments and tasks to optimize its motion performance.

[0072] In this embodiment, a robot posture control system based on intelligent recognition is provided, which is used to execute the robot posture control method based on intelligent recognition as described above, including: The image processing module continuously acquires the real-time sandbox scene video stream based on the high-definition camera; performs global contrast enhancement processing on the real-time sandbox scene video stream to obtain a global contrast optimized image sequence; The object change trajectory module is used to perform continuous frame optical flow motion tracking on the global contrast optimized image sequence to generate the optical flow change trajectory of dynamic objects; The twin model module builds a dynamic scene twin model based on the optical flow change trajectory of dynamic objects; The path planning module identifies the robot's position and posture parameters and user operation instructions; it performs dynamic avoidance path planning based on the user's operation instructions, thereby generating a target movement avoidance path; The behavior requirement module performs dynamic posture execution control based on the robot's position and posture parameters to build a smooth posture control sequence; The intelligent attitude control module performs attitude tilt inverse compensation calculation on the dynamic scene twin model based on the target movement avoidance path and attitude smoothing control sequence, and constructs an intelligent attitude control strategy.

[0073] The present invention can realize real-time monitoring and perception of the scene by continuously acquiring real-time sandbox scene video streams, improve image quality and contrast through time-series frame segmentation and global contrast enhancement processing, more accurately identify objects and details in the scene, realize recognition and tracking of dynamic objects, effectively monitor the position and motion changes of objects in the scene, generate object change trajectories through optical flow motion tracking to analyze the motion behavior and trend of objects, extract scene information including ground and obstacle data based on the optimized image sequence, provide scene reference for subsequent path planning and behavior control, and build a dynamic scene twin model through dynamic trajectory rendering, so that the system can more accurately simulate dynamic changes in the scene and identify user operation instructions. , generate target movement avoidance paths, help the robot plan safe and effective movement paths, perform dynamic avoidance path planning based on user instructions and scene information, improve the robot's navigation ability in complex environments, perform dynamic posture execution control according to user operation instructions and robot position and posture parameters, so that the robot can meet the user's specific behavioral needs, build a posture smoothing control sequence, and improve the stability and efficiency of the robot's movement. Perform robot movement simulation calculations based on target movement avoidance paths and posture control sequences to help the system predict the robot's performance in dynamic scenes, and improve the robot's posture control effect and motion stability in complex environments through posture tilt inverse compensation calculations and intelligent posture control strategies.

[0074] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0075] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A robot posture control method based on intelligent recognition, characterized in that: The robot is embedded with a high-definition camera and includes the following steps: Step S1: continuously acquiring a real-time sandbox scene video stream based on a high-definition camera; performing global contrast enhancement processing on the real-time sandbox scene video stream to obtain a global contrast optimized image sequence; Step S2: performing continuous frame optical flow motion tracking on the global contrast optimized image sequence to generate the optical flow change trajectory of the dynamic object; Step S3: construct a dynamic scene twin model based on the optical flow change trajectory of dynamic objects; Step S4: identifying the robot position and posture parameters and the user operation instructions; performing dynamic avoidance path planning for the user operation instructions, thereby generating a target movement avoidance path; Step S5: Perform dynamic posture execution control based on the robot position and posture parameters to construct a posture smooth control sequence; Step S6: Based on the target movement avoidance path and the attitude smoothing control sequence, the attitude tilt reverse compensation calculation is performed on the dynamic scene twin model to construct an intelligent attitude control strategy.

2. The robot posture control method based on intelligent recognition according to claim 1 is characterized in that: The specific steps of step S1 are: Step S11: continuously acquiring a real-time sandbox scene video stream based on a high-definition camera; Step S12: performing time-sequential frame segmentation on the real-time sandbox scene video stream to extract a sandbox scene image frame sequence; Step S13: performing low-pass filtering and noise reduction on the sandbox scene image frame sequence frame by frame to obtain a filtered and noise-reduced image frame sequence; Step S14: performing global contrast enhancement processing on the filtered denoised image frame sequence, thereby obtaining a global contrast optimized image sequence.

3. The robot posture control method based on intelligent recognition according to claim 2 is characterized in that: The specific steps of step S14 are: Extracting a first frame image based on a filtered denoised image frame sequence; Performing average brightness calculation on the first frame of image to generate an average brightness value of the image; Performing brightness distribution analysis on the first frame of image to generate image brightness distribution data; Perform brightness area division processing based on image brightness distribution data to extract dark area image and bright area image of the current frame image; Performing brightness compensation enhancement on the dark region image based on the average brightness value of the image to generate a brightness enhanced dark region image; Performing global contrast deviation identification on the first frame image based on the brightness enhanced dark image to generate a contrast deviation value; Based on the contrast deviation value, the contrast of the brightness enhanced dark area image and the bright area image is optimized and adjusted, thereby constructing a global contrast optimized image; Each frame of image is extracted based on the sandbox scene image frame sequence, and the above contrast optimization adjustment process is iterated on each frame of image to obtain a global contrast optimized image sequence.

4. The robot posture control method based on intelligent recognition according to claim 1 is characterized in that: The specific steps of step S2 are: Step S21: performing dynamic object recognition on the global contrast optimized image sequence and marking the dynamic objects; Step S22: segmenting the dynamic object into image frames, thereby obtaining dynamic object image frames; Step S23: Calculate the object space position of the dynamic object image frame to generate the dynamic object space position coordinates; Step S24: calculating the object motion change rate of consecutive frames for the dynamic object spatial position coordinates to generate the object position coordinate change rate; Step S25: performing continuous frame optical flow motion tracking on the global contrast optimized image sequence based on the object position coordinate change rate to generate an optical flow change trajectory of the dynamic object.

5. The robot posture control method based on intelligent recognition according to claim 1 is characterized in that: The specific steps of step S3 are: Step S31: performing frame-by-frame scene visual semantic analysis on the global contrast optimized image sequence to extract scene ground and scene obstacle data; Step S32: Calculate the height and flatness of the scene ground to obtain the height and flatness data of the scene ground; Step S33: constructing a three-dimensional point cloud ground scene based on the scene ground height and flatness data and scene obstacle data to obtain a static three-dimensional scene model; Step S34: Use the optical flow change trajectory of the dynamic object to dynamically render the point cloud plane scene model, thereby constructing a dynamic scene twin model.

6. The robot posture control method based on intelligent recognition according to claim 5 is characterized in that: The specific steps of step S34 are: Calculate the obstacle morphology of the scene obstacle data to obtain obstacle morphology parameters; Identify the spatial layout and position distribution of scene obstacle data and extract obstacle position distribution data; Based on the obstacle position distribution data, the scene layout structure analysis is performed on the global contrast optimization image sequence to generate the scene layout structure data; Based on the scene ground height and flatness data, the ground high and low points are sampled and fitted to generate the ground terrain feature grid; The three-dimensional point cloud ground scene is constructed based on the scene layout structure data based on the ground terrain feature grid to obtain a static three-dimensional scene model.

7. The robot posture control method based on intelligent recognition according to claim 1 is characterized in that: The specific steps of step S4 are: Step S41: Identify robot position and posture parameters and user operation instructions; Step S42: Analyze the user's operation instruction for the user's set target point to obtain the user's set target point; Step S43: Calculate the target position of the target point set by the user to generate the moving position coordinates of the target point; Step S44: performing obstacle collision avoidance analysis on the moving position coordinates of the target point based on the dynamic scene twin model, and generating obstacle collision avoidance logic; Step S45: Perform dynamic avoidance path planning based on obstacle collision avoidance logic to generate a target movement avoidance path.

8. The robot posture control method based on intelligent recognition according to claim 1 is characterized in that: The specific steps of step S5 are: Step S51: Perform robot dynamic behavior demand analysis on user operation instructions to extract dynamic behavior demand features; Step S52: performing dynamic constraint analysis on the robot position and posture parameters to generate posture dynamic constraint data; Step S53: performing temporal action sequence analysis on the dynamic behavior demand characteristics, thereby obtaining action demand sequences at multiple time points; Step S54: performing dynamic posture execution control on the action requirement sequences at multiple time points based on the posture dynamics constraint data, thereby generating a dynamic posture control sequence; Step S55: performing posture change jump smoothing on the dynamic posture control sequence to construct a posture smoothing control sequence.

9. The robot posture control method based on intelligent recognition according to claim 1, characterized in that: The specific steps of step S6 are: Step S61: performing robot movement simulation calculation on the dynamic scene twin model based on the target movement avoidance path and the posture smoothing control sequence to obtain robot movement simulation response data; Step S62: Predicting the movement posture inclination of the robot movement simulation response data based on the scene ground height and flatness data to generate a posture inclination prediction parameter; Step S63: performing posture tilt reverse compensation calculation based on the posture tilt prediction parameters to construct posture compensation control parameters; Step S64: Calculate the optimal power cost of the robot movement simulation response data to obtain the optimal power cost path; Step S65: Perform iterative scene movement optimization based on the posture compensation control parameters and the optimal power cost path to build an intelligent posture control strategy.

10. A robot posture control system based on intelligent recognition, characterized in that: Used to execute the robot posture control method based on intelligent recognition as claimed in claim 1, comprising: The image processing module continuously acquires the real-time sandbox scene video stream based on the high-definition camera; performs global contrast enhancement processing on the real-time sandbox scene video stream to obtain a global contrast optimized image sequence; The object change trajectory module is used to track the continuous frame optical flow motion of the global contrast optimized image sequence to generate the optical flow change trajectory of the dynamic object; The twin model module builds a dynamic scene twin model based on the optical flow change trajectory of dynamic objects; The path planning module identifies the robot's position and posture parameters and user operation instructions; it performs dynamic avoidance path planning based on the user's operation instructions, thereby generating a target movement avoidance path; The behavior requirement module performs dynamic posture execution control based on the robot's position and posture parameters to build a smooth posture control sequence; The intelligent attitude control module performs attitude tilt inverse compensation calculation on the dynamic scene twin model based on the target movement avoidance path and attitude smoothing control sequence, and constructs an intelligent attitude control strategy.

Images