Obstacle-overcoming method, device, equipment and storage medium for artificial intelligence equipment

Abstract

The present invention provides an obstacle crossing and escape method, device, equipment and storage medium for an artificial intelligence device, the method comprising: when the artificial intelligence device is started, obtaining a multi-view image of the environment in which the artificial intelligence device is located, and performing neural radiation field processing on the multi-view image to obtain a three-dimensional environment representation of the environment; generating an obstacle map of the environment in which the artificial intelligence device is located according to the three-dimensional environment representation; when the artificial intelligence device is working, obtaining the operation data of the artificial intelligence device, and judging in real time whether the obstacle crossing and escape conditions are met according to the operation data; if so, generating an obstacle crossing strategy of the artificial intelligence device according to the operation data and the obstacle map, and generating an execution control command corresponding to the obstacle crossing strategy, and controlling the artificial intelligence device to complete the obstacle crossing and escape according to the execution control command. This method realizes efficient obstacle crossing and escape of artificial intelligence devices by combining neural radiation field technology and real-time data analysis, and significantly improves its operation efficiency and safety.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an obstacle-overcoming method, device, equipment and storage medium for an artificial intelligence device. Background Art

[0002] With the continuous advancement of artificial intelligence technology, artificial intelligence equipment is increasingly used in daily life and industrial production. For example, home service robots play an important role in cleaning, security, etc., and industrial automation equipment also plays a key role in material handling and assembly on production lines. However, these artificial intelligence devices often encounter various obstacles in actual operation, and how to effectively avoid and cross these obstacles has become an important research topic. At present, the obstacle escape of artificial intelligence equipment mainly adopts the method based on image processing. This method collects environmental images through cameras, and then uses image processing algorithms to identify obstacles and then plan obstacle escape paths. The advantage of this method is that the hardware cost is relatively low and the implementation is relatively simple. However, the image processing-based method can usually only process two-dimensional image information. In a complex three-dimensional environment, it is difficult to fully and accurately perceive the environmental structure relying solely on two-dimensional images, resulting in poor obstacle escape effects. Summary of the invention

[0003] The main purpose of the present invention is to solve the technical problem that the existing artificial intelligence equipment is difficult to fully and accurately perceive the environmental structure based on image processing when overcoming obstacles.

[0004] A first aspect of the present invention provides an obstacle-crossing and escaping method for an artificial intelligence device, the obstacle-crossing and escaping method for an artificial intelligence device is applied to an animal trap device, and the obstacle-crossing and escaping method for an artificial intelligence device comprises:

[0005] When the artificial intelligence device is started, a multi-view image of the environment in which the artificial intelligence device is located is obtained, and a neural radiation field processing is performed on the multi-view image to obtain a three-dimensional environment representation of the environment in which the artificial intelligence device is located;

[0006] Generate an obstacle map of the environment in which the artificial intelligence device is located based on the three-dimensional environment representation;

[0007] When the artificial intelligence device is working, the operation data of the artificial intelligence device is obtained, and whether the artificial intelligence device meets the obstacle escape condition is determined in real time according to the operation data;

[0008] If so, an obstacle crossing strategy of the artificial intelligence device is generated according to the operating data and the obstacle map, and an execution control command corresponding to the obstacle crossing strategy is generated, and the artificial intelligence device is controlled according to the execution control command to complete the obstacle crossing and escape.

[0009] Optionally, in a first implementation manner of the first aspect of the present invention, when the artificial intelligence device is started, acquiring a multi-view image of an environment in which the artificial intelligence device is located, and performing neural radiation field processing on the multi-view image to obtain a three-dimensional environment representation of the environment in which the artificial intelligence device is located includes:

[0010] Using multiple cameras to collect images of the environment around the artificial intelligence device from different angles and positions to obtain multi-view images;

[0011] Inputting the multi-view images into a pre-trained neural radiation field model to obtain a volume density field and a radiosity field of the environment in which the artificial intelligence device is located;

[0012] According to the volume density field and the radiosity field, the three-dimensional space of the environment is rapidly sampled and voxelized to obtain a discretized density field representation;

[0013] An optimized three-dimensional density gradient algorithm is applied to the discretized density field representation to generate three-dimensional point cloud data that highlights the object contour and surface features, and the three-dimensional point cloud data is processed using a real-time mesh reconstruction algorithm to obtain a three-dimensional environment representation model.

[0014] Optionally, in a second implementation of the first aspect of the present invention, generating an obstacle map of the environment in which the artificial intelligence device is located according to the three-dimensional environment representation includes:

[0015] Performing three-dimensional semantic segmentation processing on the three-dimensional environment representation to obtain a preliminary classification result, wherein the preliminary classification result includes ground, obstacles, and traversable areas;

[0016] According to the preliminary classification result, geometric feature analysis is performed on the identified obstacles to obtain feature data of the obstacles, and the feature data is processed by a cross-modal variational autoencoder to obtain a unified potential representation of the obstacles;

[0017] According to the unified potential representation, classify the attributes of the obstacles to obtain detailed obstacle information, wherein the detailed obstacle information includes obstacle type and physical characteristics;

[0018] The detailed obstacle information is combined with the three-dimensional environment representation to generate an obstacle map of the environment in which the artificial intelligence device is located.

[0019] Optionally, in a third implementation manner of the first aspect of the present invention, generating an obstacle crossing strategy for the artificial intelligence device according to the operating data and the obstacle map, generating an execution control command corresponding to the obstacle crossing strategy, and controlling the artificial intelligence device to complete obstacle crossing and escape according to the execution control command includes:

[0020] According to the operation data and the obstacle map, a global optimal path search tree algorithm and a graph convolutional network are jointly processed on the motion path of the artificial intelligence device to obtain a global obstacle crossing path;

[0021] Performing a fast expansion random tree algorithm on the global obstacle crossing path to obtain a local obstacle crossing path, and performing spline interpolation processing on the motion trajectory of the artificial intelligence device according to the local obstacle crossing path to obtain an obstacle crossing action sequence;

[0022] According to the obstacle crossing action sequence, the operation data of the artificial intelligence device is jointly processed by a cross-modal variational autoencoder and a deep Q network to obtain an obstacle crossing strategy of the artificial intelligence device;

[0023] Generate an execution control command corresponding to the obstacle surmounting strategy, and control the artificial intelligence device to complete the obstacle surmounting and escape according to the execution control command.

[0024] Optionally, in a fourth implementation of the first aspect of the present invention, performing a global optimal path search tree algorithm and a graph convolutional network joint processing on the motion path of the artificial intelligence device according to the operating data and the obstacle map to obtain a global obstacle crossing path includes:

[0025] Analyze the environment around the current position of the artificial intelligence device based on the operating data and the obstacle map to obtain specific obstacle information that needs to be overcome;

[0026] Using a pre-trained graph convolutional network model, the specific obstacle information is processed to obtain an obstacle feature representation;

[0027] Inputting the obstacle feature representation into a global optimal path search tree algorithm to generate multiple potential obstacle-crossing paths and obtain an initial path set;

[0028] Performing a local path tree search on each obstacle-crossing path in the initial path set to obtain a path search space, and evaluating and sorting the paths in the path search space according to a preset path evaluation criterion to obtain a global obstacle-crossing path candidate set;

[0029] An optimal path is selected from the global obstacle-crossing path candidate set, and a smoothing process is performed on the optimal path to obtain a global obstacle-crossing path.

[0030] Optionally, in a fifth implementation of the first aspect of the present invention, the performing of a fast expansion random tree algorithm on the global obstacle crossing path to obtain a local obstacle crossing path, and performing spline interpolation processing on the motion trajectory of the artificial intelligence device according to the local obstacle crossing path to obtain an obstacle crossing action sequence includes:

[0031] Extracting a path segment near the current position of the artificial intelligence device from the global obstacle-crossing path as a starting point and a target point for local path planning, and constructing an initial structure of a fast-expanding random tree with the starting point as a root node;

[0032] Randomly sampling feasible points in the obstacle map, connecting the feasible points to the nearest nodes in the initial structure to form new branches of the fast-expanding random tree, performing collision detection on the new branches, and removing branches that collide with obstacles;

[0033] Extracting an optimal path connecting a starting point and a target point from the generated fast expanding random tree to obtain a local obstacle crossing path, and performing spline interpolation processing on the local obstacle crossing path to obtain a continuous and smooth motion trajectory;

[0034] According to the continuous and smooth motion trajectory, the position sequence of the artificial intelligence device is calculated to obtain the obstacle crossing action sequence.

[0035] Optionally, in a sixth implementation of the first aspect of the present invention, performing cross-modal variational autoencoder and deep Q network joint processing on the operation data of the artificial intelligence device according to the obstacle crossing action sequence to obtain the obstacle crossing strategy of the artificial intelligence device includes:

[0036] The obstacle-crossing action sequence and the operation data of the artificial intelligence device are fused to obtain multimodal input data, and the multimodal input data is input into a cross-modal variational autoencoder to obtain a compressed latent space representation;

[0037] Sampling the latent space representation to obtain a low-dimensional feature vector containing environment and action information, and inputting the low-dimensional feature vector into a deep Q network to obtain Q-value estimates for different actions;

[0038] According to the Q value estimation, an ε-greedy strategy is used to select an action to obtain a preliminary obstacle crossing strategy, and the preliminary obstacle crossing strategy is subjected to a safety check and constraint processing to obtain the obstacle crossing strategy of the artificial intelligence device.

[0039] A second aspect of the present invention provides an obstacle-crossing and escape-from-predicament device for an artificial intelligence device, the obstacle-crossing and escape-from-predicament device for an artificial intelligence device comprising:

[0040] An environment image processing module, used for acquiring multi-view images of the environment in which the artificial intelligence device is located when the artificial intelligence device is started, and performing neural radiation field processing on the multi-view images to obtain a three-dimensional environment representation of the environment in which the artificial intelligence device is located;

[0041] An obstacle map generation module, configured to generate an obstacle map of the environment in which the artificial intelligence device is located according to the three-dimensional environment representation;

[0042] A judgment module, used for obtaining the operation data of the artificial intelligence device when the artificial intelligence device is working, and judging in real time whether the artificial intelligence device meets the obstacle escape condition according to the operation data;

[0043] The obstacle crossing control module is used to generate an obstacle crossing strategy for the artificial intelligence device according to the operating data and the obstacle map if the artificial intelligence device meets the obstacle crossing and escape conditions, and to generate an execution control command corresponding to the obstacle crossing strategy, and to control the artificial intelligence device to complete the obstacle crossing and escape according to the execution control command.

[0044] A third aspect of the present invention provides an obstacle-crossing and escape device for an artificial intelligence device, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via lines; the at least one processor calls the instructions in the memory to enable the obstacle-crossing and escape device for the artificial intelligence device to execute the steps of the above-mentioned obstacle-crossing and escape method for the artificial intelligence device.

[0045] A fourth aspect of the present invention provides a computer-readable storage medium, which stores instructions that, when executed on a computer, enable the computer to execute the steps of the above-mentioned obstacle escape method of an artificial intelligence device.

[0046] The obstacle crossing and escape method, device, equipment and storage medium of the above-mentioned artificial intelligence device, when the artificial intelligence device is started, obtains multi-view images of the environment in which the artificial intelligence device is located, and performs neural radiation field processing on the multi-view images to obtain a three-dimensional environmental representation of the environment; generates an obstacle map of the environment in which the artificial intelligence device is located based on the three-dimensional environmental representation; when the artificial intelligence device is working, obtains the operation data of the artificial intelligence device, and judges in real time whether the obstacle crossing and escape conditions are met based on the operation data; if so, generates an obstacle crossing strategy for the artificial intelligence device based on the operation data and the obstacle map, and generates an execution control command corresponding to the obstacle crossing strategy, and controls the artificial intelligence device to complete the obstacle crossing and escape according to the execution control command. This method realizes efficient obstacle crossing and escape of artificial intelligence equipment by combining neural radiation field technology and real-time data analysis, and significantly improves its operation efficiency and safety.

[0047] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a schematic diagram of a first embodiment of an obstacle escaping method for an artificial intelligence device in an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of an embodiment of an obstacle escaping device of an artificial intelligence device in an embodiment of the present invention;

[0051] Figure 3 It is a schematic diagram of an embodiment of an obstacle escape device of an artificial intelligence device in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device end including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or device ends.

[0054] To facilitate understanding of this embodiment, a method for escaping from obstacles by an artificial intelligence device disclosed in an embodiment of the present invention is first introduced in detail. The method for escaping from obstacles by an artificial intelligence device is applied to an animal trap device. Figure 1 As shown, the method comprises the following steps:

[0055] 101. When the artificial intelligence device is started, a multi-view image of the environment in which the artificial intelligence device is located is obtained, and a neural radiation field processing is performed on the multi-view image to obtain a three-dimensional environment representation of the environment in which the artificial intelligence device is located;

[0056] In one embodiment of the present invention, when the artificial intelligence device is started, multi-perspective images of the environment in which the artificial intelligence device is located are obtained, and neural radiation field processing is performed on the multi-perspective images to obtain a three-dimensional environmental representation of the environment in which the artificial intelligence device is located, including: using multiple cameras to collect images of the environment around the artificial intelligence device from different angles and positions to obtain multi-perspective images; inputting the multi-perspective images into a pre-trained neural radiation field model to obtain a volume density field and a radiosity field of the environment in which the artificial intelligence device is located; based on the volume density field and the radiosity field, the three-dimensional space of the environment is quickly sampled and voxelized to obtain a discretized density field representation; applying an optimized three-dimensional density gradient algorithm to the discretized density field representation to generate three-dimensional point cloud data that highlights the contours and surface features of the object, and processing the three-dimensional point cloud data using a real-time mesh reconstruction algorithm to obtain a three-dimensional environmental representation model.

[0057] Specifically, when the AI ​​device is started, it first collects images of the surrounding environment from different angles and positions through multiple high-resolution cameras. These cameras are usually installed in different parts of the device to ensure that the environmental information can be captured in all directions. Next, these multi-view images are input into the pre-trained neural radiation field model. This model is a complex neural network that has been trained with a large amount of data and can understand the three-dimensional information implicit in the two-dimensional image. The core task of the model is to convert the two-dimensional image information into a three-dimensional spatial representation. It infers the volume density and radiation characteristics of each point in the space by analyzing images taken from different angles. In indoor environments, this step can effectively handle complex lighting conditions and object occlusion problems. For example, even if part of the furniture is blocked by other objects, the model can infer the approximate shape and position of the blocked part by analyzing the visible part and the surrounding environment. The output of the neural radiation field model is the volume density field and radiance field of the environment. The volume density field describes the "solidity" of each point in the space, while the radiance field describes the color and brightness information of each point. These two fields together constitute a complete three-dimensional representation of the environment. In indoor scenes, this representation method can accurately capture the shape and material characteristics of objects such as walls and furniture, and can even handle complex reflective surfaces such as glass and mirrors. However, this continuous field representation is not directly suitable for subsequent processing steps. Therefore, the next step is to quickly sample and voxelize these fields to convert the continuous representation into a discrete form that is easier for computers to process. This process is like dividing the entire space into countless small cubes (voxels), each of which stores the density and color information of that location. In indoor environments, this representation method can effectively capture detailed features such as the edges and corners of furniture and the texture of the wall. In order to further extract key features in the environment, the technical solution uses an optimized 3D density gradient algorithm to process the discretized density field. This algorithm can identify areas with drastic density changes, which usually correspond to the edges and surfaces of objects. Through this step, we get a set of 3D point cloud data, which accurately describes the contours and surface features of objects in the environment. In indoor scenes, this method can clearly outline the shape of furniture, the layout of the room, and even identify the location of small objects such as door handles and switches. Finally, in order to obtain a complete and visual 3D environment model, the technical solution uses a real-time mesh reconstruction algorithm to process the 3D point cloud data. The task of this algorithm is to connect discrete points into a continuous surface to form a complete 3D mesh model.

[0058] 102. Generate an obstacle map of the environment in which the artificial intelligence device is located based on the three-dimensional environment representation;

[0059] In one embodiment of the present invention, generating an obstacle map of the environment in which the artificial intelligence device is located based on the three-dimensional environment representation includes: performing three-dimensional semantic segmentation processing on the three-dimensional environment representation to obtain a preliminary classification result, wherein the preliminary classification result includes the ground, obstacles and passable areas; based on the preliminary classification result, performing geometric feature analysis on the identified obstacles to obtain feature data of the obstacles, and performing cross-modal variational autoencoder processing on the feature data to obtain a unified potential representation of the obstacles; based on the unified potential representation, performing attribute classification on the obstacles to obtain detailed obstacle information, wherein the detailed obstacle information includes obstacle type and physical properties; combining the detailed obstacle information with the three-dimensional environment representation to generate an obstacle map of the environment in which the artificial intelligence device is located.

[0060] Specifically, first, the system performs 3D semantic segmentation on the 3D environment representation. This step uses deep learning algorithms, such as 3D convolutional neural networks, to classify each point or voxel in the environment. The main categories of classification include ground, obstacles, and passable areas. In indoor environments, this step can effectively distinguish different elements such as floors, walls, and furniture. For example, in a typical living room scene, the system can accurately identify the floor as the ground, sofas, tables, cabinets, etc. as obstacles, and open areas are marked as passable areas. Based on the preliminary classification results, the system performs in-depth geometric feature analysis on the identified obstacles. This step involves calculating the geometric properties of each obstacle, such as size, shape, and direction. Various computer vision algorithms, such as edge detection and shape matching, are used in the analysis process. For obstacles in indoor environments, the system not only calculates their basic dimensions such as height, width, and depth, but also analyzes their shape features, such as whether they are regular geometric bodies and whether they have complex curved surfaces. In order to more comprehensively understand the characteristics of obstacles, the system processes these feature data using cross-modal variational autoencoders. The purpose of this step is to fuse data from different sensors or different feature spaces into a unified latent representation. For example, in addition to geometric features, the system may also consider visual features such as color and material of obstacles, and may even include data from other sensors (such as thermal imaging, sound waves). Through cross-modal variational autoencoders, these heterogeneous data are compressed and fused into a low-dimensional latent space to form a unified representation of obstacles. Based on this unified latent representation, the system performs a more detailed classification of obstacle attributes. This step uses machine learning algorithms, such as support vector machines or deep neural networks, to perform more detailed classification and feature analysis of obstacles. The classification results include not only the type of obstacles (such as tables, chairs, cabinets, etc.), but also their physical properties, such as material (wood, metal, glass, etc.), mobility (fixed, movable, partially movable), etc. Finally, the system combines this detailed obstacle information with the original 3D environment representation to generate the final obstacle map. This map not only contains the geometric layout of the environment, but also annotates the location, size, type and physical properties of each obstacle. In an indoor scenario, such a map might look like an augmented reality version of a room floor plan, with each piece of furniture precisely located and accompanied by detailed attribute information.

[0061] 103. When the artificial intelligence device is working, obtain the operating data of the artificial intelligence device, and determine in real time based on the operating data whether the artificial intelligence device meets the obstacle escape conditions;

[0062] In one embodiment of the present invention, in a static environment, even if there is an accurate obstacle map, the artificial intelligence device still needs to obtain operating data in real time and determine whether the obstacle escape conditions are met when working. This is because even if the environment does not change, the state and performance of the device itself may change, or the device may encounter special situations that cannot be completely avoided by pre-planning. Specifically, the device continuously collects operating data through internal sensors, including gyroscope data, accelerometer data, motor encoder data, etc. These data reflect the current state of the device, such as posture, speed, power output, etc. The central processing unit analyzes these data in real time and compares them with the expected normal working parameters. In a static environment, the judgment of obstacle escape conditions is mainly based on the abnormality of the device's own state. For example, if the device detects an abnormal tilt angle in an area marked as a flat area in the obstacle map, this may mean that the device wheels are stuck in the folds of the carpet or the gaps in the floor. Alternatively, if the device finds that its actual position during movement has a significant deviation from the path planned according to the obstacle map, this may indicate that the device's positioning system has an error and needs to be recalibrated. Another situation is that even if the environment is static, the device may enter some locations that are difficult to fully plan to avoid in advance when performing complex tasks. For example, when performing delicate operations such as grasping an object, the device may need to enter a small space, and the process of getting out may require a special obstacle-crossing strategy. When determining whether the obstacle-crossing and escape conditions are met, the system will comprehensively consider multiple factors: whether the current state of the device is abnormal (such as whether it is stuck or trapped in repetitive actions), whether the actual motion trajectory deviates significantly from the planned path, and whether the current position is in a potential difficult area marked in the obstacle map. For example, if the device finds that it cannot move as expected in an open area marked in the obstacle map, the system will determine that the current situation may require the initiation of an obstacle-crossing and escape procedure.

[0063] 104. If so, an obstacle crossing strategy for the artificial intelligence device is generated based on the operating data and the obstacle map, and an execution control command corresponding to the obstacle crossing strategy is generated, and the artificial intelligence device is controlled according to the execution control command to complete the obstacle crossing and escape.

[0064] In one embodiment of the present invention, the obstacle crossing strategy of the artificial intelligence device is generated according to the operating data and the obstacle map, and an execution control command corresponding to the obstacle crossing strategy is generated, and the artificial intelligence device is controlled to complete obstacle crossing and escape according to the execution control command, including: according to the operating data and the obstacle map, the motion path of the artificial intelligence device is jointly processed by the global optimal path search tree algorithm and the graph convolutional network to obtain a global obstacle crossing path; the global obstacle crossing path is processed by the fast expansion random tree algorithm to obtain a local obstacle crossing path, and the motion trajectory of the artificial intelligence device is processed by spline interpolation according to the local obstacle crossing path to obtain an obstacle crossing action sequence; according to the obstacle crossing action sequence, the operating data of the artificial intelligence device is jointly processed by a cross-modal variational autoencoder and a deep Q network to obtain the obstacle crossing strategy of the artificial intelligence device; the execution control command corresponding to the obstacle crossing strategy is generated, and the artificial intelligence device is controlled to complete obstacle crossing and escape according to the execution control command.

[0065] Specifically, first, the system will combine the current operating data and the pre-generated obstacle map, and use the joint processing method of the global optimal path search tree algorithm and the graph convolutional network to plan a global obstacle crossing path. This process takes into account the overall structure of the environment and the current state of the device, laying the foundation for subsequent local path planning. Then, the system will further refine this global path. By rapidly expanding the random tree algorithm, the system can generate a more refined local obstacle crossing path under the framework of the global path. This local path takes into account more detailed factors, such as the kinematic constraints of the device and the precise location of obstacles. In order to make this path smoother and more executable, the system will perform spline interpolation processing on it to obtain a series of continuous obstacle crossing action sequences. Finally, the system will input this action sequence into a complex model that combines a cross-modal variational autoencoder and a deep Q network. This model can comprehensively consider the operating data and action sequence of the device to generate the final obstacle crossing strategy. This strategy not only contains specific action instructions, but also considers the possible consequences and benefits of executing these actions. Based on the generated obstacle crossing strategies, the system will further refine and transform these strategies to generate a series of specific execution control commands. These commands are sets of instructions that can be directly understood and executed by the device's execution unit. For example, in a scenario where a low obstacle needs to be overcome, these commands may include specific instructions such as "raise the front wheel 20 degrees", "increase the rear wheel power output by 30%", and "maintain the current posture for 10 seconds". Finally, these execution control commands will be transmitted to the various execution units of the device, such as the motor controller, posture adjustment mechanism, etc. Each execution unit will adjust its working state according to the received commands. For example, the motor controller responsible for the wheel power output will adjust the motor speed and torque according to the command, while the posture adjustment mechanism will control the tilt angle of the device body. The collaborative work of these execution units enables the device to accurately perform obstacle crossing actions.

[0066] Furthermore, the method of performing a global optimal path search tree algorithm and a graph convolutional network joint processing on the movement path of the artificial intelligence device according to the operation data and the obstacle map to obtain a global obstacle crossing path includes: analyzing the environment around the current position of the artificial intelligence device according to the operation data and the obstacle map to obtain specific obstacle information that needs to be overcome; using a pre-trained graph convolutional network model to process the specific obstacle information to obtain an obstacle feature representation; inputting the obstacle feature representation into the global optimal path search tree algorithm to generate multiple potential obstacle crossing paths to obtain an initial path set; performing a local path tree search on each obstacle crossing path in the initial path set to obtain a path search space, and evaluating and sorting the paths in the path search space according to a preset path evaluation standard to obtain a global obstacle crossing path candidate set; selecting the best path from the global obstacle crossing path candidate set, and smoothing the best path to obtain a global obstacle crossing path.

[0067] Specifically, when an AI device faces an obstacle escape situation, the system first needs to conduct a comprehensive analysis of the environment around the device based on the current operating data and the pre-generated obstacle map. Specifically, the system uses a spatial indexing algorithm (such as an R-tree or an octree) to quickly locate obstacles around the device, and combines the device's position, orientation, and motion state information to build a local environment model. In this model, each obstacle is represented as a geometric entity, containing its attributes such as position, size, and shape. Next, the system calls a pre-trained graph convolutional network (GCN) model to process the obstacle information. The core advantage of the GCN model is that it can effectively process non-Euclidean data, such as graph structure data. Here, obstacles and their relationships can be represented as a graph, where nodes represent obstacles and edges represent the spatial relationships between obstacles. The GCN model can capture the complex interactions between obstacles through multi-layer convolution operations. For example, it may recognize that a narrow passage is formed between a table and a chair. The output of the model is a high-dimensional feature vector, which not only encodes the characteristics of each obstacle, but also contains information about the spatial relationship between them. The next key step is to input these GCN-processed obstacle feature representations into the global optimal path search tree algorithm. The algorithm used here may be a modified version of the Rapid Exploration Random Tree (RRT*) algorithm, which adds rewiring and path optimization functions on the basis of the standard RRT. The algorithm first randomly samples a series of points in the configuration space, which represent the possible positions and postures of the device. Then, the algorithm tries to connect these points to form a tree structure covering the entire feasible space. In this process, the obstacle feature information provided by GCN is used to evaluate the feasibility and cost of each connection. For example, the algorithm may prioritize passing through the gaps between furniture instead of trying to bypass the entire furniture group. In this way, the algorithm can generate multiple potential obstacle-crossing paths in a short time to form an initial path set. To further optimize these paths, the system performs a local path tree search for each path in the initial path set. This step uses the Local RRT algorithm, which performs local sampling and optimization around each path segment. This process takes into account more details, such as the kinematic constraints (minimum turning radius, maximum acceleration, etc.) and dynamic characteristics of the device. For example, for a wheeled robot, the algorithm takes into account the characteristics of its differential drive to ensure that the generated path is executable. The output of this step is a richer and more detailed path search space, containing multiple local variations of the original path. Next, the system evaluates and ranks all paths in the search space using a preset path evaluation criterion. This evaluation process involves multi-objective optimization, taking into account factors such as path length, energy consumption, safety (minimum distance to obstacles), smoothness, etc. The evaluation uses a weighted sum method, with each factor having a pre-defined weight.The system calculates a comprehensive score for each path and uses a quick sort algorithm to sort all the paths, ultimately obtaining a global obstacle-crossing path candidate set. Selecting the best path from this candidate set is the last critical step. Greedy algorithms or dynamic programming methods may be used here to make the final decision. The selection criteria not only consider the comprehensive score of the path, but also the reliability and robustness of the path. For example, the system may give priority to a path that is not the shortest but has a wider safety margin. Finally, smoothing the selected best path is an important step to ensure the executable nature of the path. Bezier curves or spline interpolation methods may be used here to smooth sharp turns or discontinuities in the path. The smoothing process takes into account the motion characteristics of the device and ensures that the generated path meets the motion constraints of the device. For example, for a robot with a minimum turning radius limit, the smoothing algorithm ensures that the curvature of all turns does not exceed this limit.

[0068] Furthermore, the global obstacle crossing path is processed by a rapid expansion random tree algorithm to obtain a local obstacle crossing path, and the motion trajectory of the artificial intelligence device is processed by spline interpolation according to the local obstacle crossing path to obtain an obstacle crossing action sequence, including: extracting a path segment near the current position of the artificial intelligence device from the global obstacle crossing path as the starting point and target point of the local path planning, and constructing an initial structure of a rapid expansion random tree with the starting point as the root node; randomly sampling feasible points in the obstacle map, connecting the feasible points to the nearest nodes in the initial structure to form new branches of the rapid expansion random tree, and performing collision detection on the new branches to remove branches that collide with obstacles; extracting the optimal path connecting the starting point and the target point from the generated rapid expansion random tree to obtain a local obstacle crossing path, and performing spline interpolation on the local obstacle crossing path to obtain a continuous and smooth motion trajectory; according to the continuous and smooth motion trajectory, calculating the position sequence of the artificial intelligence device to obtain an obstacle crossing action sequence.

[0069] Specifically, the system needs to extract the path segment near the current position of the device from the global obstacle path. This process usually uses a sliding window technique, and the window size may be 1.5 to 2 times the device's perception range. For example, if the device's lidar scanning range is 10 meters, the extracted path segment may be 15-20 meters long. The starting point and end point of this path segment are used as the starting point and target point of local path planning, respectively. Next, the system builds the initial structure of RRT with the starting point as the root node. This initial structure is usually a tree containing only the root node. The core idea of ​​the RRT algorithm is to quickly explore high-dimensional space through random sampling, which is particularly suitable for handling path planning problems with motion constraints. In the implementation process, the system uses efficient space partitioning data structures, such as kd trees or quadtrees, to speed up the search for nearest neighbors. Subsequently, the system performs random sampling in the obstacle map to generate feasible points. The "random" here is usually not purely uniform random, but some heuristic strategies are used to improve sampling efficiency. For example, the system may use an adaptive sampling strategy to increase the sampling density in the area between the starting point and the end point of the path, or increase sampling points near the edge of the obstacle. Another commonly used technique is target biased sampling, that is, there is a certain probability to directly sample the target point, which helps to accelerate the convergence of the algorithm to the target direction. Whenever a new sample point is generated, the system will try to connect it to the nearest node in the tree to form a new branch. This connection process involves the consideration of kinematic constraints. For example, for a differentially driven robot, the path connecting two points must meet the minimum turning radius requirement. The system may use Bezier curves or Dubins paths to generate connection paths that meet the constraints. Collision detection of newly generated branches is a key step to ensure the feasibility of the path. This is usually achieved by discretizing the path into a series of dense points and then checking whether each point collides with obstacles. In order to improve efficiency, a hierarchical collision detection strategy can be used, which first performs a rough detection, and only the paths that pass the rough detection are finely detected. Collision detection also needs to consider the actual shape and size of the device, and the device outline is usually expanded to leave a safety margin. Repeat the above sampling, connection and detection process until the tree reaches the preset number of nodes or a path to the target point is found. Then, the system extracts the optimal path connecting the start point and the target point from the generated RRT. This is usually achieved through a backtracking algorithm, starting from the target point and tracing back along the parent node link to the starting point to form a complete path. If there are multiple feasible paths, the system will select the best one based on path length, smoothness and other indicators. After obtaining the local obstacle crossing path, the next step is to perform spline interpolation on it to obtain a continuous and smooth motion trajectory. Commonly used techniques here include cubic spline interpolation or B-spline interpolation. The interpolation process needs to take into account the kinematic characteristics of the device to ensure that the generated trajectory meets constraints such as maximum curvature and maximum acceleration at any position.For example, for a wheeled robot, it may be necessary to ensure that the curvature of the trajectory does not exceed the inverse of its minimum turning radius at any point. Finally, the system needs to calculate a series of discrete position points based on this continuous and smooth motion trajectory to form an obstacle-crossing action sequence. This process usually involves equal-time sampling or equal-arc-length sampling of the trajectory. The sampling density needs to balance control accuracy and computational load, and it may usually be chosen that the device performs a position update every 0.1 to 0.5 seconds. For each sampling point, the system needs to calculate not only the position coordinates, but also the dynamic parameters such as velocity and acceleration so that the subsequent motion control system can be executed accurately.

[0070] Furthermore, according to the obstacle crossing action sequence, the operating data of the artificial intelligence device is jointly processed by a cross-modal variational autoencoder and a deep Q network to obtain the obstacle crossing strategy of the artificial intelligence device, including: fusing the obstacle crossing action sequence and the operating data of the artificial intelligence device to obtain multimodal input data, and inputting the multimodal input data into the cross-modal variational autoencoder to obtain a compressed latent space representation; sampling the latent space representation to obtain a low-dimensional feature vector containing environment and action information, and inputting the low-dimensional feature vector into a deep Q network to obtain Q value estimates of different actions; according to the Q value estimate, using the ε-greedy strategy to select an action to obtain a preliminary obstacle crossing strategy, and performing safety checks and constraint processing on the preliminary obstacle crossing strategy to obtain the obstacle crossing strategy of the artificial intelligence device.

[0071] Specifically, it first involves data fusion technology, which integrates information from different sources into a unified multimodal input. Specifically, the obstacle-crossing action sequence usually contains a series of predetermined position, speed and direction information, while the equipment operation data may include real-time status information such as current posture, motor load, sensor readings, etc. The fusion of these heterogeneous data usually adopts feature-level fusion methods, such as using a multi-layer perceptron (MLP) network to map the features of different modalities to a common feature space, or using an attention mechanism to dynamically adjust the importance weights of different modal data. The fused multimodal data is then input into a cross-modal variational autoencoder (CM-VAE). CM-VAE is an extended version of the variational autoencoder (VAE) specifically designed to process multimodal data. The core idea is to learn a latent space representation that can capture the common information of multiple modalities. In the implementation process, CM-VAE usually contains multiple encoders and decoders, one pair for each modality. The encoder maps the input data to the latent space, while the decoder attempts to reconstruct the original input from the latent representation. The training process uses a variational inference method to minimize the reconstruction error and the KL divergence between the latent space distribution and the prior distribution (usually a standard normal distribution). This method is able to learn a compact representation that contains both shared information and modality-specific information. Sampling from the latent space of CM-VAE is the next key step. Since the latent space of VAE is continuous, we can directly sample from the learned posterior distribution. This process usually uses a reparameterization trick to ensure the back propagation of gradients. The sampled vector is a low-dimensional feature representation that encodes information about the state of the environment and possible actions in a highly compressed and abstract way. This low-dimensional feature vector is then input into a deep Q network (DQN). DQN is an important algorithm in the field of reinforcement learning that combines Q learning with deep neural networks. In implementation, DQN is usually a multi-layer fully connected neural network whose input is the state representation (here, the sampled output of CM-VAE) and whose output is the Q value estimate for each possible action. The training of the network uses techniques such as experience replay and target network to improve the stability and efficiency of learning. Specifically, the system maintains an experience pool that stores past state-action-reward-next-state tuples and randomly samples from them for batch learning. At the same time, a periodically updated target network is used to calculate the temporal difference (TD) target, which helps reduce the instability of the Q-value estimate. After obtaining the Q-value estimate, the system uses the ε-greedy strategy to select actions. This strategy selects the action with the highest Q value most of the time, but retains a small probability (ε) to randomly select actions to balance exploration and utilization. The value of ε usually decays over time, causing the system to explore more in the early stages and tend to exploit the learned knowledge in the later stages. This process outputs a preliminary obstacle crossing strategy, which is a series of optimal action sequences based on the current state.However, directly executing this preliminary strategy may pose a safety hazard, so safety checks and constraint processing are required. This process may include collision detection, stability analysis, and kinematic constraint checks. For example, for an indoor service robot, the system checks whether each action may cause a collision with furniture, whether it will put the robot into an unstable state (such as a large tilt angle), and whether it violates the robot's speed and acceleration limits. These checks usually use fast approximation algorithms, such as space partitioning techniques (such as octrees) to accelerate collision detection, and simplified dynamic models to quickly evaluate stability. If some actions in the preliminary strategy are judged to be unsafe or infeasible, the system will try to correct these actions. Correction methods may include replacing unsafe actions with the closest safe actions, or inserting additional transition actions at unsafe points to ensure safety. In some cases, if a safe alternative cannot be found, the system may fall back to a more conservative strategy, such as slowing down or pausing to wait for further instructions. Through this series of complex processing steps, the system finally obtains an obstacle crossing strategy that takes into account both the environmental state and the task objectives and meets the safety and feasibility requirements. This strategy not only contains specific action instructions, but also implies adaptability to different situations. For example, for a robot that needs to pass through a narrow passage, the final obstacle crossing strategy may instruct it to first slow down and approach the entrance of the passage, and then precisely control its posture and speed to pass slowly, while adjusting in real time to deal with possible slippage or posture deviation.

[0072] In this embodiment, when the artificial intelligence device is started, a multi-view image of the environment in which the artificial intelligence device is located is obtained, and the multi-view image is processed by neural radiation field to obtain a three-dimensional environment representation of the environment; an obstacle map of the environment in which the artificial intelligence device is located is generated according to the three-dimensional environment representation; when the artificial intelligence device is working, the operation data of the artificial intelligence device is obtained, and it is judged in real time whether the obstacle crossing and escape conditions are met according to the operation data; if so, an obstacle crossing strategy of the artificial intelligence device is generated according to the operation data and the obstacle map, and an execution control command corresponding to the obstacle crossing strategy is generated, and the artificial intelligence device is controlled according to the execution control command to complete the obstacle crossing and escape. This method realizes efficient obstacle crossing and escape of artificial intelligence equipment by combining neural radiation field technology and real-time data analysis, and significantly improves its operation efficiency and safety.

[0073] The above describes the obstacle-overcoming and escape method of the artificial intelligence device in the embodiment of the present invention. The following describes the obstacle-overcoming and escape device of the artificial intelligence device in the embodiment of the present invention. Figure 2 , an embodiment of the obstacle escape device of the artificial intelligence device in the embodiment of the present invention includes:

[0074] The environment image processing module 201 is used to obtain multi-view images of the environment in which the artificial intelligence device is located when the artificial intelligence device is started, and perform neural radiation field processing on the multi-view images to obtain a three-dimensional environment representation of the environment in which the artificial intelligence device is located;

[0075] An obstacle map generation module 202, configured to generate an obstacle map of the environment in which the artificial intelligence device is located according to the three-dimensional environment representation;

[0076] The judgment module 203 is used to obtain the operation data of the artificial intelligence device when the artificial intelligence device is working, and judge in real time whether the artificial intelligence device meets the obstacle escape condition according to the operation data;

[0077] The obstacle control module 204 is used to generate an obstacle crossing strategy for the artificial intelligence device according to the operating data and the obstacle map if the artificial intelligence device meets the obstacle crossing conditions, and generate an execution control command corresponding to the obstacle crossing strategy, and control the artificial intelligence device to complete the obstacle crossing according to the execution control command.

[0078] In an embodiment of the present invention, the obstacle-crossing and escape device of the artificial intelligence device runs the obstacle-crossing and escape method of the artificial intelligence device. When the artificial intelligence device is started, the obstacle-crossing and escape device of the artificial intelligence device obtains multi-perspective images of the environment in which the artificial intelligence device is located, and performs neural radiation field processing on the multi-perspective images to obtain a three-dimensional environmental representation of the environment; generates an obstacle map of the environment in which the artificial intelligence device is located according to the three-dimensional environmental representation; when the artificial intelligence device is working, obtains the operation data of the artificial intelligence device, and determines in real time whether the obstacle-crossing and escape conditions are met according to the operation data; if so, generates an obstacle crossing strategy of the artificial intelligence device according to the operation data and the obstacle map, and generates an execution control command corresponding to the obstacle crossing strategy, and controls the artificial intelligence device to complete obstacle crossing and escape according to the execution control command. This method realizes efficient obstacle crossing and escape of artificial intelligence equipment by combining neural radiation field technology and real-time data analysis, and significantly improves its operation efficiency and safety.

[0079] above Figure 2 The obstacle escape device of the artificial intelligence device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The obstacle escape device of the artificial intelligence device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0080] Figure 33 is a schematic diagram of the structure of an obstacle-crossing and escape device of an artificial intelligence device provided by an embodiment of the present invention. The obstacle-crossing and escape device 300 of the artificial intelligence device may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage device terminals) storing application programs 333 or data 332. Among them, the memory 320 and the storage medium 330 may be short-term storage or permanent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the obstacle-crossing and escape device 300 of the artificial intelligence device. Furthermore, the processor 310 may be configured to communicate with the storage medium 330, and execute a series of instruction operations in the storage medium 330 on the obstacle-crossing and escape device 300 of the artificial intelligence device to implement the steps of the obstacle-crossing and escape method of the above-mentioned artificial intelligence device.

[0081] The obstacle escape device 300 of the artificial intelligence device may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 331, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It can be understood by those skilled in the art that Figure 3 The obstacle escape device structure of the artificial intelligence device shown does not constitute a limitation on the obstacle escape device of the artificial intelligence device provided by the present invention, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0082] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the obstacle escape method of the artificial intelligence device.

[0083] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0084] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0085] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for overcoming obstacles of an artificial intelligence device, characterized in that: The obstacle escape of the artificial intelligence device includes: When the artificial intelligence device is started, a multi-view image of the environment in which the artificial intelligence device is located is obtained, and a neural radiation field processing is performed on the multi-view image to obtain a three-dimensional environment representation of the environment in which the artificial intelligence device is located; Generate an obstacle map of the environment in which the artificial intelligence device is located according to the three-dimensional environment representation; wherein, the generating the obstacle map of the environment in which the artificial intelligence device is located according to the three-dimensional environment representation includes: performing three-dimensional semantic segmentation processing on the three-dimensional environment representation to obtain a preliminary classification result, wherein the preliminary classification result includes the ground, obstacles and passable areas; according to the preliminary classification result, perform geometric feature analysis on the identified obstacles to obtain feature data of the obstacles, and perform cross-modal variational autoencoder processing on the feature data to obtain a unified potential representation of the obstacles; according to the unified potential representation, perform attribute classification on the obstacles to obtain detailed obstacle information, wherein the detailed obstacle information includes obstacle type and physical characteristics; combine the detailed obstacle information with the three-dimensional environment representation to generate an obstacle map of the environment in which the artificial intelligence device is located; When the artificial intelligence device is working, the operation data of the artificial intelligence device is obtained, and whether the artificial intelligence device meets the obstacle escape condition is determined in real time according to the operation data; If so, an obstacle crossing strategy of the artificial intelligence device is generated according to the operating data and the obstacle map, and an execution control command corresponding to the obstacle crossing strategy is generated, and the artificial intelligence device is controlled according to the execution control command to complete the obstacle crossing and escape.

2. The obstacle escaping method of an artificial intelligence device according to claim 1, characterized in that: When the artificial intelligence device is started, a multi-view image of the environment in which the artificial intelligence device is located is obtained, and a neural radiation field processing is performed on the multi-view image to obtain a three-dimensional environment representation of the environment in which the artificial intelligence device is located, including: Using multiple cameras to collect images of the environment around the artificial intelligence device from different angles and positions to obtain multi-view images; Inputting the multi-view images into a pre-trained neural radiation field model to obtain a volume density field and a radiosity field of the environment in which the artificial intelligence device is located; According to the volume density field and the radiosity field, the three-dimensional space of the environment is rapidly sampled and voxelized to obtain a discretized density field representation; An optimized three-dimensional density gradient algorithm is applied to the discretized density field representation to generate three-dimensional point cloud data that highlights the object contour and surface features, and the three-dimensional point cloud data is processed using a real-time mesh reconstruction algorithm to obtain a three-dimensional environment representation model.

3. The obstacle escaping method of an artificial intelligence device according to claim 1, characterized in that: Generating an obstacle crossing strategy for the artificial intelligence device according to the operation data and the obstacle map, generating an execution control command corresponding to the obstacle crossing strategy, and controlling the artificial intelligence device to complete obstacle crossing and escape according to the execution control command includes: According to the operation data and the obstacle map, a global optimal path search tree algorithm and a graph convolutional network are jointly processed on the motion path of the artificial intelligence device to obtain a global obstacle crossing path; Performing a fast expansion random tree algorithm on the global obstacle crossing path to obtain a local obstacle crossing path, and performing spline interpolation processing on the motion trajectory of the artificial intelligence device according to the local obstacle crossing path to obtain an obstacle crossing action sequence; According to the obstacle crossing action sequence, the operation data of the artificial intelligence device is jointly processed by a cross-modal variational autoencoder and a deep Q network to obtain an obstacle crossing strategy of the artificial intelligence device; Generate an execution control command corresponding to the obstacle surmounting strategy, and control the artificial intelligence device to complete the obstacle surmounting and escape according to the execution control command.

4. The obstacle escaping method of an artificial intelligence device according to claim 3, characterized in that: The method of performing a global optimal path search tree algorithm and a graph convolutional network joint processing on the motion path of the artificial intelligence device according to the operation data and the obstacle map to obtain a global obstacle crossing path includes: Analyze the environment around the current position of the artificial intelligence device based on the operating data and the obstacle map to obtain specific obstacle information that needs to be overcome; Using a pre-trained graph convolutional network model, the specific obstacle information is processed to obtain an obstacle feature representation; Inputting the obstacle feature representation into a global optimal path search tree algorithm to generate multiple potential obstacle-crossing paths and obtain an initial path set; Performing a local path tree search on each obstacle-crossing path in the initial path set to obtain a path search space, and evaluating and sorting the paths in the path search space according to a preset path evaluation criterion to obtain a global obstacle-crossing path candidate set; An optimal path is selected from the global obstacle-crossing path candidate set, and a smoothing process is performed on the optimal path to obtain a global obstacle-crossing path.

5. The obstacle escaping method of an artificial intelligence device according to claim 3, characterized in that: The method of performing a fast expansion random tree algorithm on the global obstacle crossing path to obtain a local obstacle crossing path, and performing spline interpolation processing on the motion trajectory of the artificial intelligence device according to the local obstacle crossing path to obtain an obstacle crossing action sequence includes: Extracting a path segment near the current position of the artificial intelligence device from the global obstacle-crossing path as a starting point and a target point for local path planning, and constructing an initial structure of a fast-expanding random tree with the starting point as a root node; Randomly sampling feasible points in the obstacle map, connecting the feasible points to the nearest nodes in the initial structure to form new branches of the fast-expanding random tree, and performing collision detection on the new branches to remove branches that collide with obstacles; Extracting an optimal path connecting a starting point and a target point from the generated fast expanding random tree to obtain a local obstacle crossing path, and performing spline interpolation processing on the local obstacle crossing path to obtain a continuous and smooth motion trajectory; According to the continuous and smooth motion trajectory, the position sequence of the artificial intelligence device is calculated to obtain the obstacle crossing action sequence.

6. The obstacle escaping method of an artificial intelligence device according to claim 3, characterized in that: According to the obstacle crossing action sequence, the operation data of the artificial intelligence device is jointly processed by a cross-modal variational autoencoder and a deep Q network to obtain the obstacle crossing strategy of the artificial intelligence device, including: The obstacle-crossing action sequence and the operation data of the artificial intelligence device are fused to obtain multimodal input data, and the multimodal input data is input into a cross-modal variational autoencoder to obtain a compressed latent space representation; Sampling the latent space representation to obtain a low-dimensional feature vector containing environment and action information, and inputting the low-dimensional feature vector into a deep Q network to obtain Q-value estimates for different actions; According to the Q value estimation, an ε-greedy strategy is used to select an action to obtain a preliminary obstacle crossing strategy, and the preliminary obstacle crossing strategy is subjected to a safety check and constraint processing to obtain the obstacle crossing strategy of the artificial intelligence device.

7. An obstacle-crossing and escape device for artificial intelligence equipment, characterized in that: The obstacle-overcoming and escape device of the artificial intelligence device comprises: An environment image processing module, used for acquiring multi-view images of the environment in which the artificial intelligence device is located when the artificial intelligence device is started, and performing neural radiation field processing on the multi-view images to obtain a three-dimensional environment representation of the environment in which the artificial intelligence device is located; An obstacle map generation module is used to generate an obstacle map of the environment in which the artificial intelligence device is located according to the three-dimensional environment representation; wherein, the generation of the obstacle map of the environment in which the artificial intelligence device is located according to the three-dimensional environment representation includes: performing three-dimensional semantic segmentation processing on the three-dimensional environment representation to obtain a preliminary classification result, wherein the preliminary classification result includes the ground, obstacles and passable areas; according to the preliminary classification result, performing geometric feature analysis on the identified obstacles to obtain feature data of the obstacles, and performing cross-modal variational autoencoder processing on the feature data to obtain a unified potential representation of the obstacles; according to the unified potential representation, performing attribute classification on the obstacles to obtain detailed obstacle information, wherein the detailed obstacle information includes obstacle type and physical characteristics; combining the detailed obstacle information with the three-dimensional environment representation to generate an obstacle map of the environment in which the artificial intelligence device is located; A judgment module, used for obtaining the operation data of the artificial intelligence device when the artificial intelligence device is working, and judging in real time whether the artificial intelligence device meets the obstacle escape condition according to the operation data; The obstacle crossing control module is used to generate an obstacle crossing strategy for the artificial intelligence device according to the operating data and the obstacle map if the artificial intelligence device meets the obstacle crossing and escape conditions, and to generate an execution control command corresponding to the obstacle crossing strategy, and to control the artificial intelligence device to complete the obstacle crossing and escape according to the execution control command.

8. An obstacle-crossing and escape device for an artificial intelligence device, characterized in that: The obstacle escaping device of the artificial intelligence device comprises: a memory and at least one processor, wherein the memory stores instructions; The at least one processor calls the instructions in the memory to enable the obstacle escape device of the artificial intelligence device to perform the steps of the obstacle escape method of the artificial intelligence device as described in any one of claims 1-6.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the obstacle escape method of the artificial intelligence device as described in any one of claims 1-6 are implemented.

Images