Image rendering method and related methods, devices, equipment and storage medium

By acquiring and converting the rotation matrix and transfer matrix under the coordinate system in the target space, the problem of inaccurate position information in neural radiation field technology is solved, and the accuracy of scene perception and rendering quality are improved.

CN118537507BActive Publication Date: 2024-11-26PENG CHENG LAB
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410606649.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-15
Publication Date
2024-11-26
Estimated Expiration
2044-05-15

AI Technical Summary

Technical Problem

The existing neural radiation field technology is not accurate enough when acquiring position information, resulting in low accuracy in scene perception and inability to effectively utilize real-time position information under spatial coordinate systems.

Method used

By obtaining the target position in the target space, determining its position coordinates under the spatial coordinate system, and performing coordinate system conversion based on the coordinate mapping relationship, the rotation matrix and transfer matrix of the target position under the perceptual coordinate system are obtained, thereby realizing image rendering.

Benefits of technology

Improve the accuracy and rendering quality of scene perception, generate high-quality rendering pictures, and enhance the rendering accuracy of the target rendering model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118537507B_ABST
    Figure CN118537507B_ABST
Patent Text Reader

Abstract

The present application discloses an image rendering method and related methods, devices, equipment and storage media. The method includes: obtaining a target position in a target space; determining the spatial position coordinates corresponding to the target position in a spatial coordinate system corresponding to the target space; performing a coordinate system conversion on the spatial position coordinates based on a coordinate mapping relationship corresponding to the target space, and obtaining a target rotation matrix and a target transfer matrix of the target position in a perception coordinate system corresponding to the target space; performing image rendering according to the target rotation matrix and the target transfer matrix through a target rendering model, and obtaining a target rendered image corresponding to the target position. When obtaining the spatial position coordinates corresponding to the target position, the method can realize the conversion of the coordinate system based on the coordinate mapping relationship, so that the target rendering model can accurately perform image rendering according to the target rotation matrix and the target transfer matrix of the spatial position coordinates in the perception coordinate system, thereby improving the accuracy of scene perception.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and more specifically, to an image rendering method and related methods, devices, equipment and storage media. Background Art

[0002] Perception technology aims to enable computer systems to perceive and understand their surroundings, thereby simulating or enhancing human perception. Currently, researchers have proposed methods to use computer vision technology to simulate or enhance human perception. It combines technologies such as pattern recognition, machine learning, and artificial intelligence to enable computers to perceive and understand images, videos, or real-time streaming data, and extract information from them for various scenarios.

[0003] For example, in the fields of target detection, autonomous driving, and intelligent agent games. Perception technology can provide scene perception capabilities for intelligent decision-making of different objects (such as intelligent agents). Related technologies have proposed perception applications based on neural radiation fields: reconstructing the perception picture under free perspective based on the moving position of the intelligent agent. However, this method cannot accurately obtain position information, making the accuracy of scene perception low. Summary of the invention

[0004] The present application provides an image rendering method and related methods, devices, equipment and storage media, so that a target rendering model can accurately render images according to a target rotation matrix and a target transfer matrix of spatial position coordinates in a perception coordinate system, thereby improving the accuracy of scene perception.

[0005] According to one aspect of the present application, an image rendering method is provided, the method comprising: obtaining a target position in a target space; determining the spatial position coordinates corresponding to the target position in a spatial coordinate system corresponding to the target space; performing coordinate system transformation on the spatial position coordinates based on a coordinate mapping relationship corresponding to the target space, and obtaining a target rotation matrix and a target transfer matrix of the target position in a perception coordinate system corresponding to the target space; wherein the coordinate mapping relationship is obtained by mapping the spatial intersection coordinates corresponding to each grid intersection in the target space in the spatial coordinate system and the intersection rotation matrix and the intersection transfer matrix corresponding to each grid intersection in the perception coordinate system, and the grid intersection is an intersection in a spatial grid grid corresponding to the target space; performing image rendering according to the target rotation matrix and the target transfer matrix through a target rendering model, and obtaining a target rendered image corresponding to the target position.

[0006] According to one aspect of the present application, a perception prediction method is provided, the method comprising: in response to a perception instruction, obtaining a moving position in a target space; obtaining a moving rendered image corresponding to the moving position, wherein the moving rendered image is obtained by image rendering according to the above-mentioned image rendering method; performing perception prediction based on the moving rendered image to obtain a prediction result; when the prediction result indicates that there is an obstacle in the target rendered image, determining an obstacle avoidance decision, the obstacle avoidance decision at least including obstacle avoidance movement or stopping movement; when the perception result indicates that there is no obstacle in the target rendered image, generating and executing a movement instruction.

[0007] According to one aspect of the present application, an image rendering device is provided, which includes: a position acquisition module, which is used to acquire a target position in a target space; a coordinate determination module, which is used to determine the spatial position coordinates corresponding to the target position in the spatial coordinate system corresponding to the target space; a relationship conversion module, which is used to perform coordinate system conversion on the spatial position coordinates based on the coordinate mapping relationship corresponding to the target space, and obtain a target rotation matrix and a target transfer matrix of the target position in the perception coordinate system corresponding to the target space; wherein the coordinate mapping relationship is obtained by mapping the spatial intersection coordinates corresponding to each grid intersection in the target space in the spatial coordinate system and the intersection rotation matrix and the intersection transfer matrix corresponding to each grid intersection in the perception coordinate system, and the grid intersection is an intersection in the spatial grid grid corresponding to the target space; an image rendering module, which is used to perform image rendering according to the target rotation matrix and the target transfer matrix through a target rendering model, and obtain a target rendered image corresponding to the target position.

[0008] According to one aspect of the present application, an image rendering device is provided, which includes: a perception response module, which is used to obtain a moving position in a target space in response to a perception instruction; a mobile rendering module, which is used to obtain a mobile rendering image corresponding to the moving position, wherein the mobile rendering image is obtained by image rendering according to the above-mentioned image rendering method; a perception prediction module, which is used to perform perception prediction based on the mobile rendering image to obtain a prediction result; a first decision module, which is used to determine an obstacle avoidance decision when the prediction result indicates that there is an obstacle in the target rendered image, and the obstacle avoidance decision at least includes obstacle avoidance movement or stopping movement; and a second decision module, which is used to generate and execute a movement instruction when the perception result indicates that there is no obstacle in the target rendered image.

[0009] According to one aspect of the present application, a computer-readable storage medium is provided, which stores a computer program, wherein when the computer program is executed by a processor, the above-mentioned image rendering method or perception prediction method is executed.

[0010] According to one aspect of the present application, a computer device is provided, which includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is called by the processor, the image rendering method or the perception prediction method is executed.

[0011] According to one aspect of the present application, a computer program product is provided, which includes a computer program stored in a storage medium; a processor of a computer device reads the computer program from the storage medium, and the processor executes the computer program, so that the computer device executes the above-mentioned image rendering method or perceptual prediction method.

[0012] The present application can obtain the target position in the target space, and determine the spatial position coordinates corresponding to the target position in the spatial coordinate system corresponding to the target space, and then perform a coordinate system transformation on the spatial position coordinates based on the coordinate mapping relationship corresponding to the target space, and obtain the target rotation matrix and target transfer matrix of the target position in the perception coordinate system corresponding to the target space. In this way, the coordinate system transformation from the spatial coordinate system corresponding to the target space to the perception coordinate system corresponding to the target space is realized through the coordinate mapping relationship, and two different types of coordinate systems are connected in series. Among them, the coordinate mapping relationship is obtained by mapping the spatial intersection coordinates corresponding to each grid intersection in the target space in the spatial coordinate system and the intersection rotation matrix and intersection transfer matrix corresponding to each grid intersection in the perception coordinate system, and the grid intersection is the intersection in the spatial grid grid corresponding to the target space. Furthermore, the target rendering model is used to perform image rendering according to the target rotation matrix and the target transfer matrix to obtain a target rendered image corresponding to the target position. Therefore, by converting the precise spatial position coordinates acquired in real time in the spatial coordinate system into the rotation matrix and transfer matrix in the perception coordinate system, the image rendering of the target rendering model in the perception coordinate system can effectively utilize the real-time precise positioning in the spatial coordinate system, thereby improving the rendering accuracy of the target rendering model, generating high-quality rendering images, and improving the quality of scene perception.

[0013] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 A schematic diagram of a system architecture provided in an embodiment of the present application is shown.

[0016] Figure 2 A schematic diagram of system deployment provided in an embodiment of the present application is shown.

[0017] Figure 3 A schematic diagram of an application scenario provided by an embodiment of the present application is shown.

[0018] Figure 4 A flow chart of an image rendering method provided in an embodiment of the present application is shown.

[0019] Figure 5 A schematic diagram of a spatial grid provided in an embodiment of the present application is shown.

[0020] Figure 6 A schematic diagram of the conversion of a coordinate system provided in an embodiment of the present application is shown.

[0021] Figure 7 A construction flow chart of a coordinate mapping relationship provided in an embodiment of the present application is shown.

[0022] Figure 8 A schematic diagram showing a flow chart of a perception prediction method provided in an embodiment of the present application

[0023] Fig. 9 A perception flow chart of an autonomous intelligent agent provided in an embodiment of the present application is shown.

[0024] Fig.10 A flowchart of coordinate mapping relationship construction and model training provided in an embodiment of the present application is shown.

[0025] Fig.11 A schematic diagram of an arrangement of scene reconstruction provided in an embodiment of the present application is shown.

[0026] Fig.12 A schematic diagram of Colmap matching provided in an embodiment of the present application is shown.

[0027] Fig.13 It is a module block diagram of an image rendering device provided in an embodiment of the present application.

[0028] Fig.14 It is a module block diagram of a perception prediction device provided in an embodiment of the present application.

[0029] Fig.15 It is a module block diagram of a computer device provided in an embodiment of the present application.

[0030] Fig.16It is a module block diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The implementation methods described with reference to the drawings are exemplary and are only used to explain the present application, and cannot be understood as limitations on the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0032] In some processes described in the specification, claims and the above-mentioned drawings, multiple steps appearing in a specific order are included, but it should be clearly understood that these steps may not be executed in the order in which they appear in this document or may be executed in parallel. The step numbers are only used to distinguish different steps, and the numbers themselves do not represent any execution order. In addition, the descriptions such as "first", "second" and "target" in this document are used to distinguish similar objects, and are not necessarily used to describe a specific order, sequence or quantity.

[0033] It is worth noting that in the specific implementation of the present application, data such as target location and scene image are involved. When the above embodiments of the present application are applied to specific products or technologies, it is necessary to obtain permission or consent from the object, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards. For example, when the embodiment of the present application needs to obtain a scene image, a separate permission or separate consent for the scene image can be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining a separate permission or separate consent, the necessary relevant data for enabling the normal operation of the embodiment of the present application can be obtained.

[0034] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0035] The image rendering method and related methods proposed in this application involve artificial intelligence (AI) technology, which is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0036] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models are also called large models and basic models. After fine-tuning, they can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0037] Computer vision technology (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to using cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphics processing so that the computer processes the images into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, video content / behavior recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common scene perception technologies.

[0038] Neural Radiance Fields (NeRF) is a deep learning model for reconstructing 3D scenes. It is composed of the surface of objects and the light density in space, and can accurately restore the geometry and appearance of 3D scenes from the input 2D image sequence. The Neural Radiance Field model is based on neural networks. By training a deep neural network, the input 2D image sequence and the corresponding camera parameters are mapped to the radiation and density functions of the 3D scene. In this way, the model can generate realistic 3D reconstruction results and can automatically learn the geometry and lighting information in the scene during the training process. The Neural Radiance Field model shows potential application prospects in applications such as 3D reconstruction, virtual reality, and augmented reality.

[0039] Simultaneous Localization and Mapping (SLAM) is a technology used to simultaneously locate itself and build a map in an unknown environment. SLAM is often used in unmanned systems, such as autonomous mobile robots, self-driving cars, and unmanned aerial vehicles, to help them achieve autonomous navigation and path planning in unknown environments. This technology can provide real-time positioning and build 3D maps for intelligent agents moving in unknown scenes, and can determine the position and velocity of the intelligent agent in real time without relying on external positioning sensors. This is crucial for many autonomous navigation and robotics applications, enabling robots to navigate and locate in unknown environments without external navigation systems.

[0040] At present, with the rapid development of neural radiation field, this technology is gradually applied to scene perception tasks. The classic methods of neural radiation field include Mip-NeRF, NeRF++, Mip-NeRF 360 and 3D Gaussian Splatting. By collecting image data from sparse perspectives in normal lighting scenes, the static information such as radiation intensity / color texture of each point in the scene is pre-modeled using deep neural networks, and then the perception of the agent under the free perspective can be reconstructed according to the position of the agent in any weather and lighting / at any time. Therefore, this technology can provide three-dimensional dynamic perception for the agent's game decision-making.

[0041] The neural radiation field reconstructs the perceived image, which requires the position and posture of the internal coordinate system, including the rotation matrix and the transfer matrix. Since the intelligent agent cannot obtain accurate position information, the rotation matrix and the transfer matrix determined based on the position information are not accurate enough, resulting in poor quality of the perceived image reconstructed by the neural radiation field. However, the coordinate system used by the positioning system to obtain the position information and the coordinate system in which the neural radiation field reconstructs the perceived image are usually different types of coordinate systems, that is, the rotation matrix and the transfer matrix determined based on the positioning system cannot provide help for the rendering perception process of the neural radiation field.

[0042] In order to solve the above problems, the inventors have proposed the image rendering method and the perception prediction method provided in the embodiments of the present application after research. The system architecture and related application scenarios of the above methods involved in the present application are first described below.

[0043] See also Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. Figure 1 As shown, the above method provided in the embodiment of the present application can be applied in the system 100. The data acquisition device 110 is used to acquire training data, and the training data can be a scene image used for model training. After acquiring the training data, the data acquisition device 110 can store the training data in the database 120, and the training device 130 can perform network training on the preset neural network based on the training data in the database 120 until the preset neural network meets the training end condition, thereby obtaining the trained target rendering model 101.

[0044] Optionally, the training data maintained in the database 120 does not necessarily all come from the data acquisition device 110, and may also be received from other devices. For example, the execution device 140 may also serve as a data acquisition terminal, and use the scene images collected in the space as training data and store them in the database 120. For example, the execution device 140 may store the scene images collected in real time in the space in the database 120, and the data acquisition device 110 may also collect scene images from other perspectives in the space and store them in the database 120 at the same time.

[0045] The training end condition may be: the total loss value of the target loss is less than a preset value, the total loss value of the target loss is within a threshold range, or the number of training times reaches a preset number of times. The target rendering model 101 may be a deep neural network or a neural radiation field model composed of multiple neural networks. For example, a network structure that can be applied to scene rendering, such as a fully connected neural network or a multilayer perceptron, is not limited here.

[0046] During the process of the processing module 141 of the execution device 140 performing calculations and other related processing, the execution device 140 may call data, programs, etc. in the data storage system 142 for corresponding calculation processing, and store data and instructions such as processing results obtained by the calculation processing in the data storage system 142. For example, the execution device 140 may store the target rendered image generated by the target rendering model based on the target rotation matrix and the target transfer matrix in the data storage system 142.

[0047] The training device 130 and the execution device 140 can be computer devices such as servers or terminals. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), blockchains, and big data and artificial intelligence platforms. The terminal can be a smart phone, an intelligent robot, an autonomous intelligent body, and a vehicle-mounted terminal.

[0048] It should be noted that Figure 1 It is only a schematic diagram of a system architecture provided in the embodiment of the present application. The system architecture and application scenarios described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. Figure 1 The data storage system 142 in the example is an internal memory relative to the execution device 140 . In other cases, the data storage system 142 may also be placed outside the execution device 140 .

[0049] See also Figure 2 , Figure 2 Schematic diagram of a system deployment is shown. For example, the image rendering method and the perception prediction method provided in the embodiment of the present application can also be deployed in Figure 2 In the system 200 shown, the system includes a terminal 240, the Internet 230, a gateway 220, a server 210, and the like.

[0050] The terminal 240 may include various types of intelligent terminals such as a sweeping robot, a vehicle-mounted terminal, and an agent. In addition, it may be a single device or a collection of multiple devices. The terminal 240 may communicate with the Internet 230 in a wired or wireless manner to exchange data. For example, the terminal 240 may communicate with the Internet 230 via a wireless router 250.

[0051] The server 210 refers to a computer system that can provide certain services to the terminal 240. Compared with the ordinary terminal 240, the server 210 has higher requirements in terms of stability, security, performance, etc. The server 210 can be a high-performance computer in a network platform, a cluster of multiple high-performance computers, a part of a high-performance computer (for example, a virtual machine), a combination of parts of multiple high-performance computers (for example, virtual machines), etc.

[0052] The gateway 220 is also called an internetwork connector or a protocol converter. The gateway realizes network interconnection at the transport layer and is a computer system or device that acts as a converter. The gateway is a translator between two systems that use different communication protocols, data formats or languages, or even completely different architectures. At the same time, the gateway can also provide filtering and security functions. The message sent by the terminal 240 to the server 210 can be transmitted to the corresponding server 210 through the gateway 220. The message sent by the server 210 to the terminal 240 can also be transmitted to the corresponding terminal 240 through the gateway 220.

[0053] As an implementation mode, the image rendering method and related methods can be implemented completely on the server 210, can be implemented completely on the terminal, or can be implemented partially on the server 210 and partially on the terminal 240. Exemplarily, the server 210 can be used as a training device to train a target rendering model based on multiple scene images, and the terminal 240 can be used as an execution device to deploy the target rendering model locally. For example, when the terminal 240 moves to the target position in the target space, the spatial position coordinates of the target position can be transformed into a coordinate system based on the coordinate mapping relationship corresponding to the target space, and the target rotation matrix and target transfer matrix of the target position in the perception coordinate system corresponding to the target space can be obtained. Then, the terminal 240 performs image rendering according to the target rotation matrix and the target transfer matrix through the target rendering model, obtains the target rendering image corresponding to the target position, and uses the target rendering image to make corresponding decision actions, such as braking to stop obstacles or bypassing obstacles.

[0054] It should be noted that Figure 2 This is only a schematic diagram of a system deployment provided by the embodiment of the present application. The system deployment scheme described in the embodiment of the present application is only to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided by the embodiment of the present application. For example, terminal 240 can generally refer to one of multiple terminals. This embodiment only refers to Figure 2 The three terminals shown in the figure are used as examples. It is known to those skilled in the art that, with the evolution of system deployment solutions, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0055] The image rendering method provided in this application can be applied in various scenarios, such as Figure 4 Schematic diagram of application scenarios of smart cleaning shown.

[0056] The intelligent sweeping robot is an autonomous cleaning device widely used in daily life. It is suitable for various indoor environments, such as homes, offices, shops, etc. The workflow usually includes environmental perception, path planning, navigation positioning, cleaning operations, and safe obstacle avoidance. Through sensors such as lidar and cameras, the robot perceives the surrounding environment, builds maps, and identifies obstacles. Based on the map and current position, the robot plans the optimal path and uses the autonomous navigation system to accurately locate. At the same time, the robot can perceive and avoid obstacles in real time to ensure safe cleaning. The image rendering method proposed in this application can be applied to the autonomous cleaning process of the intelligent sweeping robot to achieve efficient, autonomous, and intelligent cleaning tasks, providing users with a convenient and comfortable life experience.

[0057] like Figure 3 As shown, the control system of the sweeping robot 240 as an intelligent terminal can be developed based on the image rendering method. The sweeping robot 240 can collect scene images in the room and upload the scene images to the cloud server 210 of the manufacturer through the wireless router 250. Then the cloud server 210 can obtain the target rendering model based on the scene image training. Optionally, the sweeping robot 240 can download and install the target rendering model from the cloud server 210 to achieve scene perception during the autonomous cleaning of the room.

[0058] Exemplarily, when the sweeping robot 240 moves in the room, it can collect the current target position it moves to in real time, and determine the spatial position coordinates corresponding to the target position in the spatial coordinate system corresponding to the room. Then, the sweeping robot 240 performs coordinate system conversion on the spatial position coordinates based on the coordinate mapping relationship corresponding to the room, and obtains the target rotation matrix and target transfer matrix of the target position in the perception coordinate system corresponding to the room. Furthermore, the sweeping robot 240 can perform image rendering according to the target rotation matrix and the target transfer matrix through the target rendering model to obtain the target rendered image corresponding to the target position, and then use the target rendered image to make movement decisions. For example, when there is a desk near the target position, stop moving or move to avoid obstacles.

[0059] It should be noted that Figure 3It is only a schematic diagram of an application scenario provided by an embodiment of the present application. The application scenario described in the embodiment of the present application is only to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided by the embodiment of the present application. For example, in addition to the intelligent cleaning scenario of the sweeping robot, the image rendering method proposed in this application can also be used in application scenarios such as autonomous driving and intelligent logistics. A person of ordinary skill in the art will know that with the evolution of application scenarios, the technical solution provided by the embodiment of the present invention is also applicable to similar technical problems.

[0060] According to an embodiment of the present application, an image rendering method is provided. The image rendering method can be executed by a computer device (the computer device can be a server or other terminal device with certain processing capabilities), and the computer device has at least storage, calculation and communication functions. Figure 4 FIG. 2 shows a flow chart of the image rendering method provided by this embodiment. Figure 4 As shown, the image rendering method may specifically include:

[0061] S110: Acquire the target position in the target space.

[0062] Generally, scene perception refers to rendering the perception of the scene based on the posture information of the terminal (such as an intelligent agent, etc.) in the scene. To this end, the terminal needs to obtain the specific position in the rendered scene in real time for dynamic perception. Among them, the target space refers to the real world that the terminal dynamically perceives, such as rooms, venues, and parks.

[0063] The target position may refer to a point in the target space where the terminal is to render the scene. In the rendering scene, the target position determines the viewing angle of the terminal to capture the scene image. The terminal can render the captured multiple scene images according to the target position to generate the final rendered image. For example, when a sweeping robot as a terminal performs a room cleaning task, it moves to a certain position on the cleaning track, or a designated cleaning position in the room.

[0064] S120: Determine the spatial position coordinates corresponding to the target position in the spatial coordinate system corresponding to the target space.

[0065] Since the target position determines the viewing angle of the terminal to capture the scene image during the rendering process, the target position determines the content and appearance of the scene rendering result. Different target positions will lead to different viewing angles, thereby affecting the visual effect of the final rendered picture. It is understandable that the target position plays a vital role in the rendering scene. To this end, the present application proposes to obtain the spatial position coordinates of the target position in the spatial coordinate system corresponding to the target space, so as to accurately determine the pose information of the position to be rendered based on the spatial position coordinates, so as to improve the perceived quality of scene rendering.

[0066] The spatial coordinate system refers to a coordinate system used to represent the absolute position and direction of the terminal in the target space. In an embodiment of the present application, the terminal can establish a spatial coordinate system corresponding to the target space, that is, a SLAM coordinate system, through a simultaneous localization and mapping technology (SLAM), and obtain the spatial position coordinates of the target position in the spatial coordinate system.

[0067] As an implementation method, the terminal can obtain sensor data, such as distance measurement or acceleration, through sensors such as laser radar or inertial measurement unit (IMU), and extract map features such as feature points, edges, etc. based on the sensor data, and then infer the posture information based on the map features. Further, the position and posture information of the feature points are used to construct a spatial map, and the reference coordinate system of the spatial map is the spatial coordinate system, wherein the spatial map can be a two-dimensional grid map.

[0068] Optionally, the terminal may use a sensor to obtain sensor data at the target location, and match the features extracted from the sensor data with known features in the spatial map, thereby determining the spatial position coordinates of the target location in the grid map through the matched feature points or feature descriptions.

[0069] S130: performing coordinate system transformation on the spatial position coordinates based on the coordinate mapping relationship corresponding to the target space, and obtaining a target rotation matrix and a target transfer matrix of the target position in the perception coordinate system corresponding to the target space.

[0070] Scene rendering requires obtaining pose information representing the viewing angle or camera. Specifically, for each viewing angle (or camera), there is usually a rotation matrix and a transfer matrix as pose information to describe the position and direction of the camera. The rendering model can then map the three-dimensional points in the scene to the two-dimensional image plane based on the pose information to complete the entire scene rendering process.

[0071] Considering that the coordinate system of the rendering model for scene rendering based on the rotation matrix and the transfer matrix, that is, the perception coordinate system, for example, the Colmap coordinate system, is two different types of coordinate systems from the spatial coordinate system to which the spatial position coordinates belong, the spatial position coordinates of the target position cannot be directly used for scene rendering. To this end, the present application proposes to convert the spatial position coordinates in the spatial coordinate system into the rotation matrix and the transfer matrix in the perception coordinate system through a coordinate mapping relationship, thereby realizing the transformation from the spatial coordinate system to the perception coordinate system.

[0072] Among them, the coordinate mapping relationship is obtained by mapping the spatial intersection coordinates corresponding to each grid intersection in the target space in the spatial coordinate system and the intersection rotation matrix and intersection transfer matrix corresponding to each grid intersection in the perception coordinate system through the coordinate system, and the grid intersection is the intersection in the spatial grid grid corresponding to the target space. The coordinate mapping relationship records the intersection rotation matrix and intersection transfer matrix corresponding to the spatial intersection coordinates of each grid intersection in the spatial coordinate system in the perception coordinate system in a mapping manner, for example, a one-way mapping table. The target rotation matrix refers to the rotation matrix corresponding to the target position in the perception coordinate system, and the target transfer matrix refers to the transfer matrix corresponding to the target position in the perception coordinate system.

[0073] The reason why this application proposes a method for transforming a spatial coordinate system into a perceptual coordinate system is that, considering that in different coordinate systems, although the way of describing the position may be different, the concept of relative position is still the same. Relative position refers to the positional relationship of one point relative to another point or a reference point. Their relative position does not depend on the selected coordinate system, but only depends on the selected reference point and the point to be described. The relative position is independent of the selected coordinate system. In this way, based on the coordinate mapping relationship, the intersection rotation matrix and intersection transfer matrix of the grid intersection in the perceptual coordinate system can be obtained, and the relative position of the target position with respect to the grid intersection in the spatial coordinate system can be obtained, and then the relative position can be used to calculate the target rotation matrix and target intersection transfer matrix of the target position relative to the intersection rotation matrix and intersection transfer matrix of the grid intersection in the perceptual coordinate system.

[0074] In some embodiments, two grid intersection points closest to the target position are determined from the spatial grid grid corresponding to the target space, and the distance ratio between the target position and the two grid intersection points closest to the target position in the spatial coordinate system is determined, and then the target rotation matrix and the target transfer matrix of the target position in the perception coordinate system corresponding to the target space are determined according to the distance ratio and the coordinate mapping relationship, which may specifically include:

[0075] (1) Obtain each grid intersection in the spatial grid grid corresponding to the target space.

[0076] A grid is a grid structure consisting of square cells arranged horizontally and vertically. Each cell is usually of the same size and shape to facilitate processing and analysis. Figure 5 , Figure 5 A schematic diagram of a spatial grid is shown. Figure 5 As shown, the target space can be divided into a series of grid cells of equal size, and the grid grid composed of all grid cells is the spatial grid grid. Among them, the intersection point of each two sides of the grid cell is the grid intersection point.

[0077] As an implementation method, a spatial grid grid corresponding to the target space can be obtained, and the spatial grid grid is obtained by spatially dividing the target space when constructing a coordinate mapping relationship. In the spatial grid grid, each grid intersection corresponds to the intersection of the edges of the grid cell, and the corner point of each cell can be obtained as the grid intersection by traversing the rows and columns of the grid grid.

[0078] (2) The spatial intersection coordinates of each grid intersection in the spatial coordinate system are compared with the spatial position coordinates to obtain the first intersection coordinates and the second intersection coordinates.

[0079] The first intersection coordinates are the spatial intersection coordinates of the first grid intersection closest to the target position in the spatial coordinate system, and the second intersection coordinates are the spatial intersection coordinates of the second grid intersection closest to the target position in the spatial coordinate system;

[0080] As an implementation method, the coordinate distance between the spatial intersection coordinates and the spatial position coordinates of each grid intersection in the spatial coordinate system is calculated respectively. Furthermore, the first grid intersection and the second grid intersection closest to the target position are determined by comparing each coordinate distance, and the spatial intersection coordinates corresponding to the first grid intersection and the spatial intersection coordinates corresponding to the second grid intersection are obtained.

[0081] See also Figure 6 , Figure 6 A schematic diagram of coordinate system transformation is shown in FIG. Figure 6 The target position P0 in the target space (e.g., room) is shown, and the two grid intersections closest to the target position P0 are: the first grid intersection P1 and the second grid intersection P2. The spatial position coordinates of the target position P0 are (x0, y0), the spatial position coordinates of the first grid intersection P1 are (x1, y1), and the spatial position coordinates of the second grid intersection P2 are (x2, y2).

[0082] (3) Based on the coordinate mapping relationship corresponding to the target space, the coordinates of the first intersection are transformed into a coordinate system to obtain a first rotation matrix and a first transfer matrix of the first grid intersection in the perception coordinate system corresponding to the target space; based on the coordinate mapping relationship, the coordinates of the second intersection are transformed into a coordinate system to obtain a second rotation matrix and a second transfer matrix of the second grid intersection in the perception coordinate system corresponding to the target space.

[0083] As an implementation method, the coordinate system conversion can be performed based on a single mapping table, which stores the spatial intersection coordinates of each grid intersection in the spatial coordinate system and its corresponding intersection rotation matrix and intersection transfer matrix in the perception coordinate system. Specifically, the first rotation matrix Q1 and the first transfer matrix T1 corresponding to the first intersection coordinates can be queried in the single mapping table, and the second rotation matrix Q2 and the second transfer matrix T2 corresponding to the second grid intersection can also be queried.

[0084] (4) According to the difference between the first rotation matrix and the second rotation matrix, the target rotation matrix of the target position in the perception coordinate system is determined, and according to the difference between the first transfer matrix and the second transfer matrix, the target transfer matrix of the target position in the perception coordinate system is determined.

[0085] As an implementation mode, the ratio of the distances of the target position in the spatial coordinate system (e.g., SLAM coordinate system) relative to the first grid intersection and the second grid intersection is the same as the ratio of the distances of the target position in the perception coordinate system (e.g., Colmap coordinate system) relative to the first grid intersection and the second grid intersection. Further, based on the distance ratio corresponding to the spatial coordinate system, the target transfer matrix of the target position in the perception coordinate system can be calculated according to the first rotation matrix and the first transfer matrix corresponding to the first intersection coordinates and the second rotation matrix and the second transfer matrix corresponding to the second grid intersection. Specifically, step (4) can include:

[0086] (4.1) calculating the distance ratio based on the spatial position coordinates, the first intersection coordinates and the second intersection coordinates to obtain a spatial distance ratio;

[0087] (4.2) performing geometric calculation based on the difference between the first rotation matrix and the second rotation matrix and the spatial distance ratio to obtain the target rotation matrix of the target position in the perception coordinate system;

[0088] (4.3) According to the difference between the first transfer matrix and the second transfer matrix and the spatial distance ratio, geometric calculation is performed to obtain the target transfer matrix of the target position in the perception coordinate system.

[0089] For example, Figure 6 As shown, by performing distance calculation on the spatial position coordinates and the first intersection coordinates, the first spatial distance a between the target position and the first grid intersection can be obtained, and by performing distance calculation on the target position coordinates and the second intersection coordinates, the second spatial distance b between the target position and the first grid intersection can be obtained, and then the ratio of the first spatial distance a to the second spatial distance b is used as the spatial distance ratio f.

[0090] Further, the difference Q2-Q1 between the first rotation matrix and the second rotation matrix is ​​calculated, and a geometric calculation is performed based on the difference and the spatial distance ratio f to obtain the target rotation matrix Q0 of the target position in the perception coordinate system. Further, the difference T2-T1 between the first transfer matrix and the second transfer matrix is ​​calculated, and a geometric calculation is performed based on the difference and the spatial distance ratio f to obtain the target transfer matrix T0 of the target position in the perception coordinate system. The calculation formula can be as follows:

[0091]

[0092] in, Represents the coordinates of the target position in the perception coordinate system, that is, the target rotation matrix Q0 and the target transfer matrix T0.

[0093] S140: performing image rendering according to the target rotation matrix and the target transfer matrix through the target rendering model to obtain a target rendered image corresponding to the target position.

[0094] In an embodiment of the present application, the target rendering model may be a neural radiation field model for scene rendering. The target rendering model is obtained by training based on multiple scene images collected from the target space. Specifically, a preset rendering model is obtained, and then the preset rendering model is iteratively trained based on multiple scene images corresponding to the target space until the training end condition is met, thereby obtaining a trained target rendering model.

[0095] Among them, the preset rendering model can be a pre-trained neural radiation field model, for example, a 3D Gaussian neural radiation field model. Training with multiple scene images collected from the target space can enable the target rendering model to perform scene rendering for the target space. The training end condition may be that the total loss value of the target loss function is less than a preset value, the total loss value of the target loss function no longer changes, or the number of training times reaches a preset number of times. Optionally, an optimizer can be used to optimize the target loss, and the learning rate (Learning Rate), the batch size (Batch Size) during training, and the training period (Epoch) can be set based on experimental experience.

[0096] As an implementation method, information such as the three-dimensional position (3D Position) and viewing direction (Viewing Direction) of the location point to be rendered can be input into the target rendering model, and the camera's posture (position and direction) can be described based on the target rotation matrix and the target transfer matrix so that the target rendering model can calculate the radiation value at the position and direction, thereby generating a realistic three-dimensional scene representation, that is, a target rendered image.

[0097] This embodiment can obtain the target position in the target space, and determine the spatial position coordinates corresponding to the target position in the spatial coordinate system corresponding to the target space, and then perform coordinate system transformation on the spatial position coordinates based on the coordinate mapping relationship corresponding to the target space, and obtain the target rotation matrix and target transfer matrix of the target position in the perception coordinate system corresponding to the target space. In this way, the coordinate system transformation from the spatial coordinate system corresponding to the target space to the perception coordinate system corresponding to the target space is realized through the coordinate mapping relationship, and two different types of coordinate systems are connected in series.

[0098] Furthermore, the target rendering model performs image rendering according to the target rotation matrix and the target transfer matrix to obtain a target rendering image corresponding to the target position. Thus, by converting the precise spatial position coordinates acquired in real time in the spatial coordinate system into the rotation matrix and the transfer matrix in the perception coordinate system, the image rendering of the target rendering model in the perception coordinate system can effectively utilize the real-time precise positioning in the spatial coordinate system, thereby improving the rendering accuracy of the target rendering model, generating high-quality rendering images, and improving the quality of scene perception.

[0099] The coordinate mapping relationship in the above embodiment serves as an intermediate medium for connecting the spatial coordinate system and the perceived coordinate system in series, and can map the coordinates in the spatial coordinate system to the corresponding rotation matrix and transfer matrix in the perceived coordinate system, so that the rendering model can perform image rendering based on the rotation matrix and transfer matrix. Figure 7 A method for constructing a coordinate mapping relationship provided by an embodiment of the present application is shown. The method for constructing a coordinate mapping relationship may be performed before step S130, and may specifically include:

[0100] S210: Divide the target space into spatial grids to obtain spatial grids corresponding to the target space.

[0101] Grid is a universal data representation method that facilitates cross-domain data exchange and collaboration. In the embodiment of the present application, the spatial grid can make the calculation of the position information of the target space simpler and more intuitive. For example, in the spatial grid, the coordinates of the grid intersections are defined.

[0102] As an implementation method, the corresponding spatial grid can be obtained by discretizing the continuous target space. Optionally, during the discretization of the spatial grid, the size of each grid unit can be specifically set according to the requirements of the actual task or the area of ​​the target space, which is not limited here.

[0103] S220: Acquire each grid intersection in the spatial grid grid, and determine the spatial intersection coordinates of each grid intersection in the spatial coordinate system corresponding to the target space.

[0104] As an implementation method, the size of each grid unit (i.e., the width and height of each grid unit) and the minimum and maximum values ​​of the grid on the x-axis and y-axis in the spatial coordinate system can be determined. Further, based on the range of the grid and the unit size, calculate how many grid units there are in the x-axis and y-axis directions. Further, generate coordinate values: generate coordinate values ​​of all grid intersections on the x-axis and y-axis respectively. For the coordinate values ​​on the x-axis, start from the minimum value and increase the width of one unit each time until the maximum value is reached. Similarly, for the y-axis, start from the minimum value and increase the height of one unit each time until the maximum value is reached. By using nested loops to traverse all coordinate values ​​on the x-axis and y-axis, the coordinates of all grid intersections are combined. These coordinate points, that is, the coordinates of the spatial intersections can be expressed in (x, y) form.

[0105] S230: Reconstruct the scene based on the multiple scene images corresponding to the target space, and determine the intersection rotation matrix and intersection transfer matrix of each grid intersection in the perception coordinate system corresponding to the target space according to the reconstruction result.

[0106] In the embodiment of the present application, scene reconstruction refers to three-dimensional reconstruction from an image set. Scene reconstruction can use structured light and multi-view stereo vision technology to reconstruct a three-dimensional scene by matching feature points in multiple images. During the three-dimensional reconstruction process, the rotation matrix and transfer matrix corresponding to each scene image in the image set can be calculated, which may specifically include:

[0107] (1) Acquire multiple scene images corresponding to the target space, wherein the multiple scene images at least include an intersection image of a spatial position where each grid intersection is located.

[0108] The scene image refers to the perceived image of the scene captured by the terminal from its own perspective in the target space. These scene images include the images captured by the terminal at each grid intersection, namely, the intersection images.

[0109] As an implementation method, the autonomous agent can move in the room according to a predetermined trajectory, and the predetermined trajectory may include each grid intersection, and collect images of the room through the perspective of its own camera. Considering that the perspective of collecting scene images by the terminal alone has certain limitations, auxiliary equipment can be used to collect scene images from the target space as a supplement. For example, using drones, professional cameras, etc., so as to provide diverse and rich training images for the subsequent training of the rendering model, improve the generalization ability and robustness of the rendering model, and make the picture rendered by the model closer to the scene under the real perspective.

[0110] (2) Reconstruct the scene based on multiple scene images to obtain the scene rotation matrix and scene transfer matrix of each scene image in the perception coordinate system corresponding to the target space.

[0111] As an implementation method, scene reconstruction is performed in a perceptual coordinate system corresponding to the target space. Feature points can be extracted from each scene image, and the scene can be reconstructed using the feature points. During the reconstruction process, the rotation matrix and transfer matrix corresponding to each scene image can be calculated.

[0112] (3) The position and posture of each grid intersection are determined according to the scene rotation matrix and scene transfer matrix corresponding to each scene image, and the intersection rotation matrix and intersection transfer matrix of each grid intersection in the perception coordinate system corresponding to the target space are obtained.

[0113] As an implementation method, an intersection image corresponding to each grid intersection can be obtained. Further, each intersection image can be matched with multiple scene images respectively, and the scene rotation matrix and scene transfer matrix corresponding to the scene image matched with the intersection image in the multiple scene images are used as the intersection rotation matrix and intersection transfer matrix, so as to obtain the intersection rotation matrix and intersection transfer matrix of each grid intersection in the perception coordinate system corresponding to the target space.

[0114] S240: Constructing a coordinate mapping relationship, through which the spatial intersection coordinates of each grid intersection in the spatial coordinate system correspond to an intersection rotation matrix and an intersection transfer matrix in the perception coordinate system.

[0115] As an implementation method, a coordinate mapping relationship can be constructed based on a one-way mapping table. Specifically, the spatial intersection coordinates of each grid intersection in the spatial coordinate system are used as the key of the one-way mapping table, and the intersection rotation matrix and intersection transfer matrix of each grid intersection in the perception coordinate system corresponding to the target space are used as the value of the one-way mapping table. Then, the spatial intersection coordinates, intersection rotation matrix and intersection transfer matrix constitute the key-value pairs in the one-way mapping table. The one-way mapping table is shown below:

[0116]

[0117] Among them, (x n ,y n ) represents the spatial intersection coordinates of the grid intersection in the spatial coordinate system, (Q n ,T n ) represents the intersection rotation matrix Q of the grid intersection in the perception coordinate system n and the intersection transfer matrix T n n is the number of grid intersections in the spatial grid of the target space, n>0&n∈N * For example, the spatial intersection coordinates (x i ,y i ), corresponding to the intersection rotation matrix and intersection transfer matrix (Q i,T i ).

[0118] It should be noted that for the target space in this embodiment and the target space in the above embodiment, when performing the same rendering perception task, the target space is the same space. For example, when performing the rendering perception task of the agent on room A, a coordinate mapping relationship is constructed and a target rendering model is trained based on the scene image collected from room A. Afterwards, the coordinate mapping relationship and the trained target rendering model need to be used in room A for rendering perception, but the coordinate mapping relationship and the trained target rendering model cannot be used for other room B.

[0119] This embodiment can perform spatial division on the target space to obtain the spatial grid grid corresponding to the target space, and then obtain each grid intersection in the spatial grid grid, and determine the spatial intersection coordinates of each grid intersection in the spatial coordinate system corresponding to the target space. Further, based on multiple scene images corresponding to the target space, the scene is reconstructed, and the intersection rotation matrix and intersection transfer matrix of each grid intersection in the perception coordinate system corresponding to the target space are determined according to the reconstruction results, and then a coordinate mapping relationship is constructed. Through the coordinate mapping relationship, the spatial intersection coordinates of each grid intersection in the spatial coordinate system correspond to the intersection rotation matrix and intersection transfer matrix in the perception coordinate system. In this way, the spatial position coordinates in the spatial coordinate system are converted into the rotation matrix and transfer matrix in the perception coordinate system through the coordinate mapping relationship, thereby realizing the conversion from the spatial coordinate system to the perception coordinate system.

[0120] According to an embodiment of the present application, a perception prediction method is provided. The perception prediction can be performed by a computer device (the computer device can be a server or other terminal device with certain processing capabilities), and the computer device has at least storage, calculation and communication functions. Figure 8 A schematic flow chart of the perception prediction method provided in this embodiment is shown. Fig. 9 The perception flow chart of the autonomous intelligent agent is shown below. Figure 8 and Fig. 9 A detailed description of the perception scenario of the autonomous agent in the working room.

[0121] like Fig. 9As shown, the autonomous agent can make decisions based on the perception results rendered by the trained NeRF model, such as stopping movement, accelerating movement, changing the direction of movement, etc. Among them, the SLAM positioning module refers to a module that uses real-time positioning and map building technology to locate the autonomous agent. The module can obtain the positioning coordinates (i.e., spatial position coordinates) of the autonomous agent in the SLAM coordinate system (i.e., spatial coordinate system). The autonomous agent can use the coordinate difference algorithm based on the coordinate mapping relationship to determine the new coordinates in the Colmap coordinate system (i.e., the perception coordinate system): the rotation matrix and the transfer matrix. Furthermore, the autonomous agent can use the NeRF model (i.e., the target rendering model) to render the image based on the rotation matrix and the transfer matrix to obtain a rendered image, and then generate perception results and make decisions based on the perception results, so as to form a closed loop of dynamic perception and decision-making. Specifically, it can include:

[0122] S310: In response to the perception instruction, obtain the moving position in the target space.

[0123] In the embodiment of the present application, the perception instruction may refer to a computer control instruction generated by the autonomous agent according to the perception result after the autonomous agent obtains the perception result. The perception instruction may be used to control the SLAM positioning module to obtain the coordinates of the autonomous agent in the target space, that is, the moving position.

[0124] S320: Obtain a moving rendering image corresponding to the moving position.

[0125] For example, the autonomous agent obtains the coordinates S in the SLAM coordinate system corresponding to the target space. ′ =(x ′ ,y ′ ), the coordinates of each grid intersection in the spatial grid grid of the target space can be traversed to obtain the distance coordinate S ′ The two nearest grid intersection points are denoted as S 1 =(x 1 ,y 1 ) and S 2 =(x 2 ,y 2 ). Further, according to the coordinate mapping relationship, the corresponding coordinates of the two grid intersections in the Colmap coordinate system are obtained, denoted as C 1 =(Q 1 ,T 1 ) and C 2 =(Q 2 ,T 2 ).

[0126] Furthermore, the autonomous agent can ′ =(x ′ ,y ′ ) and S1 =(x 1 ,y 1 ) and S ′ =(x ′ ,y ′ ) and S 2 =(x 2 ,y 2 ) to determine the spatial distance ratio f, and then calculate the corresponding coordinate c of the autonomous agent in the Colmap coordinate system ′ =(q ′ ,T ′ )=(f[Q 2 -T 1 ],f[t 2 -T 2 ]). Furthermore, the autonomous agent can be mapped to the corresponding coordinates C in the Colmap coordinate system. ′ =(Q ′ ,T ′ ) performs image rendering to obtain a moving rendered image corresponding to the moving position.

[0127] See also Fig.10 , Fig.10 A flowchart of coordinate mapping relationship construction and model training is shown. Fig.10 As shown, the process of constructing the coordinate mapping relationship includes collecting scene images in the target space, training a 3D Gaussian NeRF model based on the collected scene images, and then obtaining grid intersections on the spatial grid grid, and constructing a coordinate mapping relationship based on the grid intersections, for example, a one-way mapping table. For example, taking an autonomous intelligent agent as an example, the autonomous intelligent agent can be used to collect scene images in the scene, and at the same time, auxiliary equipment can be used to collect scene images from other perspectives, and then the 3D Gaussian NeRF model is trained based on the collected scene images.

[0128] Furthermore, Colmap is used to reconstruct the collected disordered multiple scene images, extract the features of each image and match them in 3D space. Fig.11 A schematic diagram of scene reconstruction is shown, Fig.11 As shown in Figure 1, Colmap reconstructs the arrangement of the images in the 3D scene after reconstructing the unordered image set. All scene images are matched with feature points, and each scene image can be associated in 3D space. Furthermore, the position and posture of each scene image in the Colmap coordinate system can be determined, that is, the rotation matrix and the transfer matrix. Fig.12 The schematic diagram of Colmap matching shown in the figure shows the feature points of a single scene image and the rotation matrix and transfer matrix of this image in the Colmap coordinate system after the Colmap matching is completed.

[0129] Furthermore, the continuous space of the real world is divided into discrete gridded grids, that is, spatial grid grids. In the spatial grid grid, each grid intersection needs to be equipped with a perception map of the autonomous agent's perspective, which has been collected when the scene image is collected. Furthermore, a one-way mapping table is constructed based on the coordinates of each grid intersection in the SLAM coordinate system and the rotation matrix and transfer matrix of each grid intersection in the Colmap coordinate system.

[0130] S330: Perform motion prediction based on the motion rendering image to obtain a prediction result.

[0131] For example, the autonomous agent can make movement predictions based on the mobile rendering image to determine the perception situation on the current movement trajectory, that is, the prediction results, such as whether there are obstacles, whether there are turns, whether there are stairs, etc. Then, according to the different perception situations indicated by the prediction results, corresponding decisions are generated.

[0132] S340: When the prediction result indicates that there is an obstacle in the target rendered image, an obstacle avoidance decision is determined.

[0133] S350: When the perception result indicates that there is no obstacle in the target rendered image, generate and execute a movement instruction.

[0134] For example, when the prediction result indicates that there is an obstacle in the target rendered image, the autonomous agent can determine an obstacle avoidance decision, which can be used to control itself to avoid the obstacle. When the perception result indicates that there is no obstacle in the target rendered image, the autonomous agent can generate and execute a movement instruction, which is to continue to move along the trajectory.

[0135] See also Fig.13 , which shows a structural block diagram of an image rendering device 400 provided in an embodiment of the present application. The device 400 may include: a position acquisition module 410, used to acquire a target position in a target space; a coordinate determination module 420, used to determine the spatial position coordinates corresponding to the target position in a spatial coordinate system corresponding to the target space; a relationship conversion module 430, used to perform a coordinate system conversion on the spatial position coordinates based on the coordinate mapping relationship corresponding to the target space, and obtain a target rotation matrix and a target transfer matrix of the target position in a perception coordinate system corresponding to the target space; wherein the coordinate mapping relationship is obtained by mapping the spatial intersection coordinates corresponding to each grid intersection in the target space in the spatial coordinate system and the intersection rotation matrix and the intersection transfer matrix corresponding to each grid intersection in the perception coordinate system, and the grid intersection is the intersection in the spatial grid grid corresponding to the target space; an image rendering module 440, used to perform image rendering according to the target rotation matrix and the target transfer matrix through a target rendering model, and obtain a target rendered image corresponding to the target position.

[0136] In some embodiments, the image rendering device 400 may also include: a space division module, which is used to perform spatial division on the target space to obtain a spatial grid grid corresponding to the target space; an intersection acquisition module, which is used to obtain each grid intersection in the spatial grid grid, and determine the spatial intersection coordinates of each grid intersection in the spatial coordinate system corresponding to the target space; a scene reconstruction module, which is used to perform scene reconstruction based on multiple scene images corresponding to the target space, and determine the intersection rotation matrix and intersection transfer matrix of each grid intersection in the perception coordinate system corresponding to the target space according to the reconstruction results; a relationship construction module, which is used to construct a coordinate mapping relationship, through which the spatial intersection coordinates of each grid intersection in the spatial coordinate system correspond to the intersection rotation matrix and intersection transfer matrix in the perception coordinate system.

[0137] In some embodiments, the scene reconstruction module may include: an image acquisition unit, used to acquire multiple scene images corresponding to the target space, the multiple scene images at least including an intersection image of the spatial position of each grid intersection; a scene reconstruction unit, used to reconstruct the scene based on the multiple scene images, and obtain a scene rotation matrix and a scene transfer matrix for each scene image in the perception coordinate system corresponding to the target space; a posture determination unit, used to determine the posture of each grid intersection according to the scene rotation matrix and the scene transfer matrix corresponding to each scene image, and obtain an intersection rotation matrix and an intersection transfer matrix for each grid intersection in the perception coordinate system corresponding to the target space.

[0138] In some embodiments, the posture determination unit can be specifically used to: obtain the intersection image corresponding to each grid intersection; perform image matching on each intersection image with multiple scene images respectively, and use the scene rotation matrix and scene transfer matrix corresponding to the scene image matched with the intersection image in the multiple scene images as the intersection rotation matrix and intersection transfer matrix, to obtain the intersection rotation matrix and intersection transfer matrix of each grid intersection in the perception coordinate system corresponding to the target space.

[0139] In some embodiments, the relationship conversion module 430 may include: an intersection acquisition unit, which is used to acquire each grid intersection in the spatial grid grid corresponding to the target space; a coordinate determination unit, which is used to compare the spatial intersection coordinates of each grid intersection in the spatial coordinate system with the spatial position coordinates to obtain a first intersection coordinate and a second intersection coordinate; wherein the first intersection coordinate is the spatial intersection coordinate of the first grid intersection closest to the target position in the spatial coordinate system, and the second intersection coordinate is the spatial intersection coordinate of the second grid intersection closest to the target position in the spatial coordinate system; a coordinate conversion unit, which is used to perform a coordinate system conversion on the first intersection coordinate based on the coordinate mapping relationship corresponding to the target space to obtain a first rotation matrix and a first transfer matrix of the first grid intersection in the perception coordinate system corresponding to the target space, and to perform a coordinate system conversion on the second intersection coordinate based on the coordinate mapping relationship to obtain a second rotation matrix and a second transfer matrix of the second grid intersection in the perception coordinate system corresponding to the target space; a coordinate determination unit, which is used to determine the target rotation matrix of the target position in the perception coordinate system according to the difference between the first rotation matrix and the second rotation matrix, and to determine the target transfer matrix of the target position in the perception coordinate system according to the difference between the first transfer matrix and the second transfer matrix.

[0140] In some embodiments, the coordinate determination unit may include: a first subunit, used to perform distance ratio calculation based on the spatial position coordinates, the first intersection coordinates and the second intersection coordinates to obtain the spatial distance ratio; a second subunit, used to perform geometric calculation based on the difference between the first rotation matrix and the second rotation matrix and the spatial distance ratio to obtain the target rotation matrix of the target position in the perception coordinate system; a third subunit, used to perform geometric calculation based on the difference between the first transfer matrix and the second transfer matrix and the spatial distance ratio to obtain the target transfer matrix of the target position in the perception coordinate system.

[0141] In some embodiments, the first subunit can be specifically used to: perform distance calculation on the spatial position coordinates and the first intersection coordinates to obtain a first spatial distance between the target position and the first grid intersection; perform distance calculation on the target position coordinates and the second intersection coordinates to obtain a second spatial distance between the target position and the first grid intersection; and use the ratio of the first spatial distance to the second spatial distance as the spatial distance ratio.

[0142] In some embodiments, the image rendering device 400 may further include: a model acquisition module for acquiring a preset rendering model; a model training module for iteratively training the preset rendering model based on multiple scene images corresponding to the target space until the training end condition is met to obtain a trained target rendering model.

[0143] The present application can obtain the target position in the target space, and determine the spatial position coordinates corresponding to the target position in the spatial coordinate system corresponding to the target space, and then perform a coordinate system transformation on the spatial position coordinates based on the coordinate mapping relationship corresponding to the target space, and obtain the target rotation matrix and target transfer matrix of the target position in the perceptual coordinate system corresponding to the target space. In this way, the coordinate system transformation from the spatial coordinate system corresponding to the target space to the perceptual coordinate system corresponding to the target space is realized through the coordinate mapping relationship, and two different types of coordinate systems are connected in series. Furthermore, the image is rendered according to the target rotation matrix and the target transfer matrix through the target rendering model to obtain a target rendered image corresponding to the target position. Therefore, by converting the precise spatial position coordinates obtained in real time in the spatial coordinate system into the rotation matrix and the transfer matrix in the perceptual coordinate system, the image rendering of the target rendering model in the perceptual coordinate system can effectively utilize the real-time precise positioning in the spatial coordinate system, thereby improving the rendering accuracy of the target rendering model, generating high-quality rendering pictures, and improving the quality of scene perception.

[0144] See also Fig.14 , which shows a structural block diagram of a perception prediction device 500 provided in an embodiment of the present application.

[0145] The apparatus 500 may include:

[0146] A sensing response module 510 is used to obtain a moving position in a target space in response to a sensing instruction;

[0147] A mobile rendering module 520, configured to obtain a mobile rendering image corresponding to a mobile position, wherein the mobile rendering image is obtained by image rendering according to the image rendering method according to any one of claims 1 to 8;

[0148] A perception prediction module 530, configured to perform perception prediction based on the mobile rendered image to obtain a prediction result;

[0149] A first decision module 540, configured to determine an obstacle avoidance decision when the prediction result indicates that there is an obstacle in the target rendered image, the obstacle avoidance decision at least including obstacle avoidance movement or stopping movement;

[0150] The second decision module 550 is used to generate and execute a movement instruction when the perception result indicates that there is no obstacle in the target rendered image.

[0151] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules, etc. can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0152] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0153] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0154] like Fig.15 As shown, the embodiment of the present application also provides a computer device 600, which includes a processor 610, a memory 620, a power supply 630 and an input unit 640. The memory 620 stores a computer program. When the computer program is called by the processor 610, the various method steps provided in the above embodiment can be implemented. Those skilled in the art can understand that the structure of the computer device shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0155] The processor 610 may include one or more processing cores. The processor 610 uses various interfaces and lines to connect various parts of the entire battery management system, and by running or executing instructions, programs, instruction sets or program sets stored in the memory 620, calls the data stored in the memory 620, executes various functions of the battery management system and processes data, and executes various functions and processes data of the computer device, thereby controlling the computer device as a whole. Optionally, the processor 610 can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 610 can integrate one or more combinations of a central processing unit 610 (CPU), a graphics processing unit 610 (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 610, but may be implemented separately through a communication chip.

[0156] The memory 620 may include a random access memory 620 (Random Access Memory, RAM), and may also include a read-only memory 620 (Read-Only Memory). The memory 620 may be used to store instructions, programs, instruction sets or program sets. The memory 620 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may also store data (such as a phone book and audio and video data) created by the computer device during use. Accordingly, the memory 620 may also include a memory controller to provide the processor 610 with access to the memory 620.

[0157] The power supply 630 can be logically connected to the processor 610 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 630 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0158] The input unit 640 can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0159] Although not shown, the computer device 600 may also include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 610 in the computer device will load the executable files corresponding to the processes of one or more computer programs into the memory 620 according to the following instructions, and the processor 610 will run the data stored in the memory 620, such as the phone book and audio and video data, to implement the various method steps provided in the aforementioned embodiments.

[0160] like Fig.16 As shown, the embodiment of the present application further provides a computer-readable storage medium 700, in which a computer program 710 is stored. The computer program 710 can be called by a processor to execute various method steps provided in the embodiment of the present application.

[0161] The computer readable storage medium may be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium includes a non-volatile computer readable storage medium (Non-Transitory Computer-Readable Storage Medium). The computer readable storage medium 700 has storage space for a computer program that executes any of the method steps in the above embodiments. These computer programs can be read from or written to one or more computer program products. The computer program can be compressed in an appropriate form.

[0162] According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes various method steps provided in the above embodiments.

[0163] The terms "comprises" and "includes" in the specification of the present application and the above-mentioned drawings and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. The term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0164] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0165] It should be understood that in the description of the embodiments of the present application, the meaning of multiple (or multiple items) is more than two, greater than, less than, exceed, etc. are understood to not include the number, and above, below, within, etc. are understood to include the number.

[0166] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0167] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units.

[0168] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. It should also be understood that the various implementation methods provided in the embodiments of the present application can be arbitrarily combined to achieve different technical effects.

[0169] The above are only preferred embodiments of the present application, and are not intended to limit the present application in any form. Although the present application has been disclosed as above with preferred embodiments, it is not intended to limit the present application. Any technical personnel in the field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present application. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. An image rendering method, characterized in that: The method comprises: Get the target position in the target space; Determining the spatial position coordinates corresponding to the target position in the spatial coordinate system corresponding to the target space; Based on the coordinate mapping relationship corresponding to the target space, the coordinate system of the spatial position is transformed to obtain a target rotation matrix and a target transfer matrix of the target position in the perception coordinate system corresponding to the target space; The coordinate mapping relationship is obtained by mapping the spatial intersection coordinates corresponding to each grid intersection in the target space in the spatial coordinate system and the intersection rotation matrix and intersection transfer matrix corresponding to each grid intersection in the perception coordinate system through the coordinate system, and the grid intersection is the intersection in the spatial grid grid corresponding to the target space; The target rendering model performs image rendering according to the target rotation matrix and the target transfer matrix to obtain a target rendered image corresponding to the target position.

2. The method according to claim 1, characterized in that The coordinate mapping relationship corresponding to the target space is constructed by the following steps: Performing spatial division on the target space to obtain a spatial grid corresponding to the target space; Obtaining each grid intersection in the spatial grid grid, and determining the spatial intersection coordinates of each grid intersection in the spatial coordinate system corresponding to the target space; Reconstructing a scene based on a plurality of scene images corresponding to the target space, and determining an intersection rotation matrix and an intersection transfer matrix of each grid intersection in a perception coordinate system corresponding to the target space according to the reconstruction result; A coordinate mapping relationship is constructed, through which the spatial intersection coordinates of each grid intersection in the spatial coordinate system correspond to an intersection rotation matrix and an intersection transfer matrix in the perception coordinate system.

3. The method according to claim 2, characterized in that The scene reconstruction is performed based on the multiple scene images corresponding to the target space, and the intersection rotation matrix and the intersection transfer matrix of each grid intersection in the perception coordinate system corresponding to the target space are determined according to the reconstruction result, including: Acquire a plurality of scene images corresponding to the target space, wherein the plurality of scene images at least include an intersection image of a spatial position where each grid intersection is located; Perform scene reconstruction based on the multiple scene images to obtain a scene rotation matrix and a scene transfer matrix for each scene image in a perception coordinate system corresponding to the target space; The position and posture of each grid intersection are determined according to the scene rotation matrix and the scene transfer matrix corresponding to each scene image, and the intersection rotation matrix and the intersection transfer matrix of each grid intersection in the perception coordinate system corresponding to the target space are obtained.

4. The method according to claim 3, characterized in that The method of determining the position and posture of each grid intersection point according to the scene rotation matrix and the scene transfer matrix corresponding to each scene image, and obtaining the intersection rotation matrix and the intersection transfer matrix of each grid intersection point in the perception coordinate system corresponding to the target space, includes: Get the intersection image corresponding to each grid intersection; Each intersection image is matched with the multiple scene images respectively, and the scene rotation matrix and the scene transfer matrix corresponding to the scene image matched with the intersection image in the multiple scene images are used as the intersection rotation matrix and the intersection transfer matrix, so as to obtain the intersection rotation matrix and the intersection transfer matrix of each grid intersection in the perception coordinate system corresponding to the target space.

5. The method according to any one of claims 1 to 4, characterized in that: The coordinate system conversion of the spatial position coordinates based on the coordinate mapping relationship corresponding to the target space to obtain a target rotation matrix and a target transfer matrix of the target position in the perception coordinate system corresponding to the target space includes: Obtain each grid intersection in the space grid grid corresponding to the target space; Compare the spatial intersection coordinates of each grid intersection in the spatial coordinate system with the spatial position coordinates to obtain first intersection coordinates and second intersection coordinates; The first intersection coordinates are the spatial intersection coordinates of the first grid intersection closest to the target position in the spatial coordinate system, and the second intersection coordinates are the spatial intersection coordinates of the second grid intersection closest to the target position in the spatial coordinate system; Based on the coordinate mapping relationship corresponding to the target space, the coordinates of the first intersection are transformed into a coordinate system to obtain a first rotation matrix and a first transfer matrix of the first grid intersection in the perception coordinate system corresponding to the target space; based on the coordinate mapping relationship, the coordinates of the second intersection are transformed into a coordinate system to obtain a second rotation matrix and a second transfer matrix of the second grid intersection in the perception coordinate system corresponding to the target space; According to the difference between the first rotation matrix and the second rotation matrix, the target rotation matrix of the target position in the perception coordinate system is determined, and according to the difference between the first transfer matrix and the second transfer matrix, the target transfer matrix of the target position in the perception coordinate system is determined.

6. The method according to claim 5, characterized in that The step of determining a target rotation matrix of the target position in the perception coordinate system according to a difference between the first rotation matrix and the second rotation matrix, and determining a target transfer matrix of the target position in the perception coordinate system according to a difference between the first transfer matrix and the second transfer matrix, comprises: Performing distance ratio calculation based on the spatial position coordinates, the first intersection coordinates, and the second intersection coordinates to obtain a spatial distance ratio; Perform geometric calculation based on the difference between the first rotation matrix and the second rotation matrix and the spatial distance ratio to obtain a target rotation matrix of the target position in the perception coordinate system; A target transfer matrix of the target position in the perception coordinate system is obtained by performing geometric calculation based on the difference between the first transfer matrix and the second transfer matrix and the spatial distance ratio.

7. The method according to claim 6, characterized in that The calculating the distance ratio based on the spatial position coordinates, the first intersection coordinates and the second intersection coordinates to obtain the spatial distance ratio includes: Performing distance calculation on the spatial position coordinates and the first intersection coordinates to obtain a first spatial distance between the target position and the first grid intersection; Performing distance calculation on the target position coordinates and the second intersection coordinates to obtain a second spatial distance between the target position and the second grid intersection; The ratio of the first spatial distance to the second spatial distance is taken as the spatial distance ratio.

8. The method according to claim 2, characterized in that: The method further comprises: Get the preset rendering model; The preset rendering model is iteratively trained based on a plurality of scene images corresponding to the target space until a training end condition is met, thereby obtaining a trained target rendering model.

9. A perception prediction method, characterized in that: The method comprises: Responding to the perception command, obtaining the movement position in the target space; Acquire a moving rendered image corresponding to the moving position, wherein the moving rendered image is obtained by image rendering according to the image rendering method according to any one of claims 1 to 8; Performing perceptual prediction based on the mobile rendered image to obtain a prediction result; When the prediction result indicates that there is an obstacle in the target rendered image, determining an obstacle avoidance decision, the obstacle avoidance decision at least including obstacle avoidance movement or stopping movement; When the perception result indicates that there is no obstacle in the target rendered image, a movement instruction is generated and executed.

10. An image rendering device, characterized in that: The device comprises: A position acquisition module is used to obtain the target position in the target space; A coordinate determination module, used to determine the spatial position coordinates corresponding to the target position in the spatial coordinate system corresponding to the target space; A relationship conversion module, used for performing coordinate system conversion on the spatial position coordinates based on the coordinate mapping relationship corresponding to the target space, to obtain a target rotation matrix and a target transfer matrix of the target position in the perception coordinate system corresponding to the target space; The coordinate mapping relationship is obtained by mapping the spatial intersection coordinates corresponding to each grid intersection in the target space in the spatial coordinate system and the intersection rotation matrix and intersection transfer matrix corresponding to each grid intersection in the perception coordinate system through the coordinate system, and the grid intersection is the intersection in the spatial grid grid corresponding to the target space; An image rendering module is used to perform image rendering according to the target rotation matrix and the target transfer matrix through a target rendering model to obtain a target rendered image corresponding to the target position.

11. A perception prediction device, characterized in that: The device comprises: A perception response module, used to obtain a moving position in a target space in response to a perception instruction; A mobile rendering module, used for acquiring a mobile rendering image corresponding to the mobile position, wherein the mobile rendering image is obtained by image rendering according to the image rendering method according to any one of claims 1 to 8; A perception prediction module, used for performing perception prediction based on the mobile rendered image to obtain a prediction result; A first decision module, configured to determine an obstacle avoidance decision when the prediction result indicates that an obstacle exists in the target rendered image, wherein the obstacle avoidance decision at least includes obstacle avoidance movement or stopping movement; The second decision module is used to generate and execute a movement instruction when the perception result indicates that there is no obstacle in the target rendered image.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions. When the computer-readable instructions are executed by a processor, the image rendering method according to any one of claims 1 to 8 or the perception prediction method according to claim 9 is implemented.

13. A computer device, characterized in that: include: Memory; A processor, wherein a computer program is stored on the memory, and when the computer program is executed by the processor, the image rendering method according to any one of claims 1 to 8, or the perception prediction method according to claim 9 is implemented.

14. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the image rendering method according to any one of claims 1 to 8, or the perception prediction method according to claim 9 is implemented.