3D scene feature extraction method, device, equipment, and computer program

Abstract

The present application discloses a method, apparatus, device, and storage medium for extracting features from a 3D scene, and related embodiments are applicable to various scenes such as artificial intelligence, reducing the cost of extracting 3D scene features. The method includes: projecting a set of visual cone rays from a target character object; when the visual cone rays hit a hit object, returning attribute information of the hit object; performing vector transformation on the object attribute information; performing feature dimension reduction on the basic ray feature vector obtained by the transformation to obtain a ray feature vector; collecting altitude value matrices corresponding to multiple different granularities using the location of the target character object as the collection center; performing feature dimension reduction on the altitude value matrices corresponding to each finer granularity to obtain an altitude map feature vector; and integrating the ray feature vector and the altitude map feature vector into a 3D scene feature.

Description

[Technical Field]

[0001] This application claims priority to a Chinese patent application filed with the China Patent Office on December 1, 2022, bearing application number 202211525532.8 and entitled "Method, device, equipment and storage medium for extracting features of three-dimensional game scenes," the entire contents of which are incorporated herein by reference.

[0002] TECHNICAL FIELD Embodiments of the present application relate to the technical field of image processing, and in particular to feature extraction techniques for three-dimensional scenes. [Background technology]

[0003] In practical applications, 3D scenes (e.g., 3D game scenes) are generally unstructured, have a complex object type, complex object shapes, and are difficult to model as data information. Therefore, how to efficiently and accurately extract scene features from 3D scenes has become a difficult problem. Currently, methods for extracting features from 3D scenes mainly include visual image feature extraction and depth map feature extraction.

[0004] However, there are shortcomings in describing 3D scenes by extracting visual image features. This is because the extracted visual image features are essentially projections of 3D scenes on a 2D plane, which only contain 2D information of the 3D scene. Furthermore, the data dimension of visual image features is high, and performing related machine learning tasks based on these visual image features incurs high learning costs.

[0005] Although depth map features contain three-dimensional information of the scene, the data dimension of depth map features is still high, there is a certain duplication and redundancy in the information contained in pixel points, depth map features only contain distance information, and the related machine learning tasks still consume high costs due to inductive learning on depth map features.

[0006] Combining visual image features and depth map features can extract richer scene features, but there still exists the problem that the dimensionality of the extracted feature data is high and the learning cost of machine learning tasks is high. Summary of the Invention [Problem to be solved by the invention]

[0007] Embodiments of the present application provide a method, device, equipment, and storage medium for extracting features of a 3D scene, ensuring that the extracted 3D scene features are concise, efficient, semantically rich, and can fully describe the 3D scene. By performing machine learning tasks based on the 3D scene features, the machine learning model's ability to perceive 3D scenes is enhanced, the machine learning model's production and development is accelerated, and the learning cost of the machine learning task is reduced. [Means for solving the problem]

[0008] One aspect of an embodiment of the present application provides a method for extracting features from a three-dimensional scene, executed by a computer device, comprising: projecting a set of cone-shaped view pyramid rays from a target character object onto a three-dimensional scene screen; If the visual cone ray hits a hit object, returning object attribute information of the hit object; A step of performing vector transformation on the received object attribute information of each hit object to obtain a basic ray feature vector; performing a feature dimension reduction process on the basic ray feature vector to obtain a ray feature vector; Collecting altitude value matrices corresponding to different granularities based on the location of the target character object as a collection center; Perform feature dimension reduction processing on the altitude value matrix corresponding to each granularity to obtain an altitude map feature vector; and combining the ray feature vector and the elevation map feature vector into a three-dimensional scene feature corresponding to the three-dimensional scene view.

[0009] Another aspect of the present application provides an apparatus for extracting features from a three-dimensional scene, comprising: a processing unit for projecting a set of cone-shaped view pyramid rays from a target character object onto a three-dimensional scene screen; an acquisition unit that returns object attribute information of the hit object when the visual cone ray hits the hit object; The processing unit further performs vector transformation on the received object attribute information of each hit object to obtain a basic ray feature vector; The processing unit further performs a feature dimension reduction process on the basic ray feature vector to obtain a ray feature vector; The processing unit further collects altitude value matrices corresponding to different granularities, with the location of the target character object as the collection center, according to different granularities; The processing unit further performs a feature dimension reduction process on the altitude value matrix corresponding to each granularity to obtain an altitude map feature vector; The apparatus further includes a determining unit for integrating the ray feature vector and the altitude map feature vector into a three-dimensional scene feature corresponding to the three-dimensional scene view.

[0010] Another aspect of the present application provides a computer apparatus, a memory for storing a program; a processor that, when executing a program in memory, implements the method of each of the above aspects; and a bus system communicatively connecting the memory and the processor.

[0011] Another aspect of the present application provides a computer-readable storage medium having stored thereon instructions that, when executed on a computer, cause the computer to perform the method of each of the above aspects. [Effects of the Invention]

[0012] As can be seen from the above technical solutions, the embodiments of the present application have the following beneficial effects: A set of visual cone rays is emitted from the target character object, and when the visual cone rays hit a hit object, the object attribute information of the hit object is returned, and vector transformation is performed on the received object attribute information of each hit object to obtain a corresponding basic ray feature vector, and a feature dimension reduction process is performed on the basic ray feature vector to obtain a ray feature vector, and based on different granularities, altitude value matrices corresponding to each granularity are collected with the location of the target character object as the collection center, and a feature dimension reduction process is performed on the altitude value matrices respectively corresponding to each granularity to obtain an altitude map feature vector, and the ray feature vector and the altitude map feature vector are then integrated into a three-dimensional scene feature corresponding to the three-dimensional scene screen.

[0013] According to the above method, the object attribute information of the hit object returned by the cone ray hitting the object can be quickly obtained by the cone ray, and then the object attribute information of the hit object is converted to obtain the corresponding basic ray feature vector, so that the basic ray feature vector contains the attribute information of the object in the 3D scene. Since the number of emitted cone rays is much lower than the number of pixels such as visual image features and depth map features, the obtained basic ray feature vector is concise and non-redundant. By performing dimensionality reduction processing on the basic ray feature vector, the dimension of the data can be further reduced effectively. In addition, by collecting altitude value matrices corresponding to different granularities, the current target character object By displaying the altitude of the surrounding terrain and performing a dimensionality reduction process on the altitude value matrix to obtain an altitude map feature vector, the dimensionality of the data can be effectively reduced and the terrain and topography information of the 3D scene can be extracted from the altitude map feature vector, thereby making the description of the 3D scene more complete. Furthermore, by integrating the ray feature vector and the altitude map feature vector into the 3D scene features, the 3D scene features are concise and efficient, have rich semantics, and contain terrain and topography information, which can fully describe the 3D scene. The integrated 3D scene features are used as sample features to train a machine learning model, which enhances the machine learning model's ability to detect 3D scenes, speeds up the production and development of machine learning models, and reduces the training costs of machine learning tasks. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a schematic diagram of the architecture of an image data control system in an embodiment of the present application; [Figure 2] 1 is a flowchart of one embodiment of a method for extracting features from a three-dimensional scene in an embodiment of the present application. [Figure 3] 10 is a flowchart of another embodiment of a method for extracting features from a three-dimensional scene in an embodiment of the present application. [Figure 4]10 is a flowchart of another embodiment of a method for extracting features from a three-dimensional scene in an embodiment of the present application. [Figure 5] 10 is a flowchart of another embodiment of a method for extracting features from a three-dimensional scene in an embodiment of the present application. [Figure 6] 10 is a flowchart of another embodiment of a method for extracting features from a three-dimensional scene in an embodiment of the present application. [Figure 7] 10 is a flowchart of another embodiment of a method for extracting features from a three-dimensional scene in an embodiment of the present application. [Figure 8] 10 is a flowchart of another embodiment of a method for extracting features from a three-dimensional scene in an embodiment of the present application. [Figure 9] 10 is a flowchart of another embodiment of a method for extracting features from a three-dimensional scene in an embodiment of the present application. [Figure 10] 10 is a flowchart of another embodiment of a method for extracting features from a three-dimensional scene in an embodiment of the present application. [Figure 11] 10 is a flowchart of another embodiment of a method for extracting features from a three-dimensional scene in an embodiment of the present application. [Figure 12] 10 is a flowchart of another embodiment of a method for extracting features from a three-dimensional scene in an embodiment of the present application. [Figure 13] 10 is a flowchart of another embodiment of a method for extracting features from a three-dimensional scene in an embodiment of the present application. [Figure 14] 1 is a flow diagram of the principle of a method for feature extraction of a three-dimensional scene in an embodiment of the present application; [Figure 15] 1 is a principle schematic diagram of a feature processing of a feature extraction method for a three-dimensional scene in an embodiment of the present application; [Figure 16] 1 is a schematic diagram illustrating the effect of a set of emitted cone rays in a method for extracting features from a three-dimensional scene in an embodiment of the present application; [Figure 17] 1 is a schematic diagram of an envelope of a set of view cone rays emitted by a feature extraction method for a three-dimensional scene in an embodiment of the present application; [Figure 18]1 is a schematic diagram of a cross-sectional distribution of a set of emitted cone rays of a three-dimensional scene feature extraction method in an embodiment of the present application, where the horizontal distance reaches a distance threshold. [Figure 19] 1 is a schematic diagram of an embodiment of a feature extraction device for a three-dimensional scene in an embodiment of the present application; [Figure 20] 1 is a schematic diagram of an embodiment of a computer device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0015] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and drawings of this application are not used to describe a particular order or sequence, but are instead used to distinguish between similar objects. Where appropriate, the data used in this manner may be interchanged, and the embodiments of the application described herein may, for example, be performed in orders other than those illustrated or described herein. Also, the terms "comprising," "corresponding," and any variations thereof are intended to cover non-exclusive inclusions; for example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to those steps or units expressly recited, but may also include other steps or units not expressly recited or inherent in the process, method, product, or apparatus.

[0016] To facilitate understanding, we first explain some terms or concepts to which the embodiments of the present application pertain. 1. Machine learning: Machine learning is a method of automatically analyzing data features to obtain rules and then using these rules to make predictions and inferences about unknown data, and is a means of realizing artificial intelligence. Machine learning includes supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning. 2. Game AI: Game AI is a program or character that combines related artificial intelligence technologies and is introduced into games to enrich the gameplay and improve the player's gaming experience. 3. 3D Game Scene: A 3D game scene is the scene where the characters are located in a 3D game, including the surrounding environment, various objects, etc., and is a collection of all spatial information. 4. Feature extraction: Feature extraction is the construction of information-rich and non-redundant derived values ​​(called feature values) based on the initial data information set. Feature extraction is the process of simplifying and reducing the dimension of the original information, which makes it easier for machine learning models to perform induction and learning. 5. Visual image features: Visual image features are the characteristics of the rendered image of the viewing angle where the character is currently located in the 3D scene. The rendered image generally contains three channels, RGB, and its visual effect basically matches the 3D scene screen visible to the user. After obtaining the visual image features, machine learning models (e.g., game AI) generally use convolution or attention mechanisms to encode the image features, and the encoded parameters are generally trainable. 6. Depth map features: Depth map features are characterized by the depth values ​​of pixel points in the field of view where the character is currently located in the 3D scene. It is necessary to calculate the distance of each object pixel point in the field of view where the character is currently located. Since the depth value matrix can be considered as an image, the processing method of a machine learning model (e.g., game AI) for depth map features is similar to that of visual image features and is generally encoded using a convolution or attention mechanism.

[0017] Here, specific embodiments of the present application involve related data such as object attribute information, and when the above embodiments of the present application are applied to specific products or technologies, user permission or consent must be obtained, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0018] Here, the 3D scene feature extraction method provided in this application can be applied to various scenes, including but not limited to artificial intelligence, cloud technology, maps, smart transportation, etc. By integrating the ray feature vector and the altitude map feature vector into the 3D scene feature, it can be applied to the learning of machine learning models, for example, to scenes with 3D scenes such as first-person shooter games, competitive games, smart transportation games, etc.

[0019] The 3D scene feature extraction method proposed in this application can be applied to the image data control system of Figure 1, which is a schematic architecture diagram of an image data control system according to an embodiment of the application. As shown in Figure 1, the server receives a 3D scene image provided by a terminal device, casts a set of visual cone rays from a target character object, and if any of the visual cone rays hits a hit object, returns the object attribute information of the hit object. Then, it performs vector transformation on the received object attribute information of each hit object to obtain a basic ray feature vector corresponding to the visual cone ray, and performs feature dimension reduction on the basic ray feature vector to obtain a ray feature vector. Furthermore, it collects altitude value matrices corresponding to each granularity at different granularities, using the location of the target character object as the collection center, and performs feature dimension reduction on the altitude value matrices corresponding to each granularity to obtain an altitude map feature vector. Finally, it combines the ray feature vector and the altitude map feature vector into a 3D scene feature corresponding to the 3D scene image.

[0020] According to the above method, the object attribute information of the hit object returned from the visual cone ray of the hit object can be quickly obtained by using the visual cone ray, and then the corresponding basic ray feature vector is converted and obtained according to the object attribute information of the hit object, so that the basic ray feature vector contains the attribute information of the object in the 3D scene. Since the number of emitted visual cone rays is much lower than the number of pixels such as visual image features and depth map features, the obtained basic ray feature vector is concise and non-redundant, and by performing a dimensionality reduction process on the basic ray feature vector, the dimension of the data can be further reduced effectively. In addition, by collecting altitude value matrices corresponding to different granularities, the altitude of the terrain around the current target character object is displayed. A dimensionality reduction process is performed on the altitude value matrix to obtain an altitude map feature vector, which effectively reduces the dimension of the data and allows terrain and topography information of the 3D scene to be extracted from the altitude map feature vector, thereby making the description of the 3D scene more complete. Furthermore, by integrating the ray feature vector and the altitude map feature vector into the 3D scene features, the 3D scene features are concise and efficient, have rich semantics, and contain terrain and topography information, which can fully describe the 3D scene. The integrated 3D scene features are used as sample features to train a machine learning model, which strengthens the machine learning model's ability to perceive 3D scenes, accelerates the production and development of machine learning models, and reduces the training cost of machine learning tasks.

[0021] Although Figure 1 shows only one terminal device, in actual scenarios, more types of terminal devices may participate in the data processing process, including, but not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, and in-vehicle terminals. The specific number and types of terminal devices will be determined based on the actual scenario, and details thereof will not be limited herein. Also, although Figure 1 shows one server, in actual scenarios, multiple servers may participate. In particular, in scenarios involving multi-model training and interaction, the number of servers will be determined based on the actual scenario, and details thereof will not be limited herein.

[0022] Here, in this embodiment, the server may be an independent physical server, a server cluster consisting of multiple physical servers, or a distributed system, or may be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), big data, and artificial intelligence platforms. The terminal device and the server may be directly or indirectly connected via wired or wireless communication, and the terminal device and the server may be connected to form a blockchain network, and this application is not limited thereto.

[0023] Combining the above introduction, the 3D scene feature extraction method of the present application will be introduced below. Referring to FIG. 2, one embodiment of the 3D scene feature extraction method in the present application includes the following steps: In step S101, a set of cone-shaped viewing cone rays is projected from the top of the target character object onto the three-dimensional scene screen.

[0024] Here, the 3D scene feature extraction method provided in the embodiment of the present application extracts 3D scene features corresponding to the 3D game scene screen, and in actual application, the 3D scene features corresponding to the 3D game scene screen can be used to train the game AI.

[0025] Here, game AI can not only provide a better gaming experience, but also support game development, such as predicting game win rates, designing more reasonable barriers and numerical values, and even designing more interesting game characters. Therefore, designing a smarter, more comprehensive game AI is an important part of game development. In 3D scene games, the game AI must be able to effectively sense the current game scene information. The information dimension of the original 3D scene is too high and the content is complex, making it difficult to directly input into a machine learning model. Therefore, features must be extracted from the original 3D scene information. However, 3D game scenes are generally unstructured, have a diverse range of objects, have complex object shapes, and are difficult to model as data information.

[0026] Therefore, in order to better support machine learning game AI based on the extracted 3D scene features, it is necessary to maintain the accuracy and completeness of the recorded 3D scene information, and also to perform dimensional reduction and simplification on the 3D scene information.

[0027] To better extract features from 3D scenes, in this embodiment, tools such as a cone ray tool and an altitude map tool are first constructed, as shown in FIG. 14. The cone ray tool (see FIG. 14) can be developed based on the ray function built into the game engine. After each ray is fired, it returns the hit point, the type of hit object, and material information. The altitude map tool (see FIG. 14) can first derive an altitude description file (e.g., altitude description files for various terrain features such as rainforest, snow, or islands, and terrain such as grass or sand) for the entire game environment (see FIG. 16 for the 3D FPS game) and provide a corresponding altitude value search interface. Here, the tool construction step only needs to be executed once after the game environment is determined, and multiple tools can be used in the same game environment.

[0028] After constructing the visual cone ray tool and the altitude map tool, the feature collection step can be performed. For example, during a test run of a game (the 3D FPS game of FIG. 16), a visual cone ray tool (the visual cone ray tool of FIG. 14) is used to cast a set of visual cone rays from the top of the game character object (i.e., the head of the game character in the 3D FPS game of FIG. 16) to the 3D game scene screen for each frame, according to the distribution of the visual cones (a set of cone-shaped visual cone rays is cast from the head of the game character in the 3D FPS game of FIG. 16), thereby obtaining ray results in real time. Because ray hit information needs to be dynamically calculated, visual cone ray feature collection must also be performed in real time as the game progresses. The number and distribution of rays in a set of visual cone rays can be set according to actual needs and are not specifically limited here.

[0029] Here, in actual application, the method provided by the embodiments of the present application can be applied to extract features of 3D game scene images, and can also be applied to other scenes containing 3D scene images to extract features of 3D scene images in other scenes, and there is no limitation on the application scenes of the method provided by the embodiments of the present application.

[0030] In step S102, if the visual cone ray hits a hit object, the object attribute information of the hit object is returned.

[0031] In an embodiment of the present application, after a set of visual cone rays is emitted from a target character object, if there is a visual cone ray in the set of visual cone rays that hits an object in the three-dimensional scene (in an embodiment of the present application, an object hit by a visual cone ray is called a hit object), the visual cone ray of the hit object will return object attribute information corresponding to the hit point, such as the position information, type information and material information of the hit object.

[0032] Specifically, after entering the feature collection stage, for example, according to the distribution of the visual cones, a set of visual cone rays is emitted from the top of the game character object (the head of the game character in the 3D FPS game of FIG. 16) (a set of cone-shaped visual cone rays is emitted from the head of the game character in the 3D FPS game of FIG. 16), and then the ray results are acquired in real time. If the set of visual cone rays includes a visual cone ray that hits an object in the 3D game scene, the visual cone ray of the hit object will return object attribute information corresponding to the hit point to the server. For example, in the 3D FPS game of FIG. 16, a set of cone-shaped visual cone rays is emitted from the head of the game character, and if one visual cone ray hits an obstacle (a container in FIG. 16), the visual cone ray that hits the obstacle will return the position information of the hit object corresponding to the hit point (for example, the hit point that currently hits the container housing, the 3D FPS game of FIG. 16). The object attribute information such as the coordinates on the current screen of an FPS game, type information of the hit object (e.g., the casing of a container), and material information (e.g., high-strength steel) is returned to the server.

[0033] In step S103, vector conversion is performed on the received object attribute information of each hit object to obtain a basic ray feature vector.

[0034] In an embodiment of the present application, after the visual cone ray returns the object attribute information of the hit objects, it performs feature processing on the received object attribute information, and performs vector transformation on the object attribute information of all the received hit objects to obtain a basic ray feature vector, so that the 3D scene features can be better obtained based on the basic ray feature vector.

[0035] Specifically, as shown in Figure 14, after returning the object attribute information of the hit objects, a feature processing step is performed. In the feature processing step, the data obtained in the feature collection step, i.e., the object attribute information of all received hit objects, is encoded, i.e., vector converted, to obtain basic ray feature vectors, which are then input into the subsequent machine learning model, allowing the machine learning game AI to perform inductive learning on the extracted 3D scene features. Here, generally, the parameters of the encoding process can also be trained and optimized.

[0036] In step S104, a feature dimension reduction process is performed on the basic ray feature vector to obtain a ray feature vector.

[0037] In an embodiment of the present application, after obtaining a basic ray feature vector, in order to enable the machine learning model to better perform inductive learning on the extracted 3D scene features using the feature vector input into the subsequent machine learning model, in this embodiment, a feature dimension reduction process is performed on the basic ray feature vector to obtain a ray feature vector with lower dimensions, thereby realizing the dimension reduction and simplification of the feature vector, thereby better realizing the dimension reduction and simplification of the 3D scene features extracted based on the ray feature vector thereafter.

[0038] Specifically, as shown in FIG. 14, after obtaining the basic ray feature vector, a feature dimension reduction process is performed on the basic ray feature vector through a neural network (as shown in FIG. 14, a high-density network is used for the ray vector), to obtain the ray feature vector, specifically, a one-dimensional ray feature vector.

[0039] In step S105, based on different granularities, the altitude value matrix corresponding to each granularity is collected with the location of the target character object as the collection center.

[0040] In an embodiment of the present application, after constructing a visual cone ray tool and an altitude map tool, altitude value matrices corresponding to each granularity are collected in all directions around the current character coordinates of the target character object based on different granularities, so that the terrain and topography information in the 3D environment can be better obtained based on the altitude value matrices, and the machine learning model can be better supported to perceive the 3D scene.

[0041] Specifically, after constructing the visual cone ray tool and the altitude map tool, the feature collection step is carried out. For example, during the test run of a game (the 3D FPS game of FIG. 16), the altitude map tool (the altitude map tool of FIG. 14) is used for each frame of the 3D game scene screen to obtain the current character coordinates of the game character object in the 3D game scene screen (the current orientation of the game character in the 3D FPS game of FIG. 16 and the current coordinates on the screen). An altitude value search interface is constructed in the altitude map tool to search for the topographic map corresponding to the current character coordinates. Furthermore, using the current character coordinates of the game character object as the collection center, grid maps corresponding to each granularity are collected in all directions around the collection center based on different granularities, for example, granularity a. Then, the altitude value matrix corresponding to each grid map is obtained based on the altitude value search interface.

[0042] Here, since the terrain state is generally stable and does not change with the operation of the game, the acquisition of the altitude value matrix and the extraction of the altitude map features can be performed offline, which further reduces the computational overhead and reduces the development cost of the game AI to a certain extent.

[0043] In step S106, a feature dimension reduction process is performed on the altitude value matrix corresponding to each granularity to obtain an altitude map feature vector.

[0044] In an embodiment of the present application, after obtaining the altitude value matrix corresponding to each granularity, the feature vectors input to the subsequent machine learning model are used to allow the machine learning model to better perform inductive learning on the extracted 3D scene features. In this embodiment, a feature dimension reduction process is performed on the altitude value matrix corresponding to each granularity to obtain an altitude map feature vector. By inputting the altitude map feature vector to the subsequent machine learning model, the machine learning model can then perform inductive learning on the extracted 3D scene features. Here, generally, parameters of the encoding process can also be learned and optimized.

[0045] Specifically, as shown in Figure 14, after obtaining the altitude value matrix corresponding to each granularity, a feature dimension reduction process is performed on the altitude value matrix corresponding to all granularities through a neural network (using a convolutional network as shown in Figure 14) to obtain an altitude map feature vector, specifically, a one-dimensional altitude feature vector.

[0046] In step S107, the ray feature vector and the height map feature vector are integrated into a 3D scene feature corresponding to the 3D scene view.

[0047] In an embodiment of the present application, after obtaining the ray feature vector and the altitude map feature vector, the ray feature vector and the altitude map feature vector are integrated into a 3D scene feature corresponding to the 3D scene screen, so that the machine learning model can then perform inductive learning on the extracted 3D scene feature.

[0048] Specifically, as shown in FIG. 14, after obtaining the ray feature vector and the altitude map feature vector, the ray feature vector and the altitude map feature vector are stitched together to obtain the 3D game scene features corresponding to the 3D game scene screen, thereby enabling the game AI based on machine learning to perform inductive learning.

[0049] An embodiment of the present application provides a feature extraction method for a 3D scene. According to the above method, the object attribute information of the hit object returned from the cone ray of the hit object can be quickly obtained by using the cone ray, and then the object attribute information of the hit object is converted to obtain the corresponding basic ray feature vector. Thus, the basic ray feature vector contains the attribute information of the object in the 3D scene. Since the number of emitted cone rays is much lower than the number of pixels such as visual image features and depth map features, the obtained basic ray feature vector is concise and non-redundant. Furthermore, by performing a dimensionality reduction process on the basic ray feature vector, the dimension of the data can be further reduced effectively. In addition, by collecting altitude value matrices corresponding to different granularities, the current target The altitude of the terrain around the character object is displayed, and a dimensionality reduction process is performed on the altitude value matrix to obtain an altitude map feature vector, which effectively reduces the dimension of the data and allows terrain and topography information of the 3D scene to be extracted from the altitude map feature vector, thereby making the description of the 3D scene more complete. Furthermore, by integrating the ray feature vector and the altitude map feature vector into the 3D scene features, the 3D scene features are concise and efficient, are semantically rich, and contain terrain and topography information, which can fully describe the 3D scene. The integrated 3D scene features are used as sample features to train a machine learning model, which strengthens the machine learning model's ability to detect 3D scenes, speeds up the production and development of machine learning models, and reduces the training costs of machine learning tasks.

[0050] Optionally, based on the embodiment corresponding to FIG. 2 above, in another preferred embodiment of the 3D scene feature extraction method provided by the embodiment of the present application, the object attribute information of the hit object includes position information of the hit object, type information and material information of the hit object, and as shown in FIG. 3 , step S103 of performing vector transformation on the received object attribute information of each hit object to obtain a basic ray feature vector includes the following steps: In step S301, vector conversion is performed on the object attribute information of each hit object, including the position information, type information, and material information of the hit object, to obtain an information vector corresponding to the visual cone ray that hits the hit object; In step S302, the information vectors corresponding to the respective cone rays are integrated into a basic ray feature vector.

[0051] In an embodiment of the present application, after returning the object attribute information of the hit object, feature processing is performed on the received object attribute information, i.e., vector conversion is performed on the position information, type information and material information of the hit object to obtain an information vector corresponding to the visual cone ray hitting the hit object, and the information vectors corresponding to the visual cone rays of all hit objects are integrated into a basic ray feature vector, so that the 3D scene features can be better obtained based on the basic ray feature vector thereafter.

[0052] Specifically, as shown in FIG. 14, after returning the object attribute information of the hit object, a feature processing step is performed. In the feature processing step, the data obtained in the feature collection step, i.e., the position information of the hit object corresponding to the visual cone ray of the hit object, the type information and material information of the hit object are encoded, i.e., vector converted, to obtain an information vector corresponding to the visual cone ray. Then, the information vectors corresponding to the visual cone rays of all hit objects are stitched to obtain a basic ray feature vector. Thereafter, the basic ray feature vector is input into a machine learning model, and inductive learning is performed on the 3D scene features extracted by the machine learning game AI.

[0053] In this way, according to the above method, vector transformation is performed on the position information, type information, and material information of the object hit by the visual cone ray to obtain an information vector corresponding to the visual cone ray, and then the information vectors corresponding to the visual cone ray of each hit object are integrated to obtain a basic ray feature vector, so that the obtained basic ray feature vector can more accurately describe the objects existing in the 3D scene, that is, realize a sufficient description of the 3D scene.

[0054] Optionally, based on the embodiment corresponding to FIG. 3 above, in another preferred embodiment of the 3D scene feature extraction method provided by the embodiment of the present application, as shown in FIG. 4, step S301 of performing vector transformation on the position information, type information and material information of the hit object to obtain an information vector corresponding to the visual cone ray hitting the hit object includes steps S401 to S403, and step S302 includes step S404; In step S401, vector normalization processing is performed on the position information of the hit object to obtain a position vector; In step S402, feature coding is performed on the type information and material information of the hit object to obtain a coded vector; In step S403, the position vector and the encoding vector are stitched to obtain an information vector corresponding to the visual cone ray that hits the hit object; In step S404, the information vectors corresponding to each of the cone rays are stitched sequentially to obtain a basic ray feature vector.

[0055] In an embodiment of the present application, after returning the object attribute information of the hit object, feature processing is performed on the received object attribute information, i.e., vector normalization processing is performed on the position information of the hit object to obtain a position vector, feature encoding is performed on the type information and material information of the hit object to obtain an encoded vector, and then the position vector and the encoded vector are stitched together to obtain an information vector corresponding to the visual cone ray that hit the hit object, and then the information vectors corresponding to the visual cone rays of all hit objects are stitched together in sequence to obtain a ray feature vector.

[0056] Specifically, as shown in Figure 14, after returning the object attribute information of the hit object, a feature processing step is performed, and in the feature processing step, a vector normalization process is performed on the data acquired in the feature collection step, i.e., the position information of the hit object. Specifically, first, a series of operations such as horizontal movement, zooming, and rotation of the three-dimensional coordinates are performed on the coordinate information of the hit point of the visual cone ray to obtain a position vector corresponding to the coordinate information; or, a normalization process is performed on the coordinate information, for example, a vector conversion is performed on the coordinate information based on a linear function (matrix multiplication, for example, a matrix composed of base vectors * coordinate value = vector) to obtain a position vector corresponding to the coordinate information; other vector normalization processing methods may also be adopted, and are not specifically limited here.

[0057] Furthermore, feature encoding is performed on the type information and material information of the hit object, specifically, one-hot encoding is adopted for the type information and material information, and then embedding processing is performed on the encoded data to obtain an embedding vector corresponding to the type information (object type embedding in Figure 15) and an embedding vector corresponding to the material information (object material embedding in Figure 15).The embedding vector corresponding to the type information (object type embedding in Figure 15) and the embedding vector corresponding to the material information (object material embedding in Figure 15) are then stitched into an encoded vector, or other feature encoding methods may be adopted for the type information and material information, and no specific limitations are imposed here.

[0058] Furthermore, as shown in FIG. 15, after obtaining a position vector corresponding to the position information of the hit object (the ray hit point coordinate vector in FIG. 15) and an encoding vector corresponding to the type information and material information of the hit object (the object type embedding and object material embedding in FIG. 15), the position vector and the encoding vector are stitched together (for example, stitched in the order of the ray hit point coordinate vector, object type embedding, and object material embedding in FIG. 15) to obtain an information vector (one ray information in FIG. 15) of one visual cone ray (i.e., the visual cone ray that hit the hit object). Then, by stitching the information vectors corresponding to the visual cone rays of all hit objects in a predetermined fixed order (for example, stitching the set of visual cone rays in the right diagram of FIG. 17 counterclockwise from outside to inside), a basic ray feature vector is obtained. Thereafter, the basic ray feature vector is input into the machine learning model, so that the machine learning model can perform inductive learning on the extracted 3D scene features.

[0059] In this way, according to the above method, a position vector is generated based on the position information of the object hit by the visual cone ray, and an encoding vector is generated based on the type information and material information of the hit object. Then, the position vector and the encoding vector are stitched together to obtain an information vector corresponding to the visual cone ray. Then, the information vectors corresponding to the visual cone ray of each hit object are stitched together in sequence to obtain a basic ray feature vector. This ensures that the obtained basic ray feature vector can more accurately describe the objects existing in the 3D scene, that is, achieve a sufficient description of the 3D scene.

[0060] Optionally, based on the embodiment corresponding to FIG. 2 above, in another preferred embodiment of the 3D scene feature extraction method provided by the embodiments of the present application, as shown in FIG. 5, step S105 of collecting altitude value matrices corresponding to each granularity based on different granularities, with the location of the target character object as the collection center, includes the following steps:

[0061] In step S501, based on different granularities, N*N grids corresponding to each granularity are collected around the location of the target character object as the collection center, where N is an integer greater than or equal to 1; In step S502, an altitude value matrix corresponding to each granularity is generated based on the N*N grid corresponding to each granularity.

[0062] In an embodiment of the present application, after constructing a visual cone ray tool and an altitude map tool, N*N grids corresponding to each granularity are collected in all directions around the current character coordinates of the target character object based on different granularities, with the current character coordinates of the target character object as the collection center. An altitude value matrix corresponding to each granularity is then generated based on the N*N grids corresponding to each granularity, thereby better obtaining terrain and topography information in a 3D environment based on the altitude value matrix, and further supporting machine learning models to perceive 3D scenes.

[0063] Specifically, after constructing the visual cone ray tool and the altitude map tool, the feature collection step is carried out. For example, during the test run of a game (3D FPS game in FIG. 16), the altitude map tool (altitude map tool in FIG. 14) is used to calculate the current character coordinates of the target character object in the 3D game scene screen (3D The current orientation of the game character in the FPS game and the current on-screen coordinates are obtained, and an altitude value search interface in the altitude map tool is constructed to search for the terrain map (game terrain in Figure 18) corresponding to the current character coordinates.Furthermore, with the current character coordinates of the target character object as the collection center, different granularities (for example, granularity a) are used as the grid side lengths, and grid maps corresponding to each granularity are collected in all four directions around the collection center (for example, as shown in Figure 18, a 4*4 grid with a grid side length of a is collected).Then, based on the altitude value search interface, altitude values ​​corresponding to the N*N grids corresponding to each granularity (for example, as shown in Figure 18, a 4*4 grid with a grid side length of a is collected) are obtained, and an altitude value matrix (for example, the 4*4 altitude value matrix in Figure 18) is generated.

[0064] In this way, according to the above method, the current character coordinates of the target character object are taken as the collection center, and N*N grids corresponding to different granularities are collected in all directions around it. An altitude value matrix corresponding to each granularity is generated based on the N*N grids corresponding to each granularity. The altitude value matrix corresponding to different granularities is used to represent the altitude of the terrain around the current target character object, and the terrain and topography information of the 3D scene is extracted, thereby making the description of the 3D scene more complete.

[0065] Optionally, based on the embodiment corresponding to FIG. 5 above, in another preferred embodiment of the 3D scene feature extraction method provided by the embodiment of the present application, as shown in FIG. 6, based on different granularities, the position where the target character object is located is taken as the collection center, and N*N grids corresponding to each granularity are collected in all directions around it. Step S501 includes steps S601 to S602, and step S502 includes step S603; In step S601, different granularities are set as unit lengths of grids to be collected having different sizes; In step S602, according to the unit lengths of the grids to be collected having different sizes, N*N grids corresponding to the unit lengths of the grids to be collected are collected in all directions around the location of the target character object as the collection center; In step S603, altitude values ​​corresponding to the center points of each grid among the N*N grids are obtained, and an altitude value matrix corresponding to the granularity corresponding to the N*N grids is generated based on the altitude values.

[0066] In an embodiment of the present application, after constructing a visual cone ray tool and an altitude map tool, first, different granularities are used as the unit length of the grid to be collected having different sizes, and N*N grids corresponding to the unit length of each grid to be collected are collected in all directions around the current character coordinates of the target character object as the collection center according to the unit length of the grid to be collected having different sizes. Then, altitude values ​​corresponding to the grid center points of the N*N grids are obtained, and altitude value matrices corresponding to each granularity are generated based on the altitude values. This can then be used to better obtain topographical and geographical information in a 3D environment based on the altitude value matrix, and further, it can better support machine learning models to perceive 3D scenes.

[0067] Specifically, after constructing the visual cone ray tool and the altitude map tool, the feature collection step is carried out. For example, during the test run of a game (the 3D FPS game in FIG. 16), the altitude map tool (the altitude map tool in FIG. 14) is used for the 3D game scene screen of each frame to obtain the current character coordinates of the target character object on the 3D game scene screen (the current orientation of the game character in the 3D FPS game in FIG. 16 and the current coordinates on the screen), and an altitude value search interface in the altitude map tool is constructed to search for the topographic map (the game terrain in FIG. 18) corresponding to the current character coordinates.

[0068] Furthermore, different granularities (e.g., granularity a in Figure 18) are used as unit lengths of grids to be collected of different sizes (e.g., unit length a of the grid to be collected in Figure 18), and further, with the current character coordinates of the target character object as the collection center, grid diagrams corresponding to each granularity (e.g., as shown in Figure 18, a 4*4 grid with a side length of a is collected) are collected on all four sides around the collection center according to unit lengths of grids to be collected of different sizes (e.g., unit length a of the grid to be collected in Figure 18).

[0069] Furthermore, based on the altitude value search interface, altitude values ​​corresponding to the center points of each grid of the N*N grid corresponding to each granularity (for example, as shown in Figure 18, collect the center points of each grid of a 4*4 grid with a side length of a) are obtained, and an altitude value matrix is ​​generated according to the correspondence between the center points of each grid of the N*N grid corresponding to each granularity and the position of each element in the N*N altitude value matrix (for example, the altitude values ​​corresponding to the center points of each grid of the N*N grid corresponding to each granularity are written into a 4*4 altitude value matrix in a corresponding manner to obtain the 4*4 altitude value matrix of Figure 18).

[0070] In this way, according to the above method, the unit length of the grid to be collected, which has different sizes, is determined for different granularities, and N*N grids are collected based on this. Furthermore, the altitude value of the center point of each grid among the N*N grids is written into the corresponding N*N matrix to obtain the altitude value matrix corresponding to this granularity, thereby ensuring that the determined altitude value matrix more accurately reflects the altitude of the terrain around the current target character object.

[0071] Optionally, based on the embodiment corresponding to FIG. 2 above, in another preferred embodiment of the 3D scene feature extraction method provided by the embodiment of the present application, as shown in FIG. 7 , step S106 of performing feature dimension reduction processing on the altitude value matrix corresponding to each granularity to obtain an altitude map feature vector includes the following steps:

[0072] In step S701, stitching and tensor transformation processes are performed on the altitude value matrix corresponding to each granularity to obtain an altitude map feature tensor; In step S702, a feature dimension reduction process is performed on the altitude map feature tensor to obtain an altitude map feature vector.

[0073] In the embodiment of the present application, after obtaining the altitude value matrix corresponding to each granularity, the feature vectors input to the subsequent machine learning model are used to better perform inductive learning on the extracted 3D scene features. In order to do this, in the embodiment, the altitude value matrix corresponding to all granularities is subjected to a feature dimension reduction process to obtain an altitude map feature vector (e.g., a one-dimensional altitude map feature vector). Here, generally, the parameters of the encoding process can also be learned and optimized.

[0074] Specifically, as shown in FIG. 15, after obtaining the altitude value matrices corresponding to each granularity (for example, as shown in FIG. 15, altitude value matrices corresponding to three different granularities), stitching and tensor transformation processes are performed on the altitude value matrices corresponding to all granularities. For example, if there are altitude value matrices corresponding to three different granularities a, b, and c, the three altitude value matrices are 4*4 altitude value matrices with the same number of matrices, so there is no need to zero-pad the matrices to obtain matrices with the same number of matrices. The matrix after stitching or merging can be obtained by stitching or merging the array horizontally and vertically using the cat function, or other stitching or merging methods can be adopted, and no specific limitations are imposed here. Then, tensor transformation is performed on the matrix after stitching or merging. For example, first, the matrix after stitching or merging is converted into numpy type data using a Numpy array. Then, the converted numpy type data is substituted into the function formula torch.from_numpy(np_array) using the torch.from_numpy function to generate an advanced map feature tensor (for example, an advanced map feature tensor with different granularity levels as shown in FIG. 15). Furthermore, other tensor transformation methods, such as the torch.tensor function, may also be adopted, and are not specifically limited here.

[0075] Furthermore, after obtaining the elevation map feature tensor (for example, the elevation map feature tensor with different granularities as shown in FIG. 15 ), a feature dimension reduction process is performed on the elevation map feature tensor (for example, the elevation map feature tensor with different granularities as shown in FIG. 15 ) via a neural network (using a convolutional network CNN as shown in FIG. 14 ) to obtain an elevation map feature vector.

[0076] Here, in order to expand the detection range of features, in this embodiment, by selecting collection granularities with different sizes, altitude value matrices corresponding to different receptive fields are obtained, and the topography and terrain of the 3D scene are more fully described based on the altitude map tensor formed from the altitude value matrices corresponding to the different receptive fields.

[0077] Optionally, based on the embodiment corresponding to Fig. 2 above, step S104 of performing feature dimension reduction processing on the basic ray feature vector to obtain a ray feature vector specifically includes a step of performing feature dimension reduction processing on the basic ray feature vector through a neural network to obtain a ray feature vector.Step S106 of performing feature dimension reduction processing on the altitude value matrix corresponding to each granularity to obtain an altitude map feature vector specifically includes a step of performing feature dimension reduction processing on the altitude value matrix corresponding to each granularity through a neural network to obtain an altitude map feature vector.

[0078] In this way, the feature dimension reduction process of the basic ray feature vector and the feature dimension reduction process of the altitude value matrix corresponding to each granularity are completed through the corresponding neural network, ensuring that the ray feature vector after the feature dimension reduction process better retains the effective information in the original basic ray feature vector, and the altitude map feature vector after the feature dimension reduction process better retains the effective information in the altitude value matrix corresponding to each granularity, so that the ray feature vector and the altitude map feature vector can more accurately describe the corresponding 3D scene.

[0079] Optionally, based on the embodiment corresponding to FIG. 2 above, in another preferred embodiment of the 3D scene feature extraction method provided by the embodiments of the present application, as shown in FIG. 8, the neural networks processing the basic ray feature vectors and the altitude value matrices (specifically, altitude map feature tensors) respectively corresponding to each granularity may be the same convolutional neural network, and step S104 of performing feature dimension reduction processing on the basic ray feature vectors to obtain ray feature vectors includes step S801, and step S106 of performing feature dimension reduction processing on the altitude value matrices respectively corresponding to each granularity to obtain altitude map feature vectors includes step S802; In step S801, a feature dimension reduction process is performed on the basic ray feature vector through a convolutional neural network to obtain a ray feature vector; In step S802, a feature dimension reduction process is performed on the altitude value matrix corresponding to each granularity via a convolutional neural network to obtain an altitude map feature vector.

[0080] In an embodiment of the present application, after obtaining the basic ray feature vector and the elevation map feature tensor (which are generated based on the elevation value matrices corresponding to each granularity), the neural networks used to perform dimension reduction on the basic ray feature vector and the elevation map feature tensor may be the same convolutional neural network. That is, the basic ray feature vector is subjected to feature dimension reduction through the convolutional neural network to obtain a one-dimensional ray feature vector, and the elevation map feature tensor is subjected to feature dimension reduction through the convolutional neural network to obtain a one-dimensional elevation map feature vector, thereby reducing the need to construct and use different neural network frameworks, reducing the amount of calculation to a certain extent, and improving the efficiency of obtaining 3D game scene features.

[0081] Specifically, after obtaining the basic ray feature vector and the elevation map feature tensor, the basic ray feature vector and the elevation map feature tensor are subjected to dimensionality reduction, respectively. Then, based on the dimensionally reduced ray feature vector and the elevation map feature vector, the dimension-reduced and simplified 3D scene features are better obtained. Therefore, the neural networks used for the dimensionality reduction of the basic ray feature vector and the elevation map feature tensor may be the same convolutional neural network. That is, the feature dimensionality reduction is performed on the basic ray feature vector through the convolutional neural network. Since the basic ray feature vector is a feature vector obtained by feature normalization and stitching in a fixed order, the basic ray feature vector can be convolved using a 1xN convolution core in the convolutional neural network based on the magnitude of the feature value to obtain a one-dimensional ray feature vector. The feature dimensionality reduction of the elevation map feature tensor through the convolutional neural network may also be performed using a conventional convolutional operation method, that is, by sequentially processing through a convolutional layer, a pooling layer, a fully connected layer, etc., to obtain a one-dimensional elevation map feature vector.

[0082] Optionally, based on the embodiment corresponding to FIG. 2 above, in another preferred embodiment of the 3D scene feature extraction method provided by the embodiment of the present application, as shown in FIG. 9, the neural network includes a first neural network and a second neural network, and the first neural network and the second neural network are different neural networks. Step S104 of performing feature dimension reduction processing on the basic ray feature vector to obtain the ray feature vector includes step S901; and step S106 of performing feature dimension reduction processing on the altitude value matrix respectively corresponding to each granularity to obtain the altitude map feature vector includes step S902; In step S901, a feature dimension reduction process is performed on the basic ray feature vector through a first neural network to obtain a ray feature vector; In step S902, a feature dimension reduction process is performed on the altitude value matrix corresponding to each granularity via a second neural network to obtain an altitude map feature vector.

[0083] In an embodiment of the present application, after obtaining the basic ray feature vector and the elevation map feature tensor (which are generated based on the elevation value matrices corresponding to each granularity), the neural networks used to perform dimension reduction on the basic ray feature vector and the elevation map feature tensor may be different, i.e., the first neural network performs feature dimension reduction on the basic ray feature vector to obtain the ray feature vector, and the second neural network performs feature dimension reduction on the elevation value matrices corresponding to all granularities to obtain the elevation map feature vector. By performing dimension reduction on different types of feature vectors through different neural networks, the corresponding dimension-reduced features can be obtained better and more accurately, thereby improving the accuracy of obtaining 3D scene features.

[0084] The first neural network and the second neural network are different neural networks, and the first neural network may be a high-density network, such as the DNN neural network of FIG. 15, and the second neural network may be a convolutional neural network, such as the CNN neural network of FIG. 15.

[0085] Specifically, as shown in FIG. 15, after obtaining the basic ray feature vector and the elevation map feature tensor, the basic ray feature vector and the elevation map feature tensor are subjected to a dimension reduction process respectively. Then, based on the dimension-reduced ray feature vector and the elevation map feature vector, the dimension-reduced and simplified 3D scene features can be better obtained. Therefore, the neural networks used to perform the dimension reduction process respectively on the basic ray feature vector and the elevation map feature tensor can be different neural networks. Because the obtained basic ray feature vector is not image data, and a convolutional neural network can more easily learn information about adjacent positions in image data, for the ray feature vector that does not necessarily contain adjacent position information, a high-density network, i.e., a first neural network, for example, the DNN neural network of FIG. 15, can be used to perform a feature dimension reduction process on the basic ray feature vector, and data related to atomic positions in the basic ray feature vector is analyzed by a multi-layer perceptron (MLP), and a forward propagation algorithm is applied to the detected data to obtain a one-dimensional ray feature vector. Since the altitude map feature tensor belongs to the image data, a convolutional neural network, i.e., a second neural network, which can more easily learn the information of adjacent positions in the image data, can be used to perform feature dimension reduction processing on the altitude map feature tensor, that is, a normal convolutional operation method can be adopted, that is, processing through a convolutional layer, a pooling layer, a fully connected layer, etc. in sequence to obtain a one-dimensional altitude map feature vector.

[0086] Optionally, based on the embodiment corresponding to FIG. 2 above, in another preferred embodiment of the 3D scene feature extraction method provided by the embodiments of the present application, as shown in FIG. 10, step S101 of projecting a set of viewing cone rays from a target character object includes step S1001, and step S102 includes step S1002; In step S1001, a set of frustum rays is emitted from the top of the target character object by simulating a frustum viewing angle; In step S1002, after each of the cone rays emitted by simulating the cone field of view reaches the length threshold, if any of the cone rays hits a hit object, the object attribute information of the hit object is returned.

[0087] In an embodiment of the present application, when extracting features from original 3D scene information, 3D scenes are generally unstructured, have a diverse variety of objects, and have complex object shapes, making them difficult to model as data information. Therefore, in this embodiment, a set of view cone rays is emitted from the top of a target character object by simulating a view cone angle. After each of the view cone rays in the set of view cone rays emitted by simulating the view cone angle reaches a length threshold, if any view cone ray hits an object, the object attribute information of the hit object is returned, so as to better grasp the objects placed in the 3D environment and the attribute information of these objects. Thereafter, 3D scene features can be better extracted based on the object attribute information, and machine learning can be better supported based on the extracted 3D scene features.

[0088] Specifically, in order to better detect objects arranged in a 3D environment and the attribute information of these objects, as shown in FIG. 14, in this embodiment, tools such as a cone ray tool and an altitude map tool are first constructed. After the cone ray tool and the altitude map tool are constructed, the feature collection step is carried out. For example, during a test run of a game (the 3D FPS game of FIG. 16), a cone ray tool (the cone ray tool of FIG. 14) is used for each frame of the 3D game scene screen to simulate the cone field of view angle and emit a set of cone rays from the top of the target character object (the head of the game character in the 3D FPS game of FIG. 16). That is, as shown in the left diagram of FIG. 17, a set of cone rays with a cone-shaped envelope is emitted according to the cone ray direction and cone ray distribution, and the ray results are obtained in real time.

[0089] Furthermore, after each of the cone rays in the set of cone rays emitted by simulating the cone field of view angle reaches a length threshold (10 m as shown in the left diagram of FIG. 17), if any of the cone rays hits an object, the object attribute information of the hit object is returned. For example, in the set of cone rays (as shown in the left diagram of FIG. 17, a set of cone rays having a conical envelope is emitted from the head of the game character), if any of the cone rays hits an object, the cone ray of the hit object returns object attribute information corresponding to the hit point to the server. For example, in the 3D In an FPS game, a set of cone-shaped cone rays is emitted from the head of a game character (as shown in the left diagram of FIG. 17, a set of cone rays with a conical envelope is emitted from the head of the game character), and the cone field of view angle is simulated. When each of the cone rays in the set of emitted cone rays reaches a length threshold (10 m as shown in the left diagram of FIG. 17), assuming that one ray hits an object 10 m away (the grass in FIG. 16), the cone ray that hit the object returns object attribute information such as the position information of the hit object corresponding to the hit point (e.g., the hit point that currently hits the grass, the coordinates on the current screen in the 3D FPS game in FIG. 16), type information of the hit object (e.g., grass ground) and material information (e.g., soil or soft soil) to the server.

[0090] Optionally, based on the embodiment corresponding to FIG. 10 above, in another preferred embodiment of the feature extraction method for a 3D game scene provided by the embodiments of the present application, as shown in FIG. 11, step S1001 of simulating a frustum viewing angle and releasing a set of frustum rays from the top of the target character object includes step S1101, and step S1002 includes step S1102; In step S1101, with the top part of the target character object as the center of a circle, obtain p uniformly distributed ray directions, and emit M ray clusters for each ray direction, the envelope of the M ray clusters is conical, each ray cluster includes p visual cone rays, and the p visual cone rays of each ray cluster are uniformly distributed in M ​​concentric circles, where p is an integer greater than or equal to 2 and M is an integer greater than or equal to 1; In step S1102, after p cone rays of each of the M ray clusters reach the length threshold, if any cone rays hit a hit object, the object attribute information of the hit object is returned.

[0091] In an embodiment of the present application, when extracting features from the original 3D scene information, 3D scenes are generally unstructured, have a diverse range of objects, and have complex object shapes, making them difficult to model as data information. Therefore, in this embodiment, the top of the target character object is taken as the center of the circle, and p uniformly distributed ray directions are first obtained. M ray clusters are then emitted for each ray direction. After p visual cone rays of each ray cluster among the M ray clusters reach a length threshold, if any visual cone rays hit an object, the object attribute information of the hit object is returned. This allows for better detection of objects placed in the environment and the attribute information of these objects. Subsequently, 3D scene features can be better extracted based on the object attribute information, and machine learning can be better supported based on the extracted 3D scene features.

[0092] A ray cluster is a ray group containing p visual cone rays, and the p visual cone rays of one emitted ray cluster are uniformly distributed in a circle, i.e., the emitted M ray clusters are uniformly distributed in M ​​concentric circles, so that the envelope of the M ray clusters is conical.

[0093] Specifically, in order to better detect objects arranged in a 3D environment and the attribute information of these objects, as shown in FIG. 14, in this embodiment, first, tools such as a cone ray tool and an altitude map tool are constructed. After constructing the cone ray tool and the altitude map tool, the feature collection step is carried out. For example, during a test run of a game (3D FPS game in FIG. 16), the cone ray tool (cone ray tool in FIG. 14) is used to capture the top part of a target character object (3D FPS game in FIG. 16) for each frame of the 3D game scene screen. First, p uniformly distributed ray directions are obtained by taking the center of the circle as the head of a game character in an FPS game. If p and M are both 5, 25 emitted cone rays (here, if p is 5 and M is 3, 15 emitted cone rays are obtained, or if p is 5 and M is 7, 35 emitted cone rays are obtained). These ray directions are uniformly distributed on the circle, i.e., one circle is divided into 0°, 72°, 144°, 216°, and 288°. Furthermore, assuming that the length threshold is 10m as shown in the left diagram of Figure 17, the maximum radius of the concentric circle is 10m, and the emitted M ray clusters are uniformly distributed among the M concentric circles, so the radii of each concentric circle are 2m, 4m, 6m, 8m and 10m, respectively. Then, the included ray angles emitted in each ray cluster are calculated as 9°, 18°, 27°, 36° and 45°. Therefore, the ray direction of each visual cone ray can be determined based on the degree on the circle and the included ray angle.

[0094] Furthermore, as shown in the right figure of Figure 17, M ray clusters are emitted for each ray direction. For example, as shown in the right figure of Figure 17, ray clusters a, b, c, d, and e are emitted for 0°, 72°, 144°, 216°, and 288°, respectively, and five concentric circles and 25 visual cone rays uniformly distributed among the concentric circles can be obtained. When the horizontal distance of the emission of the 25 visual cone rays reaches the length threshold (10 m, as shown in the left figure of Figure 17), the cross-sectional distribution shown in the right figure of Figure 17 can be obtained.

[0095] Furthermore, if any of the visual cone rays hits an object, the object attribute information of the hit object is returned. For example, if any of the visual cone rays hits an object in the set of visual cone rays (as shown in the left diagram of FIG. 17, a set of visual cone rays having a conical envelope is emitted from the head of the game character), the visual cone rays that hit the object return object attribute information corresponding to the hit point to the server. For example, the 3D In an FPS game, a set of cone-shaped cone rays is emitted from the head of a game character (as shown in the left diagram of Figure 17, a set of cone rays with a conical envelope is emitted from the head of the game character), and the cone field of view angle is simulated. Suppose each of the cone rays in the set of emitted cone rays reaches a length threshold (10 m as shown in the left diagram of Figure 17), and one cone ray hits an object 10 m away (the grass in Figure 16). The cone ray that hit the object returns object attribute information, such as the position information of the hit object corresponding to the hit point (e.g., the hit point that currently hits the grass, the coordinates on the current screen in the 3D FPS game in Figure 16), type information of the hit object (e.g., grass ground) and material information (e.g., soil or soft soil), to the server.

[0096] Here, after obtaining the information vector of one visual cone ray (one ray information in Figure 15) based on the obtained object attribute information, the information vectors corresponding to the visual cone rays of all hit objects are stitched together in a predetermined fixed order, for example, stitching in a counterclockwise order from outside to inside, or stitching in a counterclockwise order from inside to outside, etc. Regarding the stitching in a counterclockwise direction from outside to inside, for a set of view cone rays in the right diagram of FIG. 17, first, in a counterclockwise direction, i.e., for the rays in the cross-sectional distribution in the right diagram of FIG. 17, ray cluster a is stitched in a counterclockwise direction starting from 0°, i.e., stitching in the order of a0, a1, a2, a3 to a4. Similarly, other ray clusters b, c, d and e are stitched counterclockwise, and then in a counterclockwise direction from outside to inside, i.e., stitching from ray cluster a to ray cluster e in the cross-sectional distribution in the right diagram of FIG. 17, to obtain a basic ray feature vector. Thereafter, the basic ray feature vector is input into a machine learning model, and the machine learning model can perform inductive learning on the extracted 3D scene features.

[0097] For example, optionally based on the embodiment corresponding to FIG. 2 above, in another preferred embodiment of the 3D scene feature extraction method provided by the embodiments of the present application, as shown in FIG. 12 , step S107 of integrating the ray feature vector and the altitude map feature vector into a 3D scene feature corresponding to the 3D scene screen includes the following steps: In step S1201, the ray feature vector and the height map feature vector are stitched sequentially to obtain the 3D scene feature corresponding to the 3D scene image.

[0098] In an embodiment of the present application, after obtaining the ray feature vector and the altitude map feature vector, the ray feature vector and the altitude map feature vector are sequentially stitched to obtain the 3D scene features corresponding to the 3D scene image, and the 3D scene features with a fixed feature order are obtained. Thereafter, the machine learning model can perform inductive learning based on the 3D scene features with a fixed feature order.

[0099] Specifically, as shown in FIG. 15, after obtaining the ray feature vector and the altitude map feature vector, the ray feature vector and the altitude map feature vector are stitched sequentially, that is, stitched in the order from the ray feature vector to the altitude map feature vector in FIG. 15, to stitch and obtain the 3D scene features corresponding to the 3D scene image, so that the machine learning model can perform inductive learning.

[0100] Optionally, based on the embodiment corresponding to FIG. 2 above, in another preferred embodiment of the 3D scene feature extraction method provided by the embodiments of the present application, as shown in FIG. 13 , after step S107 of integrating the ray feature vector and the altitude map feature vector into a 3D scene feature corresponding to the 3D scene screen, the method further includes the following steps: In step S1301, the 3D scene features corresponding to the 3D scene image are taken as feature training samples; In step S1302, the feature training sample is input into a winning probability prediction model, and the winning probability value of the target character object that will win next time is estimated by the winning probability prediction model; In step S1303, reinforcement learning is performed on the win rate prediction model based on the probability value of the next win and the win expectancy, and the model parameters are updated.

[0101] In an embodiment of the present application, after obtaining the 3D scene features corresponding to the 3D scene screen, the 3D scene features corresponding to the 3D scene screen are used as feature training samples in a machine learning model. For example, in this embodiment, when extracting 3D scene features from a 3D game scene screen, the 3D scene features are used as feature training samples for inductive learning of a game AI through machine learning. That is, the feature training samples are input into a win probability prediction model, and the win probability prediction model is used to estimate the probability value of the target character object's next win. Then, based on the next win probability value and win expectation, reinforcement learning is performed on the win probability prediction model to update the model parameters, so that the win probability of the game AI can be better planned based on the trained win probability prediction model thereafter.

[0102] Specifically, the win probability prediction model may be implemented as a deep neural network with parameters θ using a strategy πθ(at|st). The win probability prediction model uses the observations and actions st = o1:t, a1:t-1 received in the game and the 3D scene features corresponding to each frame of the 3D scene as feature training samples, as model inputs, and selects the action at as output. Internally, the model senses the game environment based on the feature training samples, encodes the observation results ot through convolutional and fully connected layers, merges them into a vector, processes them using a deep sequence network, and finally maps them into a probability distribution of actions, i.e., the probability value of the target character object's next win. Furthermore, the model obtains expectation or value, i.e., win expectation, based on the state value function or action value function. Reinforcement learning is then performed on the win probability prediction model using an iterative algorithm, the probability value of the next win, and the win expectation until the model converges, updating the model parameters. This allows the game AI's win probability to be better predicted based on the trained win probability prediction model.

[0103] Here, after obtaining the 3D scene features corresponding to the 3D scene screen, the 3D scene features corresponding to the 3D scene screen may be used as feature training samples for inductive learning of a machine learning model, and may also be applied to reinforcement learning of other models, such as clear barrier prediction, high score prediction, and obstacle prediction, or to supervised learning of other models, and there are no specific limitations here.

[0104] In this way, by training a machine learning model using 3D scene features that are concise, efficient, and semantically rich, and that contain topographical and geographical information as sample features, the machine learning model's ability to perceive 3D scenes is enhanced, the manufacturing and development of machine learning models is accelerated, and the training costs for machine learning tasks are reduced.

[0105] The three-dimensional scene feature extraction device of the present application will be described in detail below. Please refer to FIG. 19. FIG. 19 is a schematic diagram of an embodiment of a three-dimensional scene feature extraction device in the embodiment of the present application. The three-dimensional scene feature extraction device 20 includes: a processing unit 201 for projecting a set of cone-shaped view pyramid rays from a target character object onto a three-dimensional scene screen; an acquisition unit 202 for returning object attribute information of the hit object when the visual cone ray hits the hit object; The processing unit 201 further performs vector transformation on the received object attribute information of each hit object to obtain a basic ray feature vector; The processing unit 201 further performs a feature dimension reduction process on the basic ray feature vector to obtain a ray feature vector; The processing unit 201 further collects altitude value matrices corresponding to different granularities, with the location of the target character object as the collection center; The processing unit 201 further performs a feature dimension reduction process on the altitude value matrix corresponding to each granularity to obtain an altitude map feature vector; The method further includes a determining unit 203 for integrating the ray feature vector and the altitude map feature vector into a 3D scene feature corresponding to the 3D scene view.

[0106] Optionally, based on the embodiment corresponding to FIG. 19 above, the object attribute information of the hit object includes position information of the hit object, type information and material information of the hit object. In another embodiment of the 3D scene feature extraction device provided by the embodiment of the present application, the processing unit 201 specifically: performing vector conversion on the object attribute information of each of the hit objects, including the position information, type information, and material information of the hit object, to obtain an information vector corresponding to the visual cone ray that hits the hit object; The information vectors corresponding to the respective cone rays are integrated into the basic ray feature vector.

[0107] Optionally, based on the embodiment corresponding to the above FIG. 19 , in another embodiment of the 3D scene feature extraction device provided by the embodiment of the present application, the processing unit 201 specifically: A vector normalization process is performed on the position information of the hit object to obtain a position vector; Feature coding is performed on the type information and material information of the hit object to obtain an encoded vector; The position vector and the encoding vector are stitched to obtain an information vector corresponding to the visual cone ray that hits the hit object; Specifically, the processing unit 201 sequentially stitches the information vectors corresponding to the respective cone rays to obtain a basic ray feature vector.

[0108] Optionally, based on the embodiment corresponding to the above FIG. 19 , in another embodiment of the 3D scene feature extraction device provided by the embodiment of the present application, the processing unit 201 specifically: According to different granularities, the location of the target character object is taken as the collection center, and N*N grids corresponding to each granularity are collected in all four directions around the location of the target character object, where N is an integer greater than or equal to 1; Based on the N*N grid corresponding to each granularity, an altitude value matrix corresponding to each granularity is generated.

[0109] Optionally, based on the embodiment corresponding to the above FIG. 19 , in another embodiment of the 3D scene feature extraction device provided by the embodiment of the present application, the processing unit 201 specifically: The different granularities are defined as unit lengths of grids to be collected having different sizes, According to the unit length of the grids to be collected having different sizes, N*N grids corresponding to the unit lengths of the grids to be collected are collected in all directions around the location of the target character object as the collection center; Specifically, the processing unit obtains an altitude value corresponding to the center point of each grid among the N*N grids, and generates an altitude value matrix corresponding to a granularity corresponding to the N*N grids based on the altitude values.

[0110] Optionally, based on the embodiment corresponding to the above FIG. 19 , in another embodiment of the 3D scene feature extraction device provided by the embodiment of the present application, the processing unit 201 specifically: Perform stitching and tensor transformation on the altitude value matrix corresponding to each granularity to obtain an altitude map feature tensor; A feature dimension reduction process is performed on the elevation map feature tensor to obtain an elevation map feature vector.

[0111] Optionally, based on the embodiment corresponding to the above FIG. 19 , in another embodiment of the 3D scene feature extraction device provided by the embodiment of the present application, the processing unit 201 specifically: performing a feature dimension reduction process on the basic ray feature vector via a neural network to obtain the ray feature vector; A feature dimension reduction process is performed on the altitude value matrix corresponding to each granularity via a neural network to obtain the altitude map feature vector.

[0112] Optionally, according to the embodiment corresponding to FIG. 19 above, the neural network that performs feature dimension reduction processing on the basic ray feature vector and the neural network that performs feature dimension reduction processing on the altitude value matrix corresponding to each granularity are the same convolutional neural network. In another embodiment of the 3D scene feature extraction device provided by the embodiment of the present application, the processing unit 201 specifically: Perform feature dimension reduction processing on the basic ray feature vector through a convolutional neural network to obtain a ray feature vector; Specifically, the processing unit 201 performs a feature dimension reduction process on the altitude value matrix corresponding to each granularity through a convolutional neural network to obtain an altitude map feature vector.

[0113] Optionally, based on the embodiment corresponding to the above FIG. 19 , the neural network includes a first neural network and a second neural network, and the first neural network and the second neural network are different neural networks. In another embodiment of the 3D scene feature extraction device provided by the embodiment of the present application, the processing unit 201 specifically: performing a feature dimension reduction process on the basic ray feature vector through a first neural network to obtain a ray feature vector; Specifically, the processing unit 201 performs a feature dimension reduction process on the altitude value matrix corresponding to each granularity through a second neural network to obtain an altitude map feature vector.

[0114] Optionally, based on the embodiment corresponding to the above FIG. 19 , in another embodiment of the 3D scene feature extraction device provided by the embodiment of the present application, the processing unit 201 specifically: A set of ray cone rays is emitted from the top of the target character object, simulating the ray cone angle. Specifically, the acquisition unit 202 simulates the cone field of view angle, and after each of the cone rays emitted reaches a length threshold, if any of the cone rays hits a hit object, returns object attribute information of the hit object.

[0115] Optionally, based on the embodiment corresponding to the above FIG. 19 , in another embodiment of the 3D scene feature extraction device provided by the embodiment of the present application, the processing unit 201 specifically: Using the top part of the target character object as the center of a circle, obtain p uniformly distributed ray directions, and emit M ray clusters in each ray direction, the envelope of the M ray clusters is conical, and each ray cluster includes p viewing cone rays, and the p viewing cone rays of each ray cluster are uniformly distributed in M ​​concentric circles, where p is an integer greater than or equal to 2, and M is an integer greater than or equal to 1; Specifically, the acquisition unit 202 returns object attribute information of a hit object if the p cone rays of each ray cluster among the M ray clusters reach the length threshold.

[0116] Optionally, based on the embodiment corresponding to the above FIG. 19 , in another embodiment of the 3D scene feature extraction device provided by the embodiment of the present application, the determining unit 203 specifically: The ray feature vector and the altitude map feature vector are stitched sequentially to obtain the 3D scene feature corresponding to the 3D scene image.

[0117] In a possible design, in an implementation of another aspect of an embodiment of the present application, The determining unit 203 further determines the 3D scene features corresponding to the 3D scene image as feature training samples; The processing unit 201 further inputs the feature training sample into a winning probability prediction model, and estimates the probability value of the target character object winning next time through the winning probability prediction model; The processing unit 201 further performs reinforcement learning on the win rate prediction model based on the probability value of the next win and the win expectation, and updates the model parameters.

[0118] Another aspect of the present application provides a schematic diagram of another computer device. As shown in FIG. 20, FIG. 20 is a structural schematic diagram of a computer device provided by an embodiment of the present application. The computer device 300 may vary greatly in terms of configuration and performance, and may include one or more main processors (central processing units, or CPUs) 310 (e.g., one or more processors), memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 331 or data 332. The memory 320 and storage media 330 may be temporary or permanent storage. The program stored in the storage media 330 includes one or more modules (not shown), each of which includes a series of instructions for the computer device 300. Furthermore, the main processor 310 is arranged to communicate with the storage media 330, causing the computer device 300 to execute the series of instructions stored in the storage media 330.

[0119] The computing device 300 may further include one or more power sources 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 333, such as Windows Server®, Mac OS X®, Unix®, Linux®, FreeBSD®, etc.

[0120] The above-mentioned computing device 300 further performs steps in the embodiments corresponding to FIGS.

[0121] Another aspect of the present application provides a computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the method described in the embodiments of FIGS.

[0122] Another aspect of the present application provides a computer program product including a computer program that, when executed by a processor, performs the steps of the method described in the embodiments of FIGS.

[0123] As will be understood by those skilled in the art, for ease and conciseness of description, the specific operation steps of the above-described systems, devices and units can be referred to the corresponding steps of the above-described method embodiments, and will not be described in detail here.

[0124] In some embodiments provided by the present application, the disclosed systems, devices, and methods may be realized in other forms. For example, the device embodiments described above are merely schematic, and the division of the units is merely a division of logical functions. In actual implementation, other division schemes may exist, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not implemented. Furthermore, the couplings, or direct couplings, or communication connections between the units shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, which may be electrical, mechanical, or other forms.

[0125] The units described as individual components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Based on actual needs, some or all of the units can be selected to achieve the objective of the solution of this embodiment.

[0126] Furthermore, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated units may be realized in the form of hardware or in the form of software functional units.

[0127] The integrated unit may be realized in the form of a software functional unit and stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the essence of the technical solution of the present application, or a part that contributes to the prior art, or all or a part of the technical solution, may be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or a part of the steps of the methods described in each embodiment of the present application. The above-mentioned storage medium includes various media that can store program code, such as a U disk, a portable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. [Explanation of symbols]

[0128] 20 Feature Extraction Device for 3D Scenes 201 Acquired Units 202 Processing Unit 203 Generating Units 300 Computer Equipment 310 Main Processor 320 memory 330 Storage medium 331 Application Program 332 Data 333 Operating Systems 340 power supply 350 Wired or Wireless Network Interface 360 Input / Output Interface

Claims

1. 1. A method for three-dimensional scene feature extraction implemented by a computing device, comprising: projecting a set of cone-shaped view pyramid rays from the target character object onto the three-dimensional scene screen; If the visual cone ray hits a hit object, returning object attribute information of the hit object; A step of performing vector transformation on the received object attribute information of each hit object to obtain a basic ray feature vector; performing a feature dimension reduction process on the basic ray feature vector to obtain a ray feature vector; Collecting altitude value matrices corresponding to different granularities based on the location of the target character object as a collection center; performing a feature dimension reduction process on the altitude value matrix corresponding to each granularity to obtain an altitude map feature vector; and combining the ray feature vector and the elevation map feature vector into a three-dimensional scene feature corresponding to the three-dimensional scene representation.

2. the object attribute information of the hit object includes position information of the hit object, type information and material information of the hit object; The step of performing vector transformation on the received object attribute information of each hit object to obtain a basic ray feature vector includes: a step of performing vector conversion on the object attribute information of each of the hit objects, including the position information, type information, and material information of the hit object, to obtain an information vector corresponding to the visual cone ray that hits the hit object; and a step of combining information vectors corresponding to each of the cone rays into the base ray feature vector.

3. the step of performing vector conversion on the position information, type information, and material information of the hit object included in the object attribute information to obtain an information vector corresponding to a visual cone ray that hits the hit object, performing vector normalization processing on the position information of the hit object to obtain a position vector; performing feature coding on the type information and material information of the hit object to obtain an encoded vector; and stitching the position vector and the encoding vector to obtain an information vector corresponding to a visual cone ray that hits the hit object; The step of integrating information vectors corresponding to the respective cone rays into the basic ray feature vectors comprises: The method of claim 2 , comprising sequentially stitching information vectors corresponding to each of the cone rays to obtain the base ray feature vector.

4. The step of collecting altitude value matrices corresponding to different granularities based on the location of the target character object as a collection center includes: a step of collecting N*N grids corresponding to each granularity in all directions around the location of the target character object based on different granularities, where N is an integer equal to or greater than 1; The method according to claim 1 , further comprising: generating an altitude value matrix corresponding to each of the granularities based on an N*N grid corresponding to each of the granularities.

5. The step of collecting N*N grids corresponding to each granularity in all directions around the location of the target character object based on different granularities includes: The different granularities are unit lengths of grids to be collected having different sizes; and collecting N*N grids corresponding to the unit lengths of the grids to be collected having different sizes, in all directions around the location of the target character object as the collection center, according to the unit lengths of the grids to be collected, The step of generating an altitude value matrix corresponding to each of the granularities based on an N*N grid corresponding to each of the granularities includes:

5. The method of claim 4, further comprising obtaining an altitude value corresponding to a center point of each grid among the N*N grids, and generating an altitude value matrix corresponding to the granularity corresponding to the N*N grid based on the altitude values.

6. The step of performing a feature dimension reduction process on the altitude value matrix corresponding to each granularity to obtain an altitude map feature vector includes: performing stitching and tensor transformation processing on the elevation value matrices corresponding to each granularity to obtain elevation map feature tensors; The method of claim 1 , further comprising: performing a feature dimensionality reduction process on the elevation map feature tensor to obtain the elevation map feature vector.

7. The step of performing a feature dimension reduction process on the basic ray feature vector to obtain a ray feature vector includes: performing a feature dimension reduction process on the basic ray feature vector through a neural network to obtain the ray feature vector; The step of performing a feature dimension reduction process on the altitude value matrix corresponding to each granularity to obtain an altitude map feature vector includes: The method according to claim 1 , further comprising: performing a feature dimension reduction process on the altitude value matrices corresponding to each granularity via a neural network to obtain the altitude map feature vectors.

8. the neural network for performing a feature dimension reduction process on the basic ray feature vector and the neural network for performing a feature dimension reduction process on the altitude value matrices corresponding to each granularity are the same convolutional neural network; The step of performing a feature dimension reduction process on the basic ray feature vector through a neural network to obtain the ray feature vector includes: performing a feature dimension reduction process on the basic ray feature vector through the convolutional neural network to obtain the ray feature vector; The step of performing a feature dimension reduction process on the altitude value matrix corresponding to each granularity via a neural network to obtain the altitude map feature vector includes: The method of claim 7 , further comprising: performing a feature dimension reduction process on the altitude value matrices corresponding to each granularity via the convolutional neural network to obtain the altitude map feature vectors.

9. the neural network includes a first neural network and a second neural network, the first neural network and the second neural network being different neural networks; The step of performing a feature dimension reduction process on the basic ray feature vector through a neural network to obtain the ray feature vector includes: performing a feature dimension reduction process on the basic ray feature vector via the first neural network to obtain the ray feature vector; The step of performing a feature dimension reduction process on the altitude value matrix corresponding to each granularity via a neural network to obtain the altitude map feature vector includes: The method of claim 7 , further comprising: performing a feature dimension reduction process on the altitude value matrices corresponding to each granularity via the second neural network to obtain the altitude map feature vectors.

10. said step of casting a set of view cone rays from a target character object comprises: emitting a set of said frustum rays from a top portion of said target character object by simulating a frustum field of view angle; When the visual cone ray hits a hit object, the step of returning object attribute information of the hit object includes:

10. The method according to claim 1, further comprising the step of returning object attribute information of a hit object if any of the cone rays hits a hit object after each of the cone rays emitted by simulating the cone field of view angle reaches a length threshold.

11. The step of simulating a frustum viewing angle and projecting a set of frustum rays from a top portion of the target character object includes: obtaining p ray directions uniformly distributed around a top part of the target character object as a circle center, and emitting M ray clusters in each ray direction, wherein the envelope of the M ray clusters is conical, each ray cluster includes p view cone rays, and the p view cone rays of each ray cluster are uniformly distributed in M ​​concentric circles, where p is an integer equal to or greater than 2, and M is an integer equal to or greater than 1; the step of returning object attribute information of the hit object if any of the cone rays hits a hit object after each of the cone rays emitted by simulating the cone field of view angle reaches a length threshold, 11. The method of claim 10, further comprising: returning object attribute information of a hit object if any of the cone rays hits a hit object after p cone rays of each ray cluster among the M ray clusters reach the length threshold.

12. the step of integrating the ray feature vector and the elevation map feature vector into a three-dimensional scene feature corresponding to the three-dimensional scene view includes: The method of claim 1 , further comprising sequentially stitching the ray feature vector and the elevation map feature vector to obtain a 3D scene feature corresponding to the 3D scene view.

13. After the step of integrating the ray feature vector and the elevation map feature vector into a 3D scene feature corresponding to the 3D scene view, the method further comprises: a step of taking three-dimensional scene features corresponding to the three-dimensional scene image as feature training samples; inputting the feature training sample into a win probability prediction model, and estimating a probability value of the target character object winning next time through the win probability prediction model; The method according to any one of claims 1 to 12, further comprising: a step of performing reinforcement learning on the win rate prediction model to update model parameters based on the probability value of the next win and the win expectation.

14. 1. A device for extracting features from a three-dimensional scene, comprising: a processing unit for projecting a set of cone-shaped view pyramid rays from a target character object onto a three-dimensional scene screen; an acquisition unit that returns object attribute information of the hit object when the visual cone ray hits the hit object; The processing unit further performs vector transformation on the received object attribute information of each hit object to obtain a basic ray feature vector; The processing unit further performs a feature dimension reduction process on the basic ray feature vector to obtain a ray feature vector; The processing unit further collects altitude value matrices corresponding to different granularities, with the location of the target character object as the collection center, according to different granularities; The processing unit further performs a feature dimension reduction process on the altitude value matrix corresponding to each granularity to obtain an altitude map feature vector; The apparatus further includes a determining unit for integrating the ray feature vector and the elevation map feature vector into a three-dimensional scene feature corresponding to the three-dimensional scene view.

15. A computer device comprising: a memory in which a computer program is stored, a processor, and a bus system; The processor, when executing the computer program, performs the steps of the method of any one of claims 1 to 13, The computer device is characterized in that the bus system communicatively connects the memory and the processor.

16. A computer-readable storage medium having stored thereon a computer program, the computer program being adapted to implement the steps of the method according to any one of claims 1 to 13 when executed by a processor.

17. A computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 13.

Images