Object classification method and apparatus, and computer program product, device and storage medium

Abstract

Provided in the embodiments of the present disclosure are an object classification method and apparatus, and a computer program product, a device and a storage medium. The method comprises: acquiring a three-dimensional model of an object to be classified, wherein the three-dimensional model carries texture information; projecting the three-dimensional model according to a plurality of preset projection viewing angles, so as to obtain a plurality of frames of RGB images; selecting a target RGB image from among the plurality of frames of RGB images, wherein the information amount of valid information in the target RGB image meets a preset condition, and the valid information is information which can be used for identifying the material of said object; and performing feature extraction on the target RGB image, and determining, on the basis of an extracted feature, a material category to which said object belongs. In this way, the material category of the object to be classified can be distinguished, and when the material category is distinguished, a relatively accurate classification result can also be obtained in the case of a relatively small amount of calculation.

Description

Object classification method, device, computer program product, equipment and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on March 27, 2024, with application number 202410362658.0, and invention name “Object classification method, device, computer program product, equipment and storage medium”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present disclosure relates to the field of data processing technology, and in particular to an object classification method, apparatus, computer program product, device, and storage medium. Background Art

[0003] In some scenarios, objects to be classified can be automatically classified based on their three-dimensional models to determine the category of the objects to be classified for subsequent applications, such as determining whether the objects to be classified are teeth or faces. Currently, when classifying objects to be classified, the three-dimensional models used are usually three-dimensional mesh models, that is, the three-dimensional models only have shape information and topological information, but no texture information. Furthermore, the features extracted from the projected two-dimensional image are only the shape features or topological structure features of the objects to be classified, which can only be used to distinguish objects of different shape categories, such as teeth and faces, but cannot distinguish objects of the same category but different materials. For example, it is impossible to distinguish whether the tooth model is a real tooth model or a tooth model made of paraffin. However, there are some application scenarios that require classification of objects of the same category but different materials to determine the material category of the object, and current methods cannot meet this requirement. Summary of the Invention

[0004] The present disclosure provides an object classification method, apparatus, computer program product, device, and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for object classification is provided, the method comprising:

[0006] Obtaining a three-dimensional model of the object to be classified, wherein the three-dimensional model carries texture information;

[0007] Projecting the three-dimensional model according to a plurality of preset projection viewing angles to obtain a plurality of RGB image frames;

[0008] Selecting a target RGB image from the multiple RGB images, wherein the amount of valid information in the target RGB image meets a preset condition, and the valid information is information that can be used to identify the material of the object to be classified;

[0009] Feature extraction is performed on the target RGB image, and the material category of the object to be classified is determined based on the extracted features.

[0010] According to a second aspect of the present disclosure, a computer program product is provided, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the method mentioned in the first aspect is implemented.

[0011] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a processor, a memory, and computer instructions stored in the memory for execution by the processor. When the processor executes the computer instructions, the method mentioned in the first aspect above can be implemented.

[0012] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method mentioned in the first aspect is implemented.

[0013] In the disclosed embodiments, to classify the material of an object to be classified, the disclosed embodiments contemplate combining the object's texture information with the object's material when determining its category. Specifically, a three-dimensional model carrying the texture information can be projected to obtain an RGB image. Classification of the object based on the RGB image can thereby distinguish the material category of the object to be classified and expand its application scenarios. Furthermore, to reduce computational effort, improve recognition speed, and ensure the accuracy of classification results, after projecting the three-dimensional model carrying the texture information at multiple preset projection angles to obtain multiple RGB images, RGB images containing a high amount of valid information can be selected for subsequent feature extraction and material category determination. Valid information refers to information that can be used to identify the material of the object to be classified. In this way, the material category of the object to be classified can be distinguished, meeting the needs of certain specific scenarios. Furthermore, when distinguishing material categories, relatively accurate classification results can be achieved with minimal computational effort.

[0014] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0016] FIG1 is a schematic diagram of a related art for determining the category of an object to be classified based on a three-dimensional model.

[0017] FIG2 is a schematic diagram of an object classification method according to an embodiment of the present disclosure.

[0018] FIG3 is a flowchart of an object classification method according to an embodiment of the present disclosure.

[0019] 4a and 4b are schematic diagrams of setting projection viewing angles according to two embodiments of the present disclosure.

[0020] FIG5 is a schematic diagram of determining RGB values ​​of mesh vertices of a three-dimensional mesh model based on texture coordinates according to an embodiment of the present disclosure.

[0021] FIG6 is a schematic diagram of a projection sphere of a three-dimensional model according to an embodiment of the present disclosure.

[0022] FIG7 a is a schematic diagram of setting a target point on the spherical surface of the upper hemisphere of a projection sphere according to an embodiment of the present disclosure.

[0023] FIG7 b is a schematic diagram of setting a target point on the spherical surface of the lower hemisphere of a projection sphere according to an embodiment of the present disclosure.

[0024] FIG8 is a schematic diagram of setting a projection viewing angle according to an embodiment of the present disclosure.

[0025] FIG9 is a schematic diagram of the structure of a classification model according to an embodiment of the present disclosure.

[0026] FIG10 is a schematic diagram of the logical structure of a device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0028] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms "a", "the" and "the" used in this disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items. In addition, the term "at least one" herein means any combination of at least two of any one or more of a plurality of.

[0029] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining."

[0030] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present disclosure and to make the above-mentioned purposes, features and advantages of the embodiments of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure are further described in detail below with reference to the accompanying drawings.

[0031] In some scenarios, it is possible to automatically classify objects based on their 3D models to determine their category for subsequent applications, such as determining whether they are teeth or faces. Currently, as shown in Figure 1, when classifying objects based on their 3D models, one approach involves projecting the 3D model according to multiple pre-set projection angles (e.g., projection angle 1 - projection angle n in the figure) to obtain multiple 2D images. Features are then extracted from these 2D images one by one, and the category of the object is determined based on the extracted features. Currently, when using this approach to classify objects, the 3D model used is typically a 3D mesh model, meaning that the 3D model only contains shape and topological information, but no texture information. Furthermore, the features extracted from the projected 2D images are only the shape features or topological features of the object to be classified. These features can only be used to distinguish objects of different shape categories, such as teeth and faces, but not objects of the same category but different materials. For example, it is impossible to distinguish whether a tooth model is a real tooth model or a tooth model made of paraffin. However, there are some application scenarios that require classifying objects of the same category but different materials to determine the material category of the objects. Current methods cannot meet this requirement.

[0032] For example, consider 3D tooth models. These typically include real tooth models and those made from plaster or metal. These models are similar in shape but made from different materials. Models made from different materials may have different uses. For example, real tooth models can be used to diagnose the need for braces or orthodontics, while plaster or metal models can be used for 3D printing. Therefore, a solution is needed to automatically distinguish the material types of these 3D tooth models for easier use.

[0033] In order to classify the material of the object to be classified, the applicant has thought of a method that can simultaneously combine the texture information of the object to be classified, that is, the texture information of the object to be classified can be superimposed on the three-dimensional model, and the three-dimensional model carrying the texture information can be used for projection to obtain a colored two-dimensional image. The object to be classified is classified based on the colored two-dimensional image, so that the material category of the object can be distinguished.

[0034] However, after combining the texture information, the image obtained by projecting the three-dimensional model carrying the texture information is a color image, namely an RGB image. Compared with the grayscale image obtained by previous projection, the RGB image contains more feature information. If, according to the previous scheme, multiple frames of images projected according to the preset multiple projection angles are extracted, and the category of the object to be classified is determined based on the extracted features, a problem is likely to arise. That is, if the number of projection angles is set too many, since features need to be extracted from more frames of RGB images, a problem of large computational complexity is likely to occur. If the number of projection angles is set too few, the amount of feature information obtained is likely to be small, and accurate classification cannot be performed.

[0035] Based on this, the present disclosure provides an object classification method, as shown in FIG2. First, in order to classify the material of the object to be classified, the present disclosure proposes that the texture information of the object to be classified can be combined when determining the category to which the object to be classified belongs. Specifically, a three-dimensional model carrying texture information can be projected to obtain an RGB image. The object to be classified is classified based on the RGB image, thereby distinguishing the material category of the object to be classified. Furthermore, in order to reduce the computational complexity while ensuring the accuracy of the classification results, after projecting the three-dimensional model carrying texture information according to multiple preset projection angles (such as projection angles 1-n in the figure) to obtain multiple frames of RGB images, RGB images containing more effective information (e.g., RGB image 1 and RGB image 3) can be first selected from these images for subsequent feature extraction and material category determination. Effective information refers to information that can be used to identify the material of the object to be classified. In this way, the material category of the object to be classified can be distinguished, meeting the usage requirements of some specific scenarios. At the same time, when distinguishing the material category, a relatively accurate classification result can be obtained under the premise of low computational complexity.

[0036] The object classification method provided by the embodiments of the present disclosure can be executed by various electronic devices equipped with the above-mentioned classification function software or services. For example, the electronic device can be a personal terminal such as a mobile phone, tablet, or computer, or a cloud server or server cluster. For example, in some scenarios, the method can be executed by a cloud server, and the user can upload a three-dimensional model of the object to be classified through the client, which is then sent to the cloud so that the cloud can determine the material category of the object to be classified and perform subsequent processing on the three-dimensional model based on the category.

[0037] In the embodiments of the present disclosure, the objects to be classified can be any object requiring material classification, and the 3D models of the objects to be classified can also be any 3D models carrying texture information. The 3D models can be in any form, for example, 3D mesh models, point cloud models, etc. The 3D models can be reconstructed by scanning the objects to be classified using a 3D scanning device, or they can be generated by other means, without limitation in the embodiments of the present disclosure.

[0038] For example, in some scenarios, users can use handheld 3D scanning devices to scan objects to be classified. Scanning software that communicates with the 3D scanning device can obtain the scanned data and reconstruct a 3D model of the object to be classified in real time based on the acquired data. The reconstructed 3D model is then uploaded to a cloud server, which determines the material of the 3D model and performs subsequent processing.

[0039] As shown in FIG3 , the object classification method according to an embodiment of the present disclosure may include the following steps:

[0040] S302, obtaining a three-dimensional model of the object to be classified, wherein the three-dimensional model carries texture information;

[0041] In step S302, a three-dimensional model of the object to be classified may be obtained, the three-dimensional model carrying texture information. The three-dimensional model may be a three-dimensional model in various forms, such as a three-dimensional mesh model, a three-dimensional point cloud model, and the like. Texture information may also be represented in various forms. For example, if the three-dimensional model is a three-dimensional mesh model, the texture information may be a texture map corresponding to the three-dimensional mesh model. If the three-dimensional model is a three-dimensional point cloud model, the texture information may be the RGB information of each three-dimensional point. The embodiments of the present disclosure do not limit the forms of representation of the three-dimensional model and texture information.

[0042] S304, projecting the three-dimensional model according to a plurality of preset projection viewing angles to obtain a plurality of RGB image frames;

[0043] In step S304, after obtaining the 3D model of the object to be classified, the 3D model can be projected according to multiple preset projection viewpoints to obtain multiple RGB images, where each RGB image frame corresponds to a projection viewpoint. In some scenarios, the preset multiple projection viewpoints can be fixed projection viewpoints set in advance by the user, that is, the multiple projection viewpoints are fixed for any 3D model. For example, to ensure that the multiple projection viewpoints cover as many different angles as possible, the multiple projection viewpoints can be the viewpoints corresponding to rays projected from the center of the 3D model into various directions in space at certain angular intervals. For example, as shown in Figure 4a, if the 3D model is a cube, the projection viewpoints can be viewpoints pointing from various angles in space toward the center of the cube. In some scenarios, to ensure that the multiple projection viewpoints are more consistent with the actual situation of the 3D model, the preset multiple projection viewpoints can also be determined based on the shape and texture information distribution of each 3D model. For example, the preset projection viewpoints can be views pointing toward surfaces of the 3D model with texture information or toward surfaces of the 3D model with relatively rich shape information. As shown in FIG4 b , if the three-dimensional model is a cube, and only the upper and lower surfaces of the cube have texture information, the projection perspective can be a perspective from the upper and lower surfaces to the center of the cube.

[0044] Among them, the determination of the preset multiple perspectives can be set based on actual conditions, and can be fixed or temporarily set based on the characteristics of the three-dimensional model. The embodiments of the present disclosure do not limit this.

[0045] S306, selecting a target RGB image from the multiple RGB image frames, wherein the amount of valid information in the target RGB image meets a preset condition, and the valid information is information that can be used to identify the material of the object to be classified;

[0046] In order to cover as much feature information as possible from different angles of the three-dimensional model, when setting the above-mentioned multiple projection perspectives, a uniform sampling method can generally be used to select a certain number of multiple perspectives. However, in the process of extracting features from the projected RGB image, if feature extraction is performed on the RGB images at all projection perspectives and then the category of the object to be classified is predicted based on the extracted features, it is easy to cause problems such as excessive computational complexity and low efficiency. Therefore, in step S306, after the three-dimensional model is projected based on the preset multiple projection perspectives to obtain multiple frames of RGB images, one or more target RGB images can be selected from the multiple frames based on the amount of effective information contained in each RGB image frame, wherein the amount of effective information in the target RGB image meets a preset condition, such as the amount of effective information in the target RGB image is greater than a certain threshold, or the amount of effective information in the target RGB image is the largest, or the target RGB image is the first N frames with the largest amount of effective information among the multiple RGB images, etc. Among them, the valid information is information that can be used to assist in identifying the material of the object to be classified. For example, it can be texture information that can be used to assist in identifying the material of the object to be classified, or it can be shape information, topological structure information, etc. that can be used to assist in identifying the material of the object to be classified. The embodiments of the present disclosure do not limit this.

[0047] S308 : Extract features from the target RGB image, and determine the material category of the object to be classified based on the extracted features.

[0048] In step S308, after the target RGB image is obtained through screening, feature extraction can be performed on the target RGB image, and then the material category of the object to be classified can be determined based on the extracted features. In some scenarios, feature extraction of the target RGB image can be achieved using some feature extraction networks. For example, feature extraction networks such as AlexNet, ResNet, and VGG can be used to extract features of the target RGB image to obtain the features of the target RGB image. Determining the material category of the object to be classified based on the extracted features can also be achieved through a pre-trained neural network. For example, the extracted features can be input into the neural network, and the neural network predicts the probability that the object to be classified belongs to a preset material category. In some scenarios, the target RGB image can also be directly input into a pre-trained neural network, and the neural network can extract features from the target RGB image and determine the material category of the object to be classified based on the extracted features.

[0049] In some embodiments, the effective information content of the target RGB image meets a preset condition, which may be that the effective information content of the target RGB image is greater than a preset information content threshold. Alternatively, the effective information content of the multiple RGB frames may be sorted in descending order, and then the first N RGB frames are selected as the target RGB image, where N is a positive integer. For example, assuming the preset number of projection viewing angles is 20, the target RGB image may be the first five RGB frames with the largest effective information content.

[0050] In some embodiments, the 3D model may be a 3D mesh model of the object to be classified, and the texture information may be a texture map corresponding to the 3D mesh model. When the 3D model is projected according to multiple preset projection angles to obtain multiple RGB images, the RGB values ​​of each mesh vertex in the 3D mesh model can be determined based on the texture map. For example, taking a 3D mesh model as an example, the 3D mesh model is typically represented by the position coordinates of multiple mesh vertices and the connection relationship between these mesh vertices. Texture coordinates can be pre-designed for the 3D mesh model, as shown in FIG5 . The texture coordinates represent the mapping relationship between each mesh vertex in the 3D mesh model and the pixels in the texture map. That is, each mesh vertex in the 3D mesh model corresponds to a texture coordinate, and each texture coordinate corresponds to a pixel in the texture map. The texture coordinates are used to associate the mesh vertices of the 3D model with the pixels in the texture map. Therefore, the texture coordinates corresponding to the mesh vertex can be first determined, and then the pixel corresponding to the mesh vertex can be found from the texture map based on the texture coordinates, and the texture information (i.e., RGB value) of the pixel can be assigned to the mesh vertex.

[0051] After determining the texture information of each mesh vertex, the three-dimensional mesh model can be projected according to a preset projection perspective, and the RGB value of each pixel in the projected two-dimensional image can be determined based on the RGB value of each mesh vertex to obtain multiple frames of RGB images.

[0052] In some embodiments, the preset multiple projection perspectives can be perspectives that can observe surfaces carrying valid texture information in the three-dimensional model, wherein the valid texture information is texture information that can be used to identify the material category of the object to be classified. The three-dimensional model is projected based on these perspectives, and the resulting RGB image contains more valid information.

[0053] In some embodiments, in order to set the projection perspective to cover the useful feature information in the three-dimensional model as comprehensively as possible, the projection sphere of the three-dimensional model can be first determined. As shown in Figure 6, the projection sphere of the three-dimensional model refers to a sphere whose center coincides with the center of the three-dimensional model and can enclose the three-dimensional model. Then, the spherical surface of the sphere can be sampled to obtain multiple target points, and the perspective of the target points pointing to the center of the three-dimensional model is used as the projection perspective. For example, if multiple projection perspectives are desired to cover all angles of the three-dimensional model, the entire spherical surface can be evenly sampled to obtain multiple target points, and the perspective of these target points pointing to the center of the three-dimensional model is used as the projection perspective, thereby covering all angles of observing the three-dimensional model.

[0054] Considering that the RGB image projected according to the projection perspective must contain valid texture information of the object to be classified in order to be used to identify the material of the object to be classified, and that not every surface of a 3D model carries valid texture information, when setting multiple projection perspectives, a perspective should be selected that allows observation of surfaces with valid texture information in the 3D model. Therefore, in some embodiments, when setting the multiple projection perspectives, a target area can be first determined from the projection sphere, where the target area is the spherical area directly opposite the surface of the 3D model with valid texture information. Then, sampling can be performed at intervals within this target area to obtain multiple target points, and the perspective from which each target point points toward the center of the 3D model is used as a projection perspective.

[0055] By setting the projection viewing angle in this way, more effective projection viewing angles that can accurately obtain effective information can be determined, which can not only reduce the number of projection viewing angles but also improve the accuracy of the classification results.

[0056] Among them, it is taken into account that the size of the projection area obtained after the object is projected is related to the projection distance. For example, for the same object, the greater the projection distance, the smaller the projection area obtained at the same projection angle, and vice versa. Since the amount of effective information in the RGB images obtained by projection at different projection angles is usually related to the area size of the projection area obtained after the three-dimensional model is projected, in order to facilitate the comparison of the amount of effective information in the RGB images obtained by projection at different projection angles, a projection sphere can be used to determine the projection angle and project the three-dimensional model. Since the distance from each point on the spherical surface of the projection sphere to the center of the sphere is equal, placing the three-dimensional object at the center of the sphere can ensure that the projection distance is fixed, so the projected sizes are relative and comparable. In some embodiments, if all outer surfaces of the three-dimensional model carry effective texture information, the multiple target points are evenly distributed on the entire spherical surface of the projection sphere. If all outer surfaces of the 3D model carry valid texture information, it means that valid texture information can be seen from all angles when looking at the 3D model. Therefore, in order to set the projection viewing angle to fully cover the feature information of the 3D model at all angles, the spherical surface of the entire projection sphere can be evenly sampled to obtain multiple target points, and the viewing angles pointing these target points to the center of the 3D model are used as the preset multiple projection viewing angles.

[0057] In some embodiments, as shown in FIG7a , if only the outer surface of the upper half of the 3D model carries valid texture information, this means that the valid texture information can only be seen from the upper hemisphere of the projection sphere. Therefore, when setting the projection viewing angle, the projection viewing angle can be a perspective from the upper hemisphere toward the center of the 3D model, meaning that the multiple target points are evenly distributed on the spherical surface of the upper hemisphere of the projection sphere.

[0058] In some embodiments, as shown in FIG7b , if only the outer surface of the lower half of the 3D model carries valid texture information, this means that the valid texture information can only be seen from the lower hemisphere of the projection sphere. Therefore, when setting the projection viewing angle, the projection viewing angle can be a perspective from the lower hemisphere toward the center of the 3D model, meaning that the multiple target points are evenly distributed on the spherical surface of the lower hemisphere of the projection sphere.

[0059] For example, taking the use of a 3D scanning device to scan a patient's teeth, a 3D scanning device can usually only scan information on the upper and side surfaces of the teeth, but cannot scan information on the bottom surface of the teeth. Therefore, for a tooth model reconstructed based on data scanned by a 3D scanning device, when setting the projection perspective, spherical intervals of the upper hemisphere of the projection sphere of the tooth model can be sampled to obtain multiple target points, and then the perspective pointing these target points to the center of the tooth model is used as the projection perspective.

[0060] Similarly, if only the left half or the right half of the 3D model carries valid texture information, the multiple target points are evenly distributed on the spherical surface of the left hemisphere or the right hemisphere of the projection sphere.

[0061] In some embodiments, when setting the projection perspective, target points can be evenly spaced on a horizontal and / or vertical great circle of the three-dimensional model projection sphere. The great circle of the projection sphere is a circle whose center passes through the center of the sphere and is directly equal to the diameter of the projection sphere. Samples can be taken at intervals on the horizontal great circle of the projection sphere to obtain multiple target points, and the perspective from which the target points point toward the center of the sphere is used as the projection perspective. Alternatively, sample points can be taken at intervals on the vertical great circle of the projection sphere to obtain multiple target points, and the perspective from which the target points point toward the center of the sphere is used as the projection perspective. For example, as shown in FIG8 , sample points can be taken at intervals on the horizontal great circle to obtain target points, and the perspective from which the target points point toward the center of the sphere is used as the projection perspective, with a 60° interval between adjacent projection perspectives. Similarly, sample points can be taken at intervals on the vertical great circle to obtain target points, and the perspective from which the target points point toward the center of the sphere is used as the projection perspective, with a 30° interval between adjacent projection perspectives. Of course, the specific intervals can be set based on actual needs and are not limited by the present disclosed embodiments.

[0062] In some embodiments, the amount of effective information in each frame of the target RGB image can be characterized by the perspective entropy of the projection perspective corresponding to the target RGB image, wherein the perspective entropy of each projection perspective can be used to characterize the amount of effective information in the RGB image projected using the projection perspective, for example, the greater the perspective entropy, the more effective information. When the effective texture information on the surface of the three-dimensional model is uniformly distributed, the perspective entropy of each projection perspective is positively correlated with the area of ​​the region in the three-dimensional model projected into the target RGB image at the projection perspective. For example, taking the three-dimensional model as a three-dimensional mesh model, assuming that the amount of effective texture information contained in each triangle is consistent, the greater the number of triangles in the three-dimensional model projected into the RGB image, or the greater the number of mesh vertices, the greater the perspective entropy.

[0063] Of course, in some scenarios, if the effective texture information contained in each triangle in the three-dimensional model is inconsistent, then when counting the perspective entropy of the projection perspective, a weight can be set for the triangle based on the amount of information including the effective texture information of each triangle, wherein the more effective texture information the triangle contains, the greater the weight. When counting the perspective entropy of the projection perspective, the perspective entropy can be comprehensively determined based on the number and weight of the triangles projected into the RGB image under the projection perspective. For example, assuming that the number of triangles with a weight of 1 is 100, the number of triangles with a weight of 0.8 is 200, and the number of triangles with a weight of 0.1 is 100, the perspective entropy can be calculated as follows: 100*1+0.8*200+0.1*100.

[0064] In some embodiments, the amount of effective information in each frame of the target RGB image can be characterized by the perspective entropy of the projection perspective corresponding to the target RGB image. The perspective entropy of each projection perspective is represented by the ratio of the effective pixel area in the target RGB image corresponding to the projection perspective to the total area of ​​the target RGB image. The total area of ​​the target RGB image is the area of ​​a preset fixed-size projection rectangle. The projection rectangle can be determined based on the projection circle obtained by projecting the projection sphere of the three-dimensional model at the projection perspective corresponding to the target RGB image. For example, the center of the projection rectangle can coincide with the center of the projection circle, and a rectangle with a length and width of H*W is cut from the projection circle as the projection rectangle (i.e., an H*W matrix of length and width). The effective pixel area is the area of ​​the projection area corresponding to the three-dimensional model. In related art, when characterizing perspective entropy, it is usually represented by the ratio of the area of ​​the projected area in the three-dimensional model (e.g., the area of ​​the projected triangle) to the total area of ​​all triangles in the three-dimensional model. This characterization method requires counting the area of ​​the projected triangle and the total area of ​​all triangles in the three-dimensional model, which is computationally intensive and cumbersome. Therefore, the area of ​​the projected triangular facet can be approximated as the pixel area (i.e., the effective pixel area) of the region constructed by the pixel points with RGB values ​​in the RGB image, and the total area of ​​all triangular facets in the three-dimensional model can be approximated as the area of ​​the preset projection rectangle. Since the area of ​​the projection rectangle is fixed, the amount of calculation can be reduced when calculating the perspective entropy of each projection perspective.

[0065] If the perspective entropy of each projection perspective is represented by the ratio of the effective pixel area in the target RGB image corresponding to that projection perspective to the total area of ​​the target RGB image, this calculation method assumes that each pixel in the RGB image contributes uniformly to the calculation of the perspective entropy, i.e., each pixel has a uniform weight. However, considering that when projecting a 3D model, the 3D model is located at the center of the projection sphere. Therefore, the central area of ​​the 3D model observed at the current projection perspective is projected onto the central area of ​​the RGB image. Since the central area of ​​the 3D model observed at different projection perspectives is different, the center of the RGB image obtained at different projection perspectives corresponds to different areas of the 3D model. Clearly, the central area of ​​the RGB image reflects information about the projection perspective and best reflects the characteristics of the 3D model observed at the current projection perspective, i.e., it can reflect the difference between the current projection perspective and other projection perspectives. Therefore, in some embodiments, when calculating the perspective entropy, in order to better reflect the characteristics of the current projection perspective, different weights can be assigned to each pixel within the projection area corresponding to the 3D model in the RGB image. Considering that the pixels located in the center area of ​​the RGB image can better reflect the characteristics of the current projection perspective, the weight of the pixels in this area can be set larger, while the pixels far away from the center area of ​​the RGB image are usually common features of the current projection perspective and other projection perspectives, so their weight can be set smaller. With this setting, by calculating the perspective entropy for different projection perspectives, better features can be selected, which is helpful for subsequent model training and tuning. In addition, the fusion strategy of the global feature contribution weight can calculate the contribution of the features extracted from different perspective images from a global perspective to generate contribution weights, filter and reduce invalid features or redundant features from multiple perspective images, and improve the recognition accuracy of the model.

[0066] In some embodiments, based on the above embodiments, a rectangle with a length and width of H*W can be cut out from the projection circle as the projection rectangle (i.e., length multiplied by width | H*W matrix), and then the H*W matrix is ​​multiplied by the weight coefficient of the Gaussian distribution, and the effective area pixels are multiplied by the corresponding Gaussian weight matrix to obtain the numerator of the above ratio. Since the total projection area of ​​each perspective is always the same, the denominator is consistent, so the numerator of the above ratio can be regarded as the perspective entropy of the current projection perspective.

[0067] The viewing entropy of each projection view is represented by the ratio of the weighted effective pixel area in the target RGB image corresponding to the projection view to the total area of ​​the target RGB image. The weighted effective pixel area can be obtained by weighting the pixel areas of each pixel within the projection area corresponding to the 3D model using the weights of each pixel, and then summing the weighted pixel areas. The total area of ​​the target RGB image is the area of ​​a preset fixed-size projection rectangle, which can be determined based on the projection circle obtained by projecting the projection sphere of the 3D model at the projection view corresponding to the target RGB image.

[0068] In some embodiments, in order to obtain feature information from different angles of the three-dimensional model as much as possible, the target RGB image can include multiple frames. In the related art, when extracting features from the multiple frames of target RGB images and determining the material category of the object to be classified based on the extracted features, it is usually for each frame of the target RGB image, and feature extraction can be performed on the frame of the target RGB image to obtain the features of each frame of the target RGB image. Since the features of each frame of the target RGB image usually include sub-features of multiple dimensions, in the related art, a single frame of the target RGB image is generally used as a dimension to determine the weight of each sub-feature in the features of the frame of the target RGB image, and then the corresponding sub-features in the features of the frame of the target RGB image are weighted based on the weight of each sub-feature. The weighted features of the multiple frames of the target RGB images are then spliced ​​to obtain spliced ​​features, and the category of the object to be classified is determined based on the spliced ​​features. Feature extraction of the target RGB image can be achieved through some feature extraction networks. For example, feature extraction networks such as GoogLeNet, ResNet, and MobileNet can be used to extract features from the target RGB image to obtain the features of each target RGB image.

[0069] For example, suppose that feature extraction is performed on target RGB images A, B, and C, respectively, and the feature vectors obtained are as follows: (a1, a2, a3), (b1, b2, b3), and (c1, c2, c3). In the related art, the weights of the sub-features of the three dimensions a1, a2, and a3 are first determined based on the feature vectors (a1, a2, a3), namely, r11, r12, and r13. Similarly, the weights of the sub-features of the three dimensions b1, b2, and b3 are determined based on the feature vectors (b1, b2, b3), namely, r21, r22, and r23. The weights of the sub-features of the three dimensions c1, c2, and c3 are determined based on the feature vectors (b1, b2, b3), namely, r31, r32, and r33. That is, for each frame of the target RGB image, the weights of the sub-features of each dimension are determined independently of each other, and the weights of the sub-features of each dimension are not determined from a global perspective. Then, the weighted features of multiple frames of target RGB images can be spliced ​​to obtain the feature vector of the spliced ​​features, namely: (r11a1, r12a2, r13a3, r21b1, r22b2, r23b3, r31c1, r32c2, r33c3).

[0070] The features at each viewing angle reflect the features of the same object. If the contribution weights of the sub-features of each dimension are calculated separately for the feature vectors at each viewing angle, the calculation of the contribution weights is relative. This method is only based on a single-frame target RGB image when determining the weights of the sub-features of each dimension, which lacks globality. It is very likely that the sub-features of a certain dimension of the target RGB image at a certain projection viewing angle have a low weight in the global features of the overall projection viewing angle, but the weight calculated in the local features at a single projection viewing angle is very high, which in turn affects the subsequent classification.

[0071] Based on this, in some embodiments, after feature extraction is performed on multiple frames of target RGB images respectively to obtain the features of each frame of RGB image, the features of the multiple frames of target RGB images can be spliced ​​to obtain spliced ​​features, and then the weight corresponding to the sub-feature of each dimension in the features of each frame of target RGB image can be determined based on the spliced ​​features, that is, the weight of the sub-feature of each dimension in each frame of target RGB image in all target RGB images can be determined from a global perspective, and then the sub-feature in the spliced ​​features can be weighted based on the weight corresponding to the sub-feature of each dimension to obtain the weighted spliced ​​features, and the material category to which the object to be classified belongs can be determined based on the weighted spliced ​​features.

[0072] For example, suppose that for the target RGB images A, B, and C, feature extraction is performed on them respectively, and the feature vectors obtained are as follows: (a1, a2, a3), (b1, b2, b3), and (c1, c2, c3). These three features can be spliced ​​first to obtain the feature vector of the spliced ​​feature: (a1, a2, a3, b1, b2, b3, c1, c2, c3). Then, based on the feature vector (a1, a2, a3, b1, b2, b3, c1, c2, c3), the weights of the sub-features of each dimension of a1, a2, a3, b1, b2, b3, c1, c2, and c3 can be determined, namely, r1, r2, r3, r4, r5, r6, r7, r8, and r9. That is, the weight of the sub-feature of each dimension of each frame image is determined based on all target RGB images, that is, from a global perspective. The splicing features can then be weighted based on the determined weights to obtain a feature vector of the weighted splicing features: (r1a1, r2a2, r3a3, r4b1, r5b2, r6b3, r7c1, r8c2, r9c3). In this way, the weight of each dimension of the feature in each frame of the RGB image can be determined in the overall projection perspective. That is, the weight of each sub-feature is determined from a global perspective, making the determined weight more accurate.

[0073] In some embodiments, the material category of the object to be classified can be determined directly based on the weighted splicing features. For example, the splicing features are input into a pre-trained neural network, and the neural network predicts that the object to be classified belongs to various preset material categories. In some embodiments, in order to prevent the gradient disappearance problem so that the gradient can be better transmitted back, when determining the material category of the object to be classified based on the weighted splicing features, the weighted splicing features can be fused with the original splicing features to obtain fused features, and then the material category of the object to be classified is determined based on the fused features. That is, by fusing the original features with the weighted splicing features, some information of the original splicing features can be retained, avoiding the error introduced in the weight determination process from having too much influence on the final determined features, so that the classification results obtained based on the fused features are more accurate.

[0074] In some embodiments, the method is implemented using a pre-trained classification model, the model structure of which is shown in Figure 9 and includes a multi-view projection subnetwork, a view entropy screening subnetwork, a feature extraction subnetwork, and a fusion subnetwork. The multi-view projection subnetwork is configured to project the three-dimensional model according to multiple preset projection viewpoints to obtain multiple frames of RGB images. The view entropy screening subnetwork is configured to determine the view entropy corresponding to each projection viewpoint and, based on the view entropy, screen the target RGB image from the multiple frames of RGB images. The view entropy of each projection viewpoint is used to characterize the amount of effective information in the RGB image obtained after projecting the three-dimensional model using that projection viewpoint. The feature extraction network is configured to extract features from the target RGB image to obtain features for each frame of the target RGB image. The fusion subnetwork is configured to determine the material category of the object to be classified based on the features of each frame of the target RGB image. For example, the fusion subnetwork can splice the features of the target RGB images of each frame to obtain a spliced ​​feature, and then determine the weights of the sub-features of each dimension in the features of the target RGB image based on the spliced ​​feature, and use the weight to perform weighted processing on the corresponding sub-features in the spliced ​​feature to obtain a weighted spliced ​​feature, and then the spliced ​​feature and the weighted spliced ​​feature can be fused to obtain a fused feature, and the probability of the object to be classified being a preset material category is predicted based on the fused feature to determine the material category of the object to be classified.

[0075] In some embodiments, the method is implemented using a pre-trained classification model. During training of the classification model, a sample 3D model carrying a label can be obtained, wherein the label indicates the material category of the sample 3D model, and the sample 3D model carries texture information. The sample 3D model can then be input into a preset initial model, which projects the sample 3D model according to multiple preset projection angles to obtain multiple frames of sample RGB images. A target sample RGB image is selected from the multiple frames of sample RGB images, and features are extracted from the target sample RGB image. Based on the extracted features, the material category of the sample 3D model is predicted. The model parameters of the initial model can then be continuously adjusted based on the difference between the predicted material category and the material category indicated by the label, thereby training a classification model.

[0076] In some embodiments, when selecting a target sample RGB image from multiple sample RGB images, the multiple sample RGB images can be sorted in descending order of effective information content, and then the first N frames with the most effective information are selected as the target sample RGB image. However, if all selected target sample RGB images are images with high effective information content, that is, images with good projection angles, the model learning process can be overly simplified, with the model overlearning the features of images with high effective information content, which can easily lead to overfitting. Consequently, if the trained classification model encounters images with low effective information during inference, it will be unable to make accurate judgments. To reduce overfitting during training, in some embodiments, when selecting a target sample RGB image from multiple sample RGB images, the target sample RGB image is a portion of the first M1 frames and a portion of the last M2 frames of the multiple RGB images sorted in descending order of effective information content, where M1 and M2 are positive integers and the total number of the multiple sample RGB images is M1 + M2.

[0077] For example, assuming that the multiple sets of projection perspectives are fixed at 19 sets, after projecting the sample 3D model according to these 19 sets of projection perspectives, 19 frames of sample RGB images can be obtained. The 19 frames of sample RGB images can then be sorted in descending order of the amount of effective information, and the first 6 frames of sample RGB images can be selected as the target sample RGB images. However, to prevent overfitting, 2 frames of images can be randomly selected from the remaining 13 frames of sample RGB images to replace any two of the 6 frames. That is, images with less effective information are introduced into the target RGB image. By introducing images with less effective information, more invalid information and noise information can be introduced, which can increase the optimization difficulty of model training, prevent model overfitting, and improve the robustness of the model. This setting can also improve the recognition generalization of this method for 3D models with missing teeth, 3D models with incomplete scans, and other 3D models containing a large amount of low perspective entropy.

[0078] In some embodiments, the three-dimensional model to be classified can be a three-dimensional tooth model uploaded by the user, and the material category is real teeth and non-real teeth. When it is determined that the material category of the three-dimensional model is non-real teeth, a prompt can be given to the user; when it is determined that the material category of the three-dimensional model is real teeth, a disease diagnosis can be performed on the tooth model. For example, in some scenarios, this method can be used in some intelligent diagnostic software to diagnose diseases on tooth models. Since the intelligent diagnostic software requires that the tooth model uploaded by the user is a real tooth model and cannot be a tooth model made of plaster, metal, etc., the intelligent diagnostic software can first classify the material of the three-dimensional tooth model uploaded by the user based on the method described in the above embodiment to determine whether it belongs to a real tooth or a non-real tooth (i.e., a tooth made of plaster, metal, etc.). If it is a real tooth, it will only be aligned for disease diagnosis. If it is a non-real tooth, the user will be prompted that the tooth is a non-real tooth and cannot be diagnosed.

[0079] It is not difficult to understand that the solutions described in the above embodiments can be freely combined to obtain new solutions when there is no conflict. Due to space reasons, they are not listed one by one in the embodiments of this disclosure.

[0080] Accordingly, an embodiment of the present disclosure further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, implements the method mentioned in any of the above embodiments.

[0081] Furthermore, an embodiment of the present disclosure also provides an object classification device, which includes: an acquisition module, a projection module, a selection module and a determination module, the acquisition module is configured to acquire a three-dimensional model of the object to be classified, the three-dimensional model carrying texture information; the projection module is configured to project the three-dimensional model according to a plurality of preset projection perspectives to obtain multiple frames of RGB images; the selection module is configured to select a target RGB image from the multiple frames of RGB images, wherein the amount of effective information in the target RGB image meets preset conditions, and the effective information is information that can be used to identify the material of the object to be classified; the determination module is configured to perform feature extraction on the target RGB image, and determine the material category to which the object to be classified belongs based on the extracted features.

[0082] For the specific definition of the object classification device, please refer to the definition of the object classification method above, and will not be repeated here. Each module in the above-mentioned object classification device can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the electronic device in the form of hardware, or can be stored in the memory of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0083] Furthermore, an embodiment of the present disclosure also provides an electronic device, as shown in Figure 10, which includes a processor 101, a memory 102, and computer instructions stored in the memory 102 for execution by the processor 101, and when the processor 101 executes the computer instructions, it implements any one of the methods in the above embodiments.

[0084] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the method described in any of the aforementioned embodiments when the program is executed by a processor.

[0085] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0086] Through the description of the above implementation methods, it can be seen that those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the embodiments of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments of the present disclosure.

[0087] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0088] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the functions of each module can be implemented in the same one or more software and / or hardware when implementing the embodiment of the present disclosure. It is also possible to select some or all of the modules according to actual needs to achieve the purpose of the embodiment. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0089] The above is only a specific implementation of the embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the embodiment of the present disclosure. These improvements and modifications should also be regarded as the scope of protection of the embodiment of the present disclosure. Industrial Applicability:

[0090] In the object classification method provided by the present disclosure, when determining the category to which an object to be classified belongs, the texture information of the object to be classified can be combined. Specifically, a three-dimensional model carrying the texture information can be projected to obtain an RGB image. The object to be classified is then classified based on the RGB image, thereby distinguishing the material category of the object to be classified and expanding its application scenarios. Furthermore, in order to reduce computational complexity, improve recognition speed, and ensure the accuracy of the classification results, after projecting the three-dimensional model carrying the texture information according to multiple preset projection angles to obtain multiple RGB images, RGB images containing more effective information can be first selected from these images for subsequent feature extraction and material category determination. Effective information refers to information that can be used to identify the material of the object to be classified. In this way, the material category of the object to be classified can be distinguished, meeting the needs of certain specific scenarios. At the same time, when distinguishing material categories, relatively accurate classification results can be achieved with a low computational complexity, thus having strong industrial applicability.

Claims

1. A method for object classification, wherein: The method comprises: Obtaining a three-dimensional model of the object to be classified, wherein the three-dimensional model carries texture information; Projecting the three-dimensional model according to a plurality of preset projection viewing angles to obtain a plurality of RGB image frames; Selecting a target RGB image from the multiple RGB images, wherein the amount of valid information in the target RGB image meets a preset condition, and the valid information is information that can be used to identify the material of the object to be classified; Feature extraction is performed on the target RGB image, and the material category of the object to be classified is determined based on the extracted features.

2. The method according to claim 1, wherein The amount of effective information in the target RGB image meets preset conditions, including: The amount of valid information in the target RGB image is greater than a preset information amount threshold; or The multiple RGB images are sorted in descending order according to the amount of effective information, and the target RGB image is the first N frames of the sorted multiple RGB images, where N is a positive integer.

3. The method according to claim 1, wherein The three-dimensional model is a three-dimensional mesh model of the object to be classified, the texture information is a texture map corresponding to the three-dimensional mesh model, and the three-dimensional model is projected according to a plurality of preset projection angles to obtain a plurality of RGB images, including: Determine the RGB value of each mesh vertex of the three-dimensional mesh model based on the texture map; The three-dimensional mesh model is projected according to a preset projection viewing angle, and the RGB value of each pixel point in the projected two-dimensional image is determined based on the RGB value of each mesh vertex to obtain the multi-frame RGB image.

4. The method according to claim 1, wherein The preset multiple projection viewing angles are viewing angles from which a surface of the three-dimensional model carrying valid texture information can be observed, and the valid texture information can be used to identify the material of the object to be classified; and / or The amount of effective information in each frame of the target RGB image is characterized by the viewing angle entropy of the projection viewing angle corresponding to the target RGB image. If the effective texture information on the surface of the three-dimensional model is evenly distributed, the viewing angle entropy of each projection viewing angle is positively correlated with the area of ​​the region in the three-dimensional model projected into the target RGB image.

5. The method according to claim 4, wherein The preset multiple projection viewing angles are determined in the following manner: Determining a target area from a projection sphere of the three-dimensional model, wherein a center of the projection sphere coincides with a center of the three-dimensional model and the projection sphere surrounds the three-dimensional model, and the target area is a spherical area directly facing a surface with valid texture information in the three-dimensional model; Multiple target points are obtained by sampling at intervals within the target area, and a perspective from each target point to the center of the three-dimensional model is used as a projection perspective.

6. The method according to claim 5, wherein: If all outer surfaces of the three-dimensional model carry valid texture information, the multiple target points are evenly distributed on the entire spherical surface of the projection sphere; If only the outer surface of the upper half of the three-dimensional model carries valid texture information, the multiple target points are evenly distributed on the spherical surface of the upper hemisphere of the projection sphere; if only the outer surface of the lower half of the three-dimensional model carries valid texture information, the multiple target points are evenly distributed on the spherical surface of the lower hemisphere of the projection sphere; and / or The target points are distributed at equal intervals on a large circle in the horizontal direction and / or a large circle in the vertical direction of the projection sphere.

7. The method according to claim 1, wherein The amount of effective information in each frame of the target RGB image is represented by the viewing angle entropy of the projection viewing angle corresponding to the target RGB image, and the viewing angle entropy of each projection viewing angle is represented by the ratio of the effective pixel area in the target RGB image to the total area of ​​the target RGB image; or The viewing angle entropy of each projection viewing angle is represented by the ratio of the weighted effective pixel area in the target RGB image to the total area of ​​the target RGB image; Among them, the effective pixel area is the area of ​​the projection area corresponding to the three-dimensional model, and the weighted effective pixel area is obtained by weighting the pixel area of ​​each pixel in the projection area corresponding to the three-dimensional model using the weight of each pixel, and summing the pixel areas of each pixel after weighted processing. The closer the pixel in the projection area is to the center of the RGB image, the greater the weight; the total area of ​​the target RGB image is the area of ​​a preset fixed-size projection rectangle, and the projection rectangle is determined based on the projection circle obtained by projecting the projection sphere of the three-dimensional model at the projection viewing angle corresponding to the target RGB image.

8. The method according to claim 1, wherein The target RGB image includes multiple frames, and extracting features from the target RGB image and determining the material category of the object to be classified based on the extracted features includes: Performing feature extraction on the multiple frames of target RGB images respectively to obtain features of each frame of RGB image, wherein the features of each frame of target RGB image include sub-features of multiple dimensions; Splicing the features of the multiple target RGB images to obtain spliced ​​features; Determine the weight corresponding to the sub-feature of each dimension in the features of each frame of the target RGB image based on the splicing features; Performing weighted processing on the sub-features in the splicing feature based on the weight corresponding to the sub-features in each dimension to obtain a weighted splicing feature; The material category of the object to be classified is determined based on the weighted splicing features.

9. The method according to claim 8, wherein The weighted splicing features are used to determine the material category of the object to be classified, including: Fusing the weighted splicing feature with the splicing feature to obtain a fused feature; The material category of the object to be classified is determined based on the fusion features.

10. The method according to claim 1, wherein The method is implemented by a pre-trained classification model, which includes a multi-view projection subnetwork, a view entropy screening subnetwork, a feature extraction subnetwork, and a fusion subnetwork; The multi-view projection sub-network is configured to project the three-dimensional model according to a plurality of preset projection viewpoints to obtain multiple frames of RGB images; The viewing entropy screening subnetwork is configured to determine the viewing entropy corresponding to each projection viewing angle, and screen a target RGB image from the multiple frames of RGB images based on the viewing entropy, wherein the viewing entropy of each projection viewing angle is used to characterize the amount of effective information in the RGB image obtained after projecting the three-dimensional model using the projection viewing angle; The feature extraction subnetwork is configured to perform feature extraction on the target RGB image to obtain features of each frame of the target RGB image; The fusion sub-network is configured to determine the material category of the object to be classified based on the respective features of the multiple frames of target RGB images.

11. The method according to claim 1, wherein The method is implemented by a pre-trained classification model, which is trained based on the following method: Acquire a sample three-dimensional model carrying a label, wherein the label is used to indicate a material category of the sample three-dimensional model, and the sample three-dimensional model carries texture information; The sample 3D model is input into a preset initial model, and the initial model projects the sample 3D model according to a plurality of preset projection angles to obtain a plurality of sample RGB images. A target sample RGB image is selected from the plurality of sample RGB images, and features are extracted from the target sample RGB image. The material category of the sample 3D model is predicted based on the extracted features. The plurality of RGB images are sorted in descending order according to the amount of effective information, and the target sample RGB image is a portion of the first M1 frames and a portion of the last M2 frames in the sorted plurality of RGB images, wherein M1 and M2 are positive integers, and the total number of the plurality of sample RGB images is M1+M2. The model parameters of the initial model are adjusted based on the difference between the predicted material category and the material category indicated by the label to train the classification model.

12. The method according to claim 1, wherein The three-dimensional model to be classified is a three-dimensional model of teeth uploaded by a user, and the material categories are real teeth and non-real teeth. The method further includes: Prompting the user when it is determined that the material category of the three-dimensional model is not a real tooth; When it is determined that the material type of the three-dimensional model is a real tooth, a disease diagnosis is performed on the tooth model.

13. An object classification device, wherein: The device comprises: an acquisition module, configured to acquire a three-dimensional model of the object to be classified, wherein the three-dimensional model carries texture information; A projection module is configured to project the three-dimensional model according to a plurality of preset projection viewing angles to obtain a plurality of RGB images; a selection module configured to select a target RGB image from the multiple RGB image frames, wherein the amount of valid information in the target RGB image meets a preset condition, and the valid information is information that can be used to identify the material of the object to be classified; The determination module is configured to perform feature extraction on the target RGB image and determine the material category of the object to be classified based on the extracted features.

14. A computer program product, wherein The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

15. An electronic device, wherein: The electronic device includes a processor, a memory, and computer instructions stored in the memory and executable by the processor. When the processor executes the computer instructions, the method according to any one of claims 1 to 12 is implemented.

16. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images