Three-dimensional model feature point detection method and device

Abstract

The invention provides a three-dimensional model feature point detection method and device. The method comprises the steps of obtaining first information of a to-be-detected object, determining color information corresponding to a three-dimensional model of the to-be-detected object, rendering the three-dimensional model of the to-be-detected object into a two-dimensional image according to the color information corresponding to the three-dimensional model of the to-be-detected object, performing feature point detection on the two-dimensional image to obtain a plurality of two-dimensional feature points, and converting the plurality of two-dimensional feature points into corresponding three-dimensional feature points according to a conversion relationship between the two-dimensional feature points and the three-dimensional feature points. Therefore, according to the method, a mature two-dimensional image feature point detection algorithm is used for performing feature point detection on the three-dimensional model, so that the three-dimensional feature point detection robustness of the three-dimensional model of the to-be-detected object is relatively high, and meanwhile, the method does not have a process of manually calibrating a three-dimensional model training sample set; the time cost and the labor cost consumed by manually calibrating the feature points can be effectively saved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method and device for detecting feature points of a three-dimensional model. Background Art

[0002] The detection of feature points of a three-dimensional (3D) model of an object (such as a human face, an animal face, or a vehicle) is an important task in the field of computer vision. Its purpose is to automatically detect some three-dimensional feature points from the three-dimensional model that are used to characterize the structural properties of the object. For example, by performing feature point detection on the three-dimensional face model, some representative three-dimensional feature points such as eyes, nose, mouth, eyebrows, contours, etc. are obtained. These representative three-dimensional feature points can be used as the basic steps for key tasks such as 3D reconstruction of the face or face retopology. Based on this, the detection of feature points of three-dimensional models has a wide range of applications in computer graphics, virtual reality, face recognition, facial expression analysis and other fields.

[0003] At present, a 3D model feature point detection scheme is: for a 3D model of an object, the feature point detection device first uses the local information or global information of the 3D model of the object to detect feature points, and obtains some 3D feature points of the 3D model of the object. Afterwards, the feature point detection device uses non-maximum suppression to screen these 3D feature points, and obtains 3D feature points used to characterize the structural properties of the object. However, the 3D feature points detected by this scheme are generally not stable. For example, the 3D feature points located at the boundary of the object or in the uneven surface area may change each time they are detected, resulting in low accuracy and low robustness of the 3D feature point detection of this scheme. Summary of the invention

[0004] The embodiments of the present application provide a method and device for detecting feature points of a three-dimensional model, so as to improve the robustness of feature point detection of a three-dimensional model of an object to be detected.

[0005] In a first aspect, an embodiment of the present application provides a method for detecting feature points of a three-dimensional model, which can be performed by a three-dimensional model feature point detection device or a component (such as a chip, a chip system or a circuit, etc.) that can support the three-dimensional model feature point detection device to implement the functions required by the method. Exemplarily, the three-dimensional model feature point detection device can be a service device (such as a computing device) having the functions required to implement the method, or it can also be any other device (such as a vehicle-mounted device or a robot device, etc.) having the functions required to implement the method, or it can also be a functional element (such as a plug-in, component, chip or circuit, etc.) in a service device having the functions required to implement the method, or it can also be a functional element set in any other device, which has the function of implementing the three-dimensional model feature point detection method. Optionally, taking the execution of a three-dimensional model feature point detection method by a three-dimensional model feature point detection device as an example, in this method, after obtaining the first information of the object to be detected, the three-dimensional model feature point detection device can determine the color information corresponding to the three-dimensional model of the object to be detected, wherein the first information includes the three-dimensional model of the object to be detected. Afterwards, the three-dimensional model feature point detection device can render the three-dimensional model of the object to be detected into a two-dimensional image according to the color information corresponding to the three-dimensional model of the object to be detected, and can perform feature point detection on the two-dimensional image to obtain multiple two-dimensional feature points. Then, the three-dimensional model feature point detection device can convert the multiple two-dimensional feature points according to the conversion relationship between the two-dimensional feature points and the three-dimensional feature points to obtain multiple three-dimensional feature points, wherein the multiple three-dimensional feature points are used as detection feature points of the three-dimensional model of the object to be detected.

[0006] In this method, the three-dimensional model feature point detection device can effectively render the three-dimensional model of the object to be detected into a two-dimensional image according to the color information corresponding to the three-dimensional model of the object to be detected, and then the three-dimensional feature points in the three-dimensional model of the object to be detected can be timely and accurately obtained by converting the two-dimensional feature points detected in the two-dimensional image into corresponding three-dimensional feature points. In this way, the method can accurately detect semantic feature points by taking advantage of the existing two-dimensional image feature point detection algorithm and using the existing relatively mature two-dimensional image feature point detection algorithm (or can be called the existing two-dimensional image feature point detection model) to perform feature point detection on the two-dimensional image. At the same time, the existing two-dimensional image feature point detection algorithm has strong robustness to changes in illumination, posture or expression in the two-dimensional image, and can effectively resist the change of viewing angle and illumination, so it can effectively ensure the accuracy of the three-dimensional model feature point detection, thereby effectively ensuring the detection stability of the three-dimensional feature points of the three-dimensional model of the object to be detected, so that the robustness of the feature point detection of the three-dimensional model of the object to be detected is effectively improved. In addition, this method does not require the process of manually calibrating the key feature points (i.e., certain three-dimensional feature points) of the three-dimensional model training sample set, which can effectively save the time and manpower costs consumed by manually calibrating feature points, and thus can effectively improve the feature point detection efficiency of the three-dimensional model of the object to be detected.

[0007] In a possible design, the three-dimensional model feature point detection device can determine the color information corresponding to the three-dimensional model of the object to be detected through the following possible implementation methods.

[0008] Method 1: When the first information includes a texture map corresponding to the three-dimensional model of the object to be detected, the three-dimensional model feature point detection device can determine the color information of multiple points in the three-dimensional model of the object to be detected based on the texture map corresponding to the three-dimensional model of the object to be detected. Thereafter, the three-dimensional model feature point detection device can use the color information of the multiple points as the color information corresponding to the three-dimensional model of the object to be detected.

[0009] Method 2: When the first information does not include the texture map corresponding to the three-dimensional model of the object to be detected, the three-dimensional model feature point detection device can analyze the three-dimensional model of the object to be detected to obtain attribute information of the three-dimensional model of the object to be detected. After that, the three-dimensional model feature point detection device can determine the color information corresponding to the three-dimensional model of the object to be detected based on the attribute information.

[0010] In the above design, the three-dimensional model feature point detection device can flexibly select (or adaptively select) a matching color information determination method to determine the color information corresponding to the three-dimensional model of the object to be detected by identifying whether the first information contains the texture map corresponding to the three-dimensional model of the object to be detected. In other words, for the case where the first information includes the texture map corresponding to the three-dimensional model of the object to be detected and the case where the first information does not include the texture map corresponding to the three-dimensional model of the object to be detected, the three-dimensional model feature point detection device can respectively select the corresponding color information determination method to determine the color information corresponding to the three-dimensional model of the object to be detected, so that it can be more targeted to different color information calculation scenarios, and the calculation of color information can be made more consistent with the actual situation of the first information, so that the method can be applied to more scenarios and has stronger versatility.

[0011] In a possible design, there are several possible implementations of the three-dimensional model feature point detection device determining color information corresponding to the three-dimensional model of the object to be detected based on the attribute information of the three-dimensional model of the object to be detected.

[0012] Method 1: When the attribute information includes color information of multiple points in the three-dimensional model of the object to be detected, the three-dimensional model feature point detection device can use the color information of the multiple points as the color information corresponding to the three-dimensional model of the object to be detected.

[0013] Method 2: When the attribute information does not include color information of multiple points in the three-dimensional model of the object to be detected, the three-dimensional model feature point detection device can establish a point registration relationship between the three-dimensional model of the object to be detected and the three-dimensional model of the standard object. Thereafter, the three-dimensional model feature point detection device can determine the color information corresponding to the three-dimensional model of the object to be detected based on the point registration relationship and the texture map corresponding to the three-dimensional model of the standard object, wherein the three-dimensional model of the standard object is used as a reference template for the three-dimensional model of the object to be detected, and the texture map corresponding to the three-dimensional model of the standard object is used to characterize the surface properties of the three-dimensional model of the standard object.

[0014] In the above design, the three-dimensional model feature point detection device can flexibly select (or adaptively select) a matching color information calculation method to calculate the color information corresponding to the three-dimensional model of the object to be detected by identifying whether the attribute information contains the color information of multiple points in the three-dimensional model of the object to be detected. In other words, for the case where the attribute information includes the color information of multiple points in the three-dimensional model of the object to be detected and the case where the attribute information does not include the color information of multiple points in the three-dimensional model of the object to be detected, the three-dimensional model feature point detection device can respectively select the corresponding color information determination method to determine the color information corresponding to the three-dimensional model of the object to be detected, so that it can be more targeted to different color information calculation scenarios, and the calculation of color information can be made more in line with the actual situation of the attribute information, so that the method can be applied to more scenarios and has stronger versatility.

[0015] In a possible design, a three-dimensional model feature point detection device establishes a point registration relationship between a three-dimensional model of an object to be detected and a three-dimensional model of a standard object, including: the three-dimensional model feature point detection device can first display a plurality of second calibration point information of the standard object, and obtain a plurality of first calibration point information of the three-dimensional model of the object to be detected, and then the three-dimensional model feature point detection device can use a rigid registration objective function to rigidly transform the three-dimensional model of the standard object according to the plurality of first calibration points of the three-dimensional model of the object to be detected and the plurality of second calibration points of the three-dimensional model of the standard object, determine a plurality of first registration points in the three-dimensional model of the standard object, and then The three-dimensional model feature point detection device can perform non-rigid transformation on the multiple first registration points of the three-dimensional model of the standard object according to the multiple first registration points in the three-dimensional model of the standard object using a non-rigid registration objective function to determine the multiple second registration points in the three-dimensional model of the standard object, and can determine the point registration relationship according to the one-to-one correspondence between the multiple second registration points and the multiple points in the three-dimensional model of the object to be detected, wherein the multiple first calibration points are obtained by the user by calibrating the multiple second calibration points one by one in the three-dimensional model of the object to be detected, and the multiple second registration points are used as points having a registration relationship with the multiple second points in the three-dimensional model of the object to be detected.

[0016] In the above design, the three-dimensional model feature point detection device comprehensively determines the one-to-one correspondence between multiple second registration points in the three-dimensional model of the standard object and multiple points in the three-dimensional model of the object to be detected (i.e., point registration relationship) by combining the rigid registration objective function and the non-rigid registration objective function. This can make the establishment of the point registration relationship more accurate, and at the same time ensure that the calculation of the multiple second registration points is more accurate, so that the calculation of the color information of the multiple points in the three-dimensional model of the object to be detected can be more accurate, and thus the rendering abnormality caused by inaccurate color information can be effectively avoided.

[0017] In one possible design, the three-dimensional model feature point detection device determines the color information corresponding to the three-dimensional model of the object to be detected based on the point registration relationship and the texture map corresponding to the three-dimensional model of the standard object, including: the three-dimensional model feature point detection device can first determine the color information of multiple second registration points from the texture map corresponding to the three-dimensional model of the standard object, and then, the three-dimensional model feature point detection device can use the color information of the multiple second registration points as the color information of the corresponding multiple points in the three-dimensional model of the object to be detected based on the point registration relationship, and then, the three-dimensional model feature point detection device can use the color information of the multiple points as the color information corresponding to the three-dimensional model of the object to be detected.

[0018] In the above design, after the three-dimensional model feature point detection device determines the color information of multiple second registration points from the texture map corresponding to the three-dimensional model of the standard object, it can accurately determine the color information of multiple second points through the one-to-one correspondence between the multiple second registration points and the multiple second points.

[0019] In one possible design, the first information is obtained in response to a user's editing operation on the detection of the object to be detected in a configuration interface (i.e., a configuration interface for users to upload relevant information of the object to be detected). For example, taking the three-dimensional feature point detection service interface displayed by the target application installed on the terminal device as an example, the user edits and submits the first information of the object to be detected (such as the name or identification of the object to be detected, the three-dimensional model of the object to be detected, the texture map corresponding to the three-dimensional model of the object to be detected, the name or identification of the standard object, the three-dimensional model of the standard object, or the texture map corresponding to the three-dimensional model of the standard object, etc.) on the three-dimensional feature point detection service interface. The terminal device obtains the first information in response to the user's editing operation on the first information of the object to be detected on the three-dimensional feature point detection service interface. Afterwards, the terminal device can send the first information to the three-dimensional model feature point detection device.

[0020] In the above design, by providing a configuration interface for the user, the user can flexibly edit the first information of the object to be detected on the configuration interface, which can meet the user's needs for three-dimensional feature point detection of three-dimensional models of different objects to be detected, and help to realize on-demand detection of three-dimensional feature points of the three-dimensional model of the object to be detected. At the same time, it can make it more convenient for the user to detect the three-dimensional feature points of the object to be detected.

[0021] In a possible design, the conversion relationship between the two-dimensional feature points and the three-dimensional feature points satisfies the following form:

[0022] p=KTP

[0023] Among them, K is used to represent the camera intrinsic parameter matrix, T is used to represent the transformation matrix, p is used to represent any two-dimensional feature point in the two-dimensional image, and P is used to represent the three-dimensional feature point corresponding to the two-dimensional feature point in the three-dimensional model of the object to be detected.

[0024] In the above design, on the one hand, the three-dimensional model feature point detection device can accurately render the three-dimensional model of the object to be detected to obtain a two-dimensional image by utilizing the conversion relationship in the above form. On the other hand, the three-dimensional model feature point detection device can also accurately convert the two-dimensional feature points in the two-dimensional image into corresponding three-dimensional feature points in the three-dimensional model of the object to be detected by utilizing the conversion relationship in the above form.

[0025] In the second aspect, the embodiment of the present application provides a possible three-dimensional model feature point detection device. Exemplarily, the three-dimensional model feature point detection device may be a service device (such as a computing device) having the functions required to implement the method, or may be any other device (such as a vehicle-mounted device or a robot device, etc.) having the functions required to implement the method, or may be a functional element (such as a plug-in, component, chip or circuit, etc.) in a service device having the functions required to implement the method, or may be a functional element set in any other device, and the functional element has the function of implementing the three-dimensional model feature point detection method. Exemplarily, taking the three-dimensional model feature point detection device as a chip set in a computing device as an example, when the three-dimensional model feature point detection device is a chip, the three-dimensional model feature point detection device includes a transceiver and a processor, but does not include a memory. Among them, the transceiver exists as an input and output interface, and the input and output interface is used for the chip to realize the transceiver of the computing device. The input and output interface may include an input interface and / or an output interface, the input interface can realize the reception of the computing device, and the output interface can be used to realize the sending of the computing device. The processor is used to read and execute corresponding computer programs or instructions so that the corresponding functions of the three-dimensional model feature point detection device are realized.

[0026] When the three-dimensional model feature point detection device is used to implement the function of the three-dimensional model feature point detection device in the method embodiment of the first aspect, the beneficial effects can be referred to the description of the three-dimensional model feature point detection device in the first aspect, which will not be repeated here. The three-dimensional model feature point detection device has the function of implementing the behavior of the three-dimensional model feature point detection device in the method example of the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In a possible design, the three-dimensional model feature point detection device includes an acquisition module, a determination module, a rendering module, a feature point detection module and a feature point conversion module. The acquisition module is used to acquire first information of the object to be detected, wherein the first information may include a three-dimensional model of the object to be detected. The determination module is used to determine the color information corresponding to the three-dimensional model of the object to be detected. The rendering module is used to render the three-dimensional model of the object to be detected into a two-dimensional image according to the color information. The feature point detection module is used to perform feature point detection on the two-dimensional image to obtain multiple two-dimensional feature points. The feature point conversion module is used to convert multiple two-dimensional feature points according to the conversion relationship between the two-dimensional feature points and the three-dimensional feature points to obtain multiple three-dimensional feature points, and the multiple three-dimensional feature points are used as detection feature points of the three-dimensional model of the object to be detected. These modules can perform the corresponding functions in any possible design of the first aspect above, and refer to the detailed description in the method example for details, which will not be repeated here.

[0027] In the third aspect, an embodiment of the present application provides a possible computing device, which has the function of implementing the behavior of the three-dimensional model feature point detection device in the method example of the first aspect above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The computing device includes a communication interface, a memory, and a processor. Among them, the communication interface is used to communicate with other devices, the memory is used to store computer programs or instructions, and the processor is coupled to the memory and the communication interface. When the processor executes the computer program or instruction, the computing device executes the method in any possible design of the first aspect above.

[0028] In a fourth aspect, an embodiment of the present application provides a possible computing device cluster, which has the function of implementing the behavior of the three-dimensional model feature point detection device in the method example of the first aspect above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The computing device cluster includes at least one computing device, any computing device includes a communication interface, a processor and a memory, and the processor in any computing device is configured to support the three-dimensional model feature point detection device to perform part or all of the functions of the method in any possible design of the first aspect above. The memory is coupled to the processor, which stores the necessary computer program instructions and data for the three-dimensional model feature point detection device.

[0029] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer program instructions. When the computer program instructions are executed by a computing device, the computing device executes a method in any possible design of the first aspect above.

[0030] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computing device, enables the computing device to execute a method in any possible design of the first aspect described above.

[0031] In a seventh aspect, an embodiment of the present application further provides a computer chip, which is coupled to a memory, and is used to read a computer program stored in the memory to execute a method in any possible design of the first aspect above.

[0032] In an eighth aspect, an embodiment of the present application further provides a chip system, the chip system including a processor for supporting a computer device to implement the method in any possible design of the first aspect above. In one possible design, the chip system also includes a memory, the memory being used to store the necessary programs and data of the computer device. The chip system may be composed of a computer chip, or may include a computer chip and other discrete devices.

[0033] Based on the implementations provided in the above aspects, the present application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of a possible application scenario provided by an embodiment of the present application is exemplified;

[0035] Figure 2 A schematic diagram showing a flow chart of a method for detecting feature points of a three-dimensional model provided in an embodiment of the present application;

[0036] Figure 3a A schematic diagram of a three-dimensional feature point detection service interface for assisting a user in uploading relevant information of an object to be detected provided in an embodiment of the present application is exemplarily shown;

[0037] Figure 3b A schematic diagram of a 3D feature point detection service interface for assisting a user in selecting a 3D model of a standard object to be referenced is exemplarily shown in an embodiment of the present application;

[0038] Figure 3c A schematic diagram of another 3D feature point detection service interface for assisting a user in selecting a 3D model of a standard object to be referenced provided in an embodiment of the present application is exemplarily shown;

[0039] Figure 4 A schematic diagram showing a possible structure of a three-dimensional model feature point detection device provided in an embodiment of the present application is exemplified;

[0040] Figure 5 A schematic diagram showing a possible structure of a computing device provided by an embodiment of the present application is exemplified;

[0041] Figure 6 A schematic diagram showing a possible computing device cluster structure provided by an embodiment of the present application is exemplified;

[0042] Figure 7 A schematic diagram illustrating the structure of another possible computing device cluster provided in an embodiment of the present application is exemplified. DETAILED DESCRIPTION

[0043] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0044] The following introduces possible application scenarios to which the load balancing service deployment method provided by the present application is applicable. It should be noted that these introductions are for the convenience of understanding by those skilled in the art and do not constitute a limitation on the scope of protection claimed by the present application.

[0045] Figure 1 The following is a schematic diagram showing a possible application scenario to which the present application embodiment is applicable. Figure 1 As shown, the application scenario may include a terminal device 100 and a service device 200. Optionally, the terminal device 100 and the service device 200 may be connected to each other through one or more networks. The network may be a wired network or a wireless network, for example, the wireless network may be a wireless fidelity (WIREless-Fidelity, WIFI) network, or a mobile cellular network, or other forms of networks, which are not limited in the embodiments of the present application.

[0046] Wherein, the terminal device 100 is installed with a target application for assisting in detecting the feature points of the three-dimensional model of the object to be detected. Wherein, the user can edit and submit the relevant information of the object to be detected to the service device 200 through the target application. In one example, the relevant information of the object to be detected includes the three-dimensional model of the object to be detected, and in another example, the relevant information of the object to be detected includes the three-dimensional model of the object to be detected and the texture map corresponding to the three-dimensional model of the object to be detected. For example, the target application can be a client, a web page application or a small program embedded in other applications, etc. Optionally, a display can be included in the terminal device to present the configuration interface provided by the service device 200 (such as a configuration interface for assisting users to upload the relevant information of the object to be detected), and realize human-computer interaction. For example, the display can display a configuration interface, such as a three-dimensional feature point detection service interface provided by the display service device 200 for assisting users to upload the relevant information of the object to be detected. Wherein, the information edited by the user or the information provided to the user, etc. are displayed on the configuration interface.

[0047] Exemplarily, the terminal device 100 may be a smart phone, a tablet computer, a desktop computer, a computer with wireless transceiver function (such as a laptop computer, etc.), a PPD (Personal Digital Assistant), a mobile internet device (MID), a vehicle terminal, a wearable device with wireless communication function (such as a smart watch, a smart bracelet, smart glasses or a smart helmet, etc.), a virtual reality (VR) device, an augmented reality (AR) device or a smart home device (such as a smart speaker or a smart TV, etc.), etc. It should be understood that the present application does not limit the specific device form of the terminal device.

[0048] The service device 200 is a device (such as a server or computing device) used to provide services for users to perform corresponding operations on the target application installed on the terminal device 100. For example, the service device 200 can be a three-dimensional model feature point detection system, or it can also be a computing device (such as a computer), a computing device cluster or other arbitrary device with a three-dimensional model feature point detection function, or it can also be a functional element (such as a three-dimensional model feature point detection device or a chip with a three-dimensional model feature point detection function, etc.) set in a computing device, a computing device cluster or other arbitrary device. Exemplarily, the three-dimensional model feature point detection device can be a chip or a circuit, etc.; the three-dimensional model feature point detection system can be an independent physical server, or it can also be a server cluster or distributed system composed of multiple physical servers, or it can also be a cloud server for providing cloud services, cloud computing, cloud storage, cloud communications, network services, security services, big data and other cloud computing services.

[0049] Exemplarily, the terminal device 100 is a desktop computer, the target application is a web browser installed on the desktop computer for assisting users to upload relevant information of the object to be detected, and the service device 200 is a computing device with a three-dimensional model feature point detection function. When a user (such as user A) needs to detect the feature points of the three-dimensional model of a certain object to be detected (such as face A), user A can log in to the configuration interface (such as a three-dimensional feature point detection service interface) provided by the service device 200 for assisting users to upload relevant information of the object to be detected through the web browser installed on the desktop computer used. The three-dimensional feature point detection service interface is used to provide users with editing of relevant information of the object to be detected. In one example, user A can select the three-dimensional model of the face A to be detected on the three-dimensional feature point detection service interface for uploading. In another example, user A can also select the three-dimensional model of the face A to be detected on the three-dimensional feature point detection service interface and select the texture map corresponding to the three-dimensional model of face A to be uploaded together. After the computing device obtains the three-dimensional model of face A (or the three-dimensional model of face A and the texture map corresponding to the three-dimensional model of face A) uploaded by user A through a web browser, a feature point detection process for the three-dimensional model of face A can be executed.

[0050] It should be noted that Figure 1 Only one possible application scenario is provided schematically, and this schematic application scenario is intended to more clearly illustrate the technical solution of the embodiment of the present application, and does not limit the application scenario of the three-dimensional model feature point detection method provided by the present application. Figure 1 The form and quantity of each device in the application scenario shown are only for example and do not constitute a limitation of the present application. Figure 1The names of the various devices in the application scenarios shown are only examples, and the names of the various devices in the specific implementation may also be other names, which are not specifically limited in this application. In addition, it is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0051] The following is based on Figure 1 The application scenario shown is used to introduce in detail the specific implementation of the three-dimensional model feature point detection method in the embodiment of the present application.

[0052] Figure 2 The following is a flow chart showing a method for detecting feature points of a three-dimensional model provided in an embodiment of the present application. Figure 1 The method flow may be executed by a 3D model feature point detection device or a component (such as a chip, a chip system or a circuit, etc.) that can support the 3D model feature point detection device to implement the functions required by the method. Exemplarily, the 3D model feature point detection device may be Figure 1 The service device 200 shown in FIG. 200 (such as a computing device with a 3D model feature point detection function or a computing device cluster with a 3D model feature point detection function, etc.), or it may also be Figure 1 The functional element (such as a plug-in, component, chip or circuit, etc.) in the service device 200 shown in the figure has the function of implementing the three-dimensional model feature point detection method. For example, taking the service device 200 as a computing device with a three-dimensional model feature point detection function, the three-dimensional model feature point detection device can be the computing device with the three-dimensional model feature point detection function, or it can also be an independent unit embedded in the computing device for implementing the three-dimensional model feature point detection function, or it can also be a functional component (such as a chip) encapsulated in the computing device for implementing the three-dimensional model feature point detection function. In order to facilitate the introduction of the technical solution provided in the embodiment of the present application, the following is an introduction taking the three-dimensional model feature point detection method executed by the three-dimensional model feature point detection device as an example. Figure 2 As shown, the method includes:

[0053] Step 201: A three-dimensional model feature point detection device obtains first information of an object to be detected.

[0054] Exemplarily, the object to be detected may be an object with a three-dimensional structure. For example, the object to be detected may include but is not limited to a human face, an animal face, a building, a smart home device (such as a television or an air conditioner, etc.) or a vehicle, etc.

[0055] In one example, the first information may be obtained by the 3D model feature point detection device in response to the user's editing operation on the information of the object to be detected on the configuration interface (i.e., a user interface for assisting the user in uploading relevant information of the object to be detected (such as the 3D model of the object to be detected), such as a 3D feature point detection service interface) provided by the 3D model feature point detection device and displayed on the target application of the terminal device. Exemplarily, a schematic diagram of the 3D feature point detection service interface displayed by the target application for assisting the user in uploading relevant information of the object to be detected (such as the first information) can be seen in Figure 3a In this Figure 3a The illustrated three-dimensional feature point detection service interface displays some relevant information of the object to be detected that the user needs to edit, such as the name or logo of the object to be detected, the three-dimensional model of the object to be detected, or the texture map corresponding to the three-dimensional model of the object to be detected.

[0056] For example, taking the configuration interface displayed by the target application as a three-dimensional feature point detection service interface displayed by a web browser for assisting users to upload relevant information of an object to be detected as an example, when a certain user (such as user B) needs to detect the feature points of a three-dimensional model of a certain object to be detected (such as face B), user B can log in to the three-dimensional feature point detection service interface through a web browser installed on the terminal device used, and select the three-dimensional model of face B on the three-dimensional feature point detection service interface for upload. In response to the user B's selection and upload operation on the three-dimensional feature point detection service interface, the terminal device obtains the three-dimensional model of face B and generates first information, which includes the three-dimensional model of face B. Afterwards, the terminal device can send the first information to the three-dimensional model feature point detection device.

[0057] For another example, taking the configuration interface displayed by the target application as a three-dimensional feature point detection service interface displayed by a web browser for assisting users to upload relevant information of the object to be detected as an example, when user B needs to detect feature points of the three-dimensional model of face B, user B can log in to the three-dimensional feature point detection service interface through the web browser installed on the terminal device used, and select the three-dimensional model of face B and the texture map corresponding to the three-dimensional model of face B on the three-dimensional feature point detection service interface for upload. In response to the user B's selection of the upload operation on the three-dimensional feature point detection service interface, the terminal device obtains the three-dimensional model of face B and the texture map corresponding to the three-dimensional model of face B, and generates first information, which includes the three-dimensional model of face B and the texture map corresponding to the three-dimensional model of face B. Afterwards, the terminal device can send the first information to the three-dimensional model feature point detection device.

[0058] In another example, the first information may be obtained by the three-dimensional model feature point detection device from a database (such as a database for storing relevant information of the object to be detected).

[0059] For example, when a user (such as user B) needs to detect the feature points of the three-dimensional model of face B, a first request can be sent to the three-dimensional model feature point detection device through the terminal device. The first request is used to request the detection of the feature points of the three-dimensional model of face B. In one possible implementation, after obtaining the first request, the three-dimensional model feature point detection device can obtain first information from the database according to the first request, and the first information includes the three-dimensional model of face B. In another possible implementation, after obtaining the first request, the three-dimensional model feature point detection device can obtain first information from the database according to the first request, and the first information includes the three-dimensional model of face B and the texture map corresponding to the three-dimensional model of face B.

[0060] Exemplarily, the first information may include, but is not limited to, a three-dimensional model of an object to be detected, a texture map corresponding to the three-dimensional model of an object to be detected, a three-dimensional model of a standard object, or a texture map corresponding to the three-dimensional model of a standard object. The texture map corresponding to the three-dimensional model of an object to be detected is used to characterize the surface properties of the three-dimensional model of an object to be detected, the three-dimensional model of a standard object is used as a reference template for the three-dimensional model of an object to be detected, and the texture map corresponding to the three-dimensional model of a standard object is used to characterize the surface properties of the three-dimensional model of a standard object. For example, surface properties may include, but are not limited to, color information (or may be referred to as color information, such as RGB information), height information, surface roughness (such as a smooth surface or an uneven surface), or normal information, etc.

[0061] Step 202: The three-dimensional model feature point detection device determines the color information corresponding to the three-dimensional model of the object to be detected.

[0062] Step 203: The 3D model feature point detection device renders the 3D model of the object to be detected into a two-dimensional (2D) image according to the color information corresponding to the 3D model of the object to be detected.

[0063] Step 204: The three-dimensional model feature point detection device performs feature point detection on the two-dimensional image to obtain a plurality of two-dimensional feature points.

[0064] Step 205: The three-dimensional model feature point detection device converts the multiple two-dimensional feature points according to the conversion relationship between the two-dimensional feature points and the three-dimensional feature points to obtain multiple three-dimensional feature points.

[0065] Optionally, after acquiring the first information of the object to be detected, the three-dimensional model feature point detection device can perform texture map recognition on the first information to obtain a recognition result, and can flexibly select (or adaptively select) different detection processes to determine the three-dimensional feature points of the three-dimensional model of the object to be detected based on the recognition result. The recognition result is used to indicate that the first information contains the texture map corresponding to the three-dimensional model of the object to be detected, or is used to indicate that the first information does not contain the texture map corresponding to the three-dimensional model of the object to be detected. In this way, the three-dimensional model feature point detection device can flexibly select (or adaptively select) an appropriate detection process to detect the three-dimensional feature points of the three-dimensional model of the object to be detected based on the first information, thereby making the feature point detection of the three-dimensional model of the object to be detected more targeted and flexible.

[0066] Exemplarily, based on the above content, the detection process is introduced below through the following possible examples.

[0067] Detection process one: When the first information contains a texture map (such as a UV texture map) corresponding to the three-dimensional model of the object to be detected, the three-dimensional model feature point detection device can determine the color information of multiple points in the three-dimensional model of the object to be detected based on the texture map corresponding to the three-dimensional model of the object to be detected. Afterwards, the three-dimensional model feature point detection device can render the three-dimensional model of the object to be detected into a two-dimensional image based on the color information of multiple points in the three-dimensional model of the object to be detected. Then, the three-dimensional model feature point detection device performs feature point detection on the two-dimensional image to obtain multiple two-dimensional feature points. Finally, the three-dimensional model feature point detection device can convert multiple two-dimensional feature points into corresponding multiple three-dimensional feature points based on the conversion relationship between the two-dimensional feature points and the three-dimensional feature points. The multiple three-dimensional feature points are used as detection feature points of the three-dimensional model of the object to be detected.

[0068] For example, when the three-dimensional model feature point detection device performs feature point detection on a two-dimensional image, it can use an existing relatively mature two-dimensional image feature point detection algorithm (such as the dlib key point detection algorithm, the Face-alignment detection algorithm, or other two-dimensional feature point detection algorithms based on deep learning, etc.) to perform feature point detection on the two-dimensional image to obtain multiple two-dimensional feature points contained in the two-dimensional image. Among them, the multiple two-dimensional feature points all have obvious semantic information. In this way, the three-dimensional model feature point detection device can achieve accurate detection of semantic feature points in the two-dimensional image by adopting the existing two-dimensional image feature point detection algorithm (or it can be called the existing two-dimensional image feature point detection model), and can achieve strong robustness to changes in illumination, posture or expression in the two-dimensional image, and can effectively resist changes in viewing angle and illumination, thereby effectively solving the problem that the existing traditional three-dimensional feature point detection algorithm cannot accurately detect accurate semantic information (such as three-dimensional feature points).

[0069] Exemplarily, the following describes the implementation process of a three-dimensional model feature point detection device determining color information of multiple points in a three-dimensional model of an object to be detected based on a texture map corresponding to the three-dimensional model of the object to be detected through the following possible examples.

[0070] Example 1: A three-dimensional model feature point detection device can calculate the color information (such as RGB information) of multiple points in the three-dimensional model of the object to be detected by bilinear interpolation according to the texture map corresponding to the three-dimensional model of the object to be detected. Exemplarily, the multiple points in the three-dimensional model of the object to be detected may refer to all the points in the three-dimensional model of the object to be detected, or may refer to some of the points in the three-dimensional model of the object to be detected.

[0071] Example 2: A three-dimensional model feature point detection device can calculate the color information of multiple points in the three-dimensional model of the object to be detected by nearest neighbor interpolation based on the texture map corresponding to the three-dimensional model of the object to be detected.

[0072] It should be understood that the above content is only an exemplary introduction to several implementation methods of three-dimensional model feature point detection devices to calculate the color information of multiple points in the three-dimensional model of the object to be detected. The three-dimensional model feature point detection device can also adopt other implementation methods (such as linear interpolation) to calculate the color information of multiple points in the three-dimensional model of the object to be detected, and the embodiments of the present application will not list them one by one.

[0073] For example, the conversion relationship between two-dimensional feature points and three-dimensional feature points can satisfy the following form:

[0074] p=KTP

[0075] Among them, K is used to represent the camera intrinsic parameter matrix; T is used to represent the transformation matrix; p is used to represent any two-dimensional feature point in the two-dimensional image, or it can also be used to represent the position coordinate information of any two-dimensional feature point in the two-dimensional image; P is used to represent the three-dimensional feature point corresponding to the two-dimensional feature point in the three-dimensional model of the object to be detected, or it can also be used to represent the position coordinate information of the three-dimensional feature point corresponding to the two-dimensional feature point in the three-dimensional model of the object to be detected.

[0076] When the three-dimensional model feature point detection device converts multiple two-dimensional feature points, the conversion relationship in the above form can be used to back-project each of the multiple two-dimensional feature points back to the three-dimensional space to obtain the three-dimensional feature point corresponding to the two-dimensional feature point in the three-dimensional model of the object to be detected. Exemplarily, taking the object to be detected as a human face as an example, the three-dimensional feature point can be used to characterize the structural attributes of the human face, and the three-dimensional feature point can be a feature point located on one of the multiple parts of the human face (such as the forehead, eyes, eyebrows, nose or mouth, etc.).

[0077] Exemplarily, for the above detection process 1, taking the object to be detected as a certain face (such as face A) as an example, when the first information contains the UV texture map corresponding to the three-dimensional model of face A, the three-dimensional model feature point detection device can calculate the color information of multiple points in the three-dimensional model of face A by bilinear interpolation according to the texture map corresponding to the three-dimensional model of face A. Afterwards, the three-dimensional model feature point detection device can render the three-dimensional model of face A into a two-dimensional image according to the color information of multiple points in the three-dimensional model of face A. Then, the three-dimensional model feature point detection device can use a certain two-dimensional image feature point detection algorithm (such as Face-alignment detection algorithm) to perform feature point detection on the two-dimensional image to obtain multiple two-dimensional feature points with obvious semantic information. Finally, the three-dimensional model feature point detection device can use the above-mentioned conversion relationship to back-project each of the multiple two-dimensional feature points back to the three-dimensional space, and obtain the three-dimensional feature points in the three-dimensional model of face A that correspond to the multiple two-dimensional feature points. Optionally, the three-dimensional model feature point detection device may also use the above-mentioned conversion relationship when rendering the three-dimensional model of face A into a two-dimensional image.

[0078] Detection process two: When the first information does not contain the texture map corresponding to the three-dimensional model of the object to be detected, the three-dimensional model feature point detection device parses the three-dimensional model of the object to be detected, obtains the attribute information of the three-dimensional model of the object to be detected, and can perform color information recognition on the attribute information of the three-dimensional model of the object to be detected to obtain a recognition result. In one example, the attribute information of the three-dimensional model of the object to be detected includes the position coordinate information of multiple points in the three-dimensional model of the object to be detected. In another example, the attribute information of the three-dimensional model of the object to be detected includes the position coordinate information of multiple points in the three-dimensional model of the object to be detected and the color information of multiple points in the three-dimensional model of the object to be detected. Among them, the recognition result is used to indicate that the attribute information of the three-dimensional model of the object to be detected contains the color information of multiple points in the three-dimensional model of the object to be detected, or is used to indicate that the attribute information of the three-dimensional model of the object to be detected does not contain the color information of multiple points in the three-dimensional model of the object to be detected.

[0079] Afterwards, the three-dimensional model feature point detection device can select different color information calculation methods to determine the color information corresponding to the three-dimensional model of the object to be detected according to the recognition result, and render the three-dimensional model of the object to be detected into a two-dimensional image according to the color information corresponding to the three-dimensional model of the object to be detected. Then, the three-dimensional model feature point detection device performs feature point detection on the two-dimensional image to obtain multiple two-dimensional feature points. Finally, the three-dimensional model feature point detection device can convert multiple two-dimensional feature points into corresponding multiple three-dimensional feature points according to the conversion relationship between the two-dimensional feature points and the three-dimensional feature points, and the multiple three-dimensional feature points are used as the detection feature points of the three-dimensional model of the object to be detected.

[0080] Exemplarily, the following describes the implementation process of the three-dimensional model feature point detection device selecting different color information calculation methods to determine the color information corresponding to the three-dimensional model of the object to be detected according to the recognition result through the following possible implementation methods.

[0081] Method 1: When the attribute information of the three-dimensional model of the object to be detected includes color information of multiple points in the three-dimensional model of the object to be detected, the three-dimensional model feature point detection device can use the color information of multiple points in the three-dimensional model of the object to be detected as the color information corresponding to the three-dimensional model of the object to be detected.

[0082] Method 2: When the attribute information of the three-dimensional model of the object to be detected does not contain the color information of multiple points in the three-dimensional model of the object to be detected, the three-dimensional model feature point detection device can establish a point registration relationship between the three-dimensional model of the object to be detected and the three-dimensional model of the standard object. Afterwards, the three-dimensional model feature point detection device can determine the color information corresponding to the three-dimensional model of the object to be detected based on the point registration relationship and the texture map corresponding to the three-dimensional model of the standard object.

[0083] Based on the above-mentioned method 2, the implementation process of the three-dimensional model feature point detection device establishing the point registration relationship between the three-dimensional model of the object to be detected and the three-dimensional model of the standard object is introduced through the following possible implementation methods.

[0084] Implementation method 1: When the first information includes multiple first calibration points of the three-dimensional model of the object to be detected and multiple second calibration points of the three-dimensional model of the standard object, the three-dimensional model feature point detection device can use the rigid registration objective function to perform a rigid transformation on the three-dimensional model of the standard object according to the multiple first calibration points of the three-dimensional model of the object to be detected and the multiple second calibration points of the three-dimensional model of the standard object, and determine multiple first registration points in the three-dimensional model of the standard object. Afterwards, the three-dimensional model feature point detection device can use the non-rigid registration objective function to perform a non-rigid transformation on the multiple first registration points of the three-dimensional model of the standard object according to the multiple first registration points in the three-dimensional model of the standard object, and determine multiple second registration points in the three-dimensional model of the standard object. Then, the three-dimensional model feature point detection device can determine the point registration relationship according to the one-to-one correspondence between the multiple second registration points and the multiple points. Among them, the multiple first calibration points can be obtained by the user by calibrating the multiple second calibration points one by one in the three-dimensional model of the object to be detected, and the number of the multiple first calibration points is the same as the number of the multiple second calibration points.

[0085] In a possible implementation, the three-dimensional model feature point detection device can determine the first variable value combination when the rigid registration objective function takes the minimum function value according to the multiple first calibration points of the three-dimensional model of the object to be detected and the multiple second calibration points of the three-dimensional model of the standard object. Wherein, the first variable value combination can include the scale, rotation matrix and translation matrix between the three-dimensional model of the object to be detected and the three-dimensional model of the standard object. Afterwards, the three-dimensional model feature point detection device can convert the multiple first points in the three-dimensional model of the standard object according to the first variable value combination, obtain the corresponding multiple second points (i.e., multiple first registration points), and can determine the second variable value combination when the non-rigid registration objective function takes the minimum function value according to the multiple second points and the points in the three-dimensional model of the object to be detected that are closest to the multiple second points. Wherein, the second variable value combination can include multiple third points (i.e., multiple second registration points), and the multiple third points are points obtained by adjusting (or optimizing) the multiple second points respectively through the non-rigid registration objective function, and the multiple third points are also points in the three-dimensional model of the standard object. Then, the 3D model feature point detection device determines the third point closest to each point in the 3D model of the object to be detected from the multiple third points, and determines the correspondence between the point and the third point. Finally, the 3D model feature point detection device establishes a point registration relationship (or a correspondence relationship between points) between the 3D model of the object to be detected and the 3D model of the standard object based on the one-to-one correspondence between the multiple points in the 3D model of the object to be detected and the third points closest to the multiple points.

[0086] Exemplarily, based on the above-mentioned implementation method 1, the implementation process of a three-dimensional model feature point detection device obtaining multiple first calibration points of the three-dimensional model of the object to be detected and multiple second calibration points of the three-dimensional model of the standard object is introduced through the following possible examples.

[0087] Example 1: When the first information is obtained by the three-dimensional model feature point detection device in response to the user's editing operation on the three-dimensional feature point detection service interface displayed by the target application of the terminal device for assisting the user to upload the relevant information of the object to be detected, the multiple first calibration points of the three-dimensional model of the object to be detected and the multiple second calibration points of the three-dimensional model of the standard object included in the first information are obtained by the terminal device in response to the user on the three-dimensional feature point detection service interface according to the feature point calibration information of the three-dimensional model of the standard object selected by the user. Afterwards, the terminal device carries the multiple first calibration points of the three-dimensional model of the object to be detected and the multiple second calibration points of the three-dimensional model of the standard object in the first information and sends them to the three-dimensional model feature point detection device. Among them, the feature point calibration information may include the feature point calibration quantity and / or the feature point calibration position. Optionally, the three-dimensional feature point detection service interface may also include an edit box for the three-dimensional model of the standard object, which provides a plurality of optional three-dimensional models of standard objects, and each optional three-dimensional model of the standard object has different feature point calibration information to choose from.

[0088] For example, when a user (such as user A) logs in to a three-dimensional feature point detection service interface for assisting users in uploading relevant information of an object to be detected through a target application (such as a web browser) installed on a terminal device to upload the first information of an object to be detected, the user can select a three-dimensional model of a referenced standard object on the three-dimensional feature point detection service interface, and can perform feature point calibration on the three-dimensional model of the object to be detected based on the feature point calibration information of the selected three-dimensional model of the standard object. In response to the user's selection and calibration operations on the three-dimensional feature point detection service interface, the terminal device can obtain a three-dimensional model of the object to be detected having multiple first calibration points and a three-dimensional model of a standard object having multiple second calibration points, and can generate first information based on the three-dimensional model of the object to be detected having multiple first calibration points and the three-dimensional model of the standard object having multiple second calibration points, and send the first information to a three-dimensional model feature point detection device. Optionally, the terminal device may also obtain the texture map corresponding to the three-dimensional model of the object to be detected and / or the texture map corresponding to the three-dimensional model of the standard object in response to the user's selection operation on the three-dimensional feature point detection service interface for the texture map corresponding to the three-dimensional model of the object to be detected and / or the texture map corresponding to the three-dimensional model of the standard object, and may carry the texture map corresponding to the three-dimensional model of the object to be detected and / or the texture map corresponding to the three-dimensional model of the standard object in the first information and send it to the three-dimensional model feature point detection device.

[0089] Example 2: When the first information is obtained from a database by a three-dimensional model feature point detection device, the database stores a plurality of three-dimensional models of objects to be detected with different numbers of calibration points. For example, taking the three-dimensional model of the object to be detected as the three-dimensional model of face A as an example, the database stores a three-dimensional model of face A with 7 calibration points, a three-dimensional model of face A with 8 calibration points, a three-dimensional model of face A with 9 calibration points, or a three-dimensional model of face A with other numbers of calibration points (such as calibration points greater than 9), or the database may also store a texture map corresponding to the three-dimensional model of face A. Optionally, the database may also store a three-dimensional model of a standard face with different numbers of calibration points (such as 7 calibration points, 8 calibration points, 9 calibration points or other numbers of calibration points), or may also store a texture map corresponding to the three-dimensional model of a standard face.

[0090] For example, when a user (such as user B) needs to detect the three-dimensional model of a certain object to be detected (such as face A), a second request can be sent to the three-dimensional model feature point detection device through the terminal device. The second request is used to request the detection of the three-dimensional model of face A carrying 7 calibration points. After obtaining the second request, the three-dimensional model feature point detection device can select the three-dimensional model of face A carrying 7 calibration points from the database according to the second request. In one example, when three-dimensional models of standard faces carrying different numbers of calibration points are stored in the database, the three-dimensional model feature point detection device can also select a three-dimensional model of a standard face carrying 7 calibration points from the database according to the second request. Optionally, when the database stores a texture map corresponding to the three-dimensional model of a standard face, the three-dimensional model feature point detection device can also obtain the texture map corresponding to the three-dimensional model of the standard face from the database according to the second request. In another example, when the three-dimensional model of a standard face with different numbers of calibration points is not stored in the database, and the three-dimensional model of a standard face with different numbers of calibration points is stored in the memory or other storage area of ​​the three-dimensional model feature point detection device, the three-dimensional model feature point detection device can select the three-dimensional model of a standard face with 7 calibration points from the memory or other storage area according to the second request. Optionally, when the texture map corresponding to the three-dimensional model of the standard face is not stored in the database and the texture map corresponding to the three-dimensional model of the standard face is stored in the memory or other storage area of ​​the three-dimensional model feature point detection device, the three-dimensional model feature point detection device can also obtain the texture map corresponding to the three-dimensional model of the standard face from the memory or other storage area according to the second request.

[0091] Implementation method 2: When the first information does not include multiple first calibration points of the three-dimensional model of the object to be detected and multiple second calibration points of the three-dimensional model of the standard object and the first information is from the terminal device, the three-dimensional model feature point detection device can send a first indication information to the terminal device. The first indication information is used to instruct the user to select the three-dimensional model of the standard object to be referenced (such as the three-dimensional model of the standard object with a certain number of calibration points) on the three-dimensional feature point detection service interface (such as service interface B) provided by the three-dimensional model feature point detection device displayed by the target application in the terminal device for assisting the user in selecting the three-dimensional model of the standard object to be referenced, and according to the feature point calibration information of the three-dimensional model of the selected standard object, the feature point calibration of the three-dimensional model of the object to be detected is performed on the three-dimensional feature point detection service interface (such as service interface A) used by the user to edit the relevant information of the object to be detected (such as the first information). After receiving the first indication information, the terminal device displays the service interface B to the user, prompting the user to select the three-dimensional model of the standard object with a certain number of calibration points (such as 7 calibration points) on the service interface B. In response to the user's selection operation on the service interface B for the three-dimensional model of the standard object carrying 7 calibration points, the terminal device obtains the three-dimensional model of the standard object carrying 7 calibration points. Exemplarily, in one example, the service interface B may be actively pushed to the terminal device by the three-dimensional model feature point detection device after sending the first indication information to the terminal device, and presented to the user in the target application. In another example, the service interface B may also be a pop-up message pushed to the terminal device by the three-dimensional model feature point detection device after sending the first indication information to the terminal device, and the pop-up message is displayed in the target application, and the prompt information included in the pop-up message is used to prompt the user to click the button for displaying the service interface B, or the pop-up message includes a URL for assisting the user to open the service page B.

[0092] In addition, in response to the user's feature point calibration operation on the service interface A for the three-dimensional model of the object to be detected, the terminal device obtains the three-dimensional model of the object to be detected with 7 calibration points. Afterwards, the terminal device can send the three-dimensional model of the object to be detected with 7 calibration points and the three-dimensional model of the standard object with 7 calibration points to the three-dimensional model feature point detection device. Among them, the three-dimensional model of the standard object with different numbers of calibration points is provided by the three-dimensional model feature point detection device, and of course the three-dimensional model feature point detection device can also provide the texture map corresponding to the three-dimensional model of the standard object.

[0093] In one example, the target application displays a 3D feature point detection service interface for assisting the user in selecting a 3D model of a standard object to be referenced. Figure 3b In this Figure 3bIn the illustrated three-dimensional feature point detection service interface, some calibration templates of three-dimensional models of standard objects are displayed for users to choose, such as calibration template 1 of the three-dimensional model of the standard object, calibration template 2 of the three-dimensional model of the standard object, calibration template 3 of the three-dimensional model of the standard object, calibration template 4 of the three-dimensional model of the standard object, or calibration template 5 of the three-dimensional model of the standard object. Among them, each calibration template of the three-dimensional model of the standard object corresponds to a certain number of calibration points, and the positions of the calibration points of the number are also marked. For example, the number of calibration points corresponding to the calibration template 1 of the three-dimensional model of the standard object is 7, and the positions of the 7 calibration points are also marked, and the number of calibration points corresponding to the calibration template 2 of the three-dimensional model of the standard object is 8, and the positions of the 8 calibration points are also marked. Exemplarily, the calibration template of the three-dimensional model of the standard object can be displayed to the user in the form of a picture (such as a three-dimensional picture), so that the user can intuitively and clearly see how many calibration points there are, and where the position of each calibration point is, so that the user can accurately refer to the calibration template of the three-dimensional model of the standard object to calibrate the feature points of the three-dimensional model of the object to be detected. For example, taking the case where the calibration template of the three-dimensional model of the standard object is presented to the user in the form of a three-dimensional picture, when the user selects the three-dimensional picture of the calibration template of the three-dimensional model of a certain standard object (such as the calibration template 1 of the three-dimensional model of the standard object), the user can refer to the number and position of the calibration points in the three-dimensional picture of the calibration template 1 of the three-dimensional model of the standard object to calibrate the three-dimensional model of the object to be detected. In response to the user's selection operation of the calibration template 1 of the three-dimensional model of the standard object, the terminal device can also obtain the user's selection information of the calibration template 1 of the three-dimensional model of the standard object, and can notify the three-dimensional model feature point detection device of the information that the user has selected the calibration template 1 of the three-dimensional model of the standard object.

[0094] In another example, the target application displays a 3D feature point detection service interface for assisting the user in selecting a 3D model of a standard object to be referenced. Figure 3c In this Figure 3cIn the illustrated three-dimensional feature point detection service interface, a drop-down selection box is displayed for users to select the calibration template of the three-dimensional model of the standard object. The drop-down selection box includes a plurality of different calibration templates of the three-dimensional model of the standard object, such as calibration template 1 of the three-dimensional model of the standard object, calibration template 2 of the three-dimensional model of the standard object, calibration template 3 of the three-dimensional model of the standard object, calibration template 4 of the three-dimensional model of the standard object, or calibration template 5 of the three-dimensional model of the standard object. Among them, each calibration template of the three-dimensional model of the standard object corresponds to a certain number of calibration points, and the positions of the calibration points are also marked. For example, the number of calibration points corresponding to calibration template 1 of the three-dimensional model of the standard object is 7, and the positions of the 7 calibration points are also marked. For example, taking the example of presenting the calibration template of the three-dimensional model of the standard object to the user in the form of a three-dimensional picture, when the user selects the calibration template of the three-dimensional model of a certain standard object (such as calibration template 1 of the three-dimensional model of the standard object), in the calibration template, the calibration template is selected. Figure 3c In the illustrated calibration point template display area of ​​the three-dimensional model of the standard object, a three-dimensional picture of the three-dimensional model of the standard object carrying seven calibration points is displayed.

[0095] When the first information does not include multiple first calibration points of the three-dimensional model of the object to be detected and multiple second calibration points of the three-dimensional model of the standard object and the first information comes from a database, if the database stores multiple three-dimensional models of the object to be detected carrying different numbers of calibration points and three-dimensional models of standard objects carrying different numbers of calibration points, the three-dimensional model feature point detection device can select the three-dimensional model of the object to be detected carrying a certain number of calibration points (for example, 7 calibration points) in the database, and can select the three-dimensional model of the standard object carrying the 7 calibration points in the database.

[0096] Afterwards, after acquiring the three-dimensional model of the object to be detected carrying a certain number of calibration points (for example, 7 calibration points) and the three-dimensional model of the standard object carrying the same number of calibration points, the three-dimensional model feature point detection device can establish a point registration relationship between the three-dimensional model of the object to be detected and the three-dimensional model of the standard object according to the number of calibration points carried by the three-dimensional model of the object to be detected and the number of calibration points carried by the three-dimensional model of the standard object. Among them, the implementation method of the three-dimensional model feature point detection device in the second implementation method establishing the point registration relationship between the three-dimensional model of the object to be detected and the three-dimensional model of the standard object according to the number of calibration points carried by the three-dimensional model of the object to be detected and the number of calibration points carried by the three-dimensional model of the standard object can refer to the implementation method of the three-dimensional model feature point detection device in the above-mentioned first implementation method establishing the point registration relationship between the three-dimensional model of the object to be detected and the three-dimensional model of the standard object according to the multiple first calibration points of the three-dimensional model of the object to be detected and the multiple second calibration points of the three-dimensional model of the standard object, which will not be repeated here.

[0097] After the three-dimensional model feature point detection device establishes a point registration relationship between the three-dimensional model of the object to be detected and the three-dimensional model of the standard object through the above-mentioned implementation method 1 or implementation method 2, the color information of multiple third points (i.e., multiple second registration points) can be determined from the texture map corresponding to the three-dimensional model of the standard object. Afterwards, the three-dimensional model feature point detection device can, based on the one-to-one correspondence between the multiple third points and the multiple points in the three-dimensional model of the object to be detected, use the color information of the third point corresponding to each point in the three-dimensional model of the object to be detected as the color information of the point, thereby obtaining the color information of multiple points in the three-dimensional model of the object to be detected. Then, the three-dimensional model feature point detection device can use the color information of the multiple points as the color information corresponding to the three-dimensional model of the object to be detected.

[0098] Optionally, the implementation method of the three-dimensional model feature point detection device in implementation method one determining the color information of multiple second registration points in the three-dimensional model of the standard object from the texture map corresponding to the three-dimensional model of the standard object can be referred to the implementation method of the three-dimensional model feature point detection device in the above-mentioned detection process one determining the color information of multiple points in the three-dimensional model of the object to be detected, which will not be repeated here.

[0099] Exemplarily, taking the object to be detected as a certain face (such as face A) as an example, when the first information includes multiple first calibration points of the three-dimensional model of face A and multiple second calibration points of the three-dimensional model of the standard face, the three-dimensional model feature point detection device can calculate the scale s, rotation matrix R and translation matrix t between the three-dimensional model of face A and the three-dimensional model of the standard face when the rigid registration objective function takes the minimum function value based on the multiple first calibration points of the three-dimensional model of face A and the multiple second calibration points of the three-dimensional model of the standard face by using the rigid registration objective function of the rigid registration algorithm (Iterative Cloest Point, ICP). Afterwards, the three-dimensional model feature point detection device can convert the multiple first points in the three-dimensional model of the standard face according to the scale s, the rotation matrix R, and the translation matrix t to obtain the corresponding multiple second points. Exemplarily, the multiple first points in the three-dimensional model of the standard face can refer to all the points in the three-dimensional model of the standard face, or can also refer to some of the points in the three-dimensional model of the standard face.

[0100] Exemplarily, the rigid registration objective function satisfies the following form:

[0101]

[0102] Among them, x i A second calibration point in the three-dimensional model used to represent a standard face, y iIt is used to represent a first calibration point of the three-dimensional model of face A, and n is used to represent the number of feature point calibrations (i.e., the number of multiple first calibration points or multiple second calibration points). For example, n is an integer greater than or equal to 7.

[0103] Exemplarily, the first registration point satisfies the following form:

[0104] X ′ =sRX+t

[0105] Among them, X ′ It is used to represent the first registration point (ie, the second point), and X is used to represent a point in the three-dimensional model of the standard face.

[0106] After obtaining multiple second points (i.e., multiple first registration points), the three-dimensional model feature point detection device can further optimize the multiple second points by using the non-rigid registration objective function of the non-rigid registration algorithm (Non-rigid Iterative Cloest Point, NICP), and obtain multiple third points when the non-rigid registration objective function takes the minimum function value. Among them, the multiple third points are points obtained after adjusting (or optimizing) the multiple second points respectively by the non-rigid registration objective function. Among them, the three-dimensional model feature point detection device first determines the points in the three-dimensional model of face A that are closest to the multiple second points respectively. Afterwards, the three-dimensional model feature point detection device can calculate the multiple optimized second points when the non-rigid registration objective function takes the minimum function value based on the multiple second points and the points in the three-dimensional model of face A that are closest to the multiple second points respectively, that is, multiple third points, by using the non-rigid registration objective function of NICP. In addition, the three-dimensional model feature point detection device can find the third point closest to the point (i.e., the nearest neighbor point) from multiple third points for each point in the three-dimensional model of face A, and can determine the corresponding relationship between the point and the third point, so that the one-to-one corresponding relationship between the multiple points in the three-dimensional model of the object to be detected and the third points closest to the multiple points can be determined, so that the point registration relationship between the three-dimensional model of the object to be detected and the three-dimensional model of the standard object can be established. Then, the three-dimensional model feature point detection device can establish the point registration relationship between the three-dimensional model of the standard face and the three-dimensional model of face A according to the one-to-one corresponding relationship between the multiple points in the three-dimensional model of face A and the third points closest to the multiple points. Finally, the three-dimensional model feature point detection device can determine the color information of multiple third points from the texture map (such as UV texture map) corresponding to the three-dimensional model of the standard face, and can use the point registration relationship between the three-dimensional model of the standard face and the three-dimensional model of face A to use the color information of a certain third point as the color information of the point corresponding to the third point in the three-dimensional model of face A, so that the color information of multiple points in the three-dimensional model of face A can be obtained. Exemplarily, the texture map corresponding to the three-dimensional model of the standard human face may be pre-stored by the three-dimensional model feature point detection device, or the terminal device may respond to the user's selection operation on the configuration interface for the texture map corresponding to the three-dimensional model of the standard human face, and send the texture map corresponding to the three-dimensional model of the standard human face in the first information to the three-dimensional model feature point detection device, or the three-dimensional model feature point detection device may obtain it from other devices. The embodiments of the present application are not limited to this.

[0107] Exemplarily, the non-rigid registration objective function satisfies the following form:

[0108]

[0109] Wherein, N is used to represent the number of multiple first points (such as all points) in the three-dimensional model of the standard face, and X i ′ A first registration point in the three-dimensional model used to represent a standard face, NN Y (X i ′ ) is used to represent the point set Y in the three-dimensional model of face A and the first registration point X i ′ The closest point.

[0110] It can be seen from the above steps 201 to 205 that after the three-dimensional model feature point detection device obtains the first information of the object to be detected, it can perform texture mapping recognition on the first information, and according to the recognition result, flexibly select (or adaptively select) the corresponding color information determination method to determine the color information corresponding to the three-dimensional model of the object to be detected, so that it can be more targeted and applicable to different color information calculation scenarios, and the calculation of color information can be made more consistent with the actual situation of the first information, so that the feature point detection of the three-dimensional model of the object to be detected can be made more targeted and flexible. Afterwards, the three-dimensional model feature point detection device can render the three-dimensional model of the object to be detected into a two-dimensional image according to the color information corresponding to the determined three-dimensional model of the object to be detected. Then, the three-dimensional model feature point detection device can timely and accurately obtain the three-dimensional feature points in the three-dimensional model of the object to be detected by converting the two-dimensional feature points detected in the two-dimensional image into corresponding three-dimensional feature points. Thus, the method utilizes the advantages of the existing two-dimensional image feature point detection algorithm, and uses the existing relatively mature two-dimensional image feature point detection algorithm to detect the feature points of the two-dimensional image, so as to accurately detect the semantic feature points. At the same time, the two-dimensional image feature point detection algorithm has strong robustness to changes in illumination, posture or expression in the two-dimensional image, and can effectively resist the change of viewing angle and illumination, so as to effectively ensure the accuracy of the three-dimensional model feature point detection, thereby effectively ensuring the detection stability of the three-dimensional feature points of the three-dimensional model of the object to be detected, and effectively solving the problem that the existing traditional three-dimensional model feature point detection algorithm cannot accurately detect the precise semantic feature points. In addition, the method does not have the process of manually calibrating the key feature points (i.e., certain three-dimensional feature points) of the three-dimensional model training sample set, which can effectively save the time cost and manpower cost consumed by manually calibrating the feature points, so as to effectively improve the feature point detection efficiency of the three-dimensional model of the object to be detected, thereby effectively solving the problem that the existing three-dimensional feature point detection algorithm lacks accurate three-dimensional feature point calibration data for training, and at the same time can solve the problem that the existing three-dimensional feature point detection algorithm needs to manually calibrate the key feature points of the three-dimensional model training sample set, resulting in low efficiency of the three-dimensional feature point detection of the three-dimensional model of the object to be detected.

[0111] It should be noted that in the description of the present application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there may be three relationships, for example, A and / or B, which can represent: the situation where A exists alone, A and B exist at the same time, and B exists alone, wherein A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are an "or" relationship. "At least one of the following (individuals)" or its similar expression refers to any combination of these items, including any combination of single items (individuals) or plural items (individuals). For example, "at least one of A, B and C" includes A, B, C, AB, AC, BC or ABC. And, unless otherwise specified, the ordinal numbers such as "first", "second", and "third" mentioned in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the order, timing, priority or importance of multiple objects. In addition, the terms "including", "comprising", "having" and their variations appearing in the present application all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0112] In addition, it should be noted that each step involved in the above embodiments can be performed by a corresponding device, or by a chip, processor, or chip system in the device, and the embodiments of the present application do not limit them. The above embodiments are only described by taking the corresponding device as an example.

[0113] It should be noted that in the above embodiments, some steps can be selected for implementation, and the order of the steps in the diagram can be adjusted for implementation, and this application does not limit this. It should be understood that executing some steps in the diagram, adjusting the order of the steps, or combining them for specific implementation all fall within the scope of protection of this application.

[0114] It is understandable that, in order to implement the functions in the above embodiments, the various devices involved in the above embodiments include hardware structures and / or software modules corresponding to the execution of the various functions. It should be easily appreciated by those skilled in the art that, in combination with the units and method steps of the various examples described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0115] It should be noted that the "steps" in the embodiments of the present application are only for illustration, and are a method of expression used to better understand the embodiments, and do not constitute a substantial limitation on the execution of the scheme of the present application. For example, the "steps" can also be understood as "features". In addition, the steps do not constitute any limitation on the execution order of the scheme of the present application. Any changes in the order of steps, step merging, or step splitting made on this basis that do not affect the implementation of the overall scheme, and the resulting new technical solutions are also within the scope of the disclosure of the present application.

[0116] Based on the same concept, the present application also provides a possible 3D model feature point detection device, which is suitable for Figure 1 In the application scenario shown in FIG. 1 , the three-dimensional model feature point detection device can be Figure 1 The service device 200 shown in the figure (such as a computing device with a 3D model feature point detection function or a computing device cluster with a 3D model feature point detection function, etc.), or it can also be any other device with the functions required to implement the method (such as a vehicle-mounted device or a robot device, etc.), or it can also be Figure 1 The functional element (such as a plug-in, component, chip or circuit, etc.) in the service device 200 shown in the figure, or it can also be a functional element set in any other device, and the functional element has the function of implementing the three-dimensional model feature point detection method. Optionally, the three-dimensional model feature point detection device is used to implement the three-dimensional model feature point detection method provided in the above embodiment, or the module (such as a chip) of the three-dimensional model feature point detection device is used to implement the three-dimensional model feature point detection method provided in the above embodiment, so the beneficial effects of the above embodiment can also be achieved. Exemplarily, taking the three-dimensional model feature point detection device as a chip set in a computing device as an example, when the three-dimensional model feature point detection device is a chip, the three-dimensional model feature point detection device includes a transceiver and a processor, but does not include a memory. Among them, the transceiver exists as an input and output interface, and the input and output interface is used for the chip to realize the transceiver of the computing device. The input and output interface may include an input interface and / or an output interface, the input interface can realize the reception of the computing device, and the output interface can be used to realize the sending of the computing device. The processor is used to read and execute corresponding computer programs or instructions so that the corresponding functions of the three-dimensional model feature point detection device are realized. Optionally, when the chip implements the corresponding functions of the 3D model feature point detection device in the above embodiment, the input and output interface can implement the transceiver operation performed by the 3D model feature point detection device in the above embodiment; the processor can implement other operations except the transceiver operation performed by the 3D model feature point detection device in the above embodiment. For specific related descriptions, please refer to the related descriptions in the above embodiment, which will not be described in detail here.

[0117] See also Figure 4 The 3D model feature point detection device 400 includes an acquisition module 401 (or a transceiver module for sending and receiving data), a determination module 402, a rendering module 403, a feature point detection module 404, and a feature point conversion module 405. Optionally, the 3D model feature point detection device 400 may also include a display module, which is used to display a configuration interface.

[0118] When the 3D model feature point detection device 400 is used to implement Figure 2 The functions of the three-dimensional model feature point detection device in the method embodiment shown are as follows: the acquisition module 401 is used to acquire first information of the object to be detected, wherein the first information may include the three-dimensional model of the object to be detected. The determination module 402 is used to determine the color information corresponding to the three-dimensional model of the object to be detected. The rendering module 403 is used to render the three-dimensional model of the object to be detected into a two-dimensional image according to the color information. The feature point detection module 404 is used to perform feature point detection on the two-dimensional image to obtain multiple two-dimensional feature points. The feature point conversion module 405 is used to convert multiple two-dimensional feature points according to the conversion relationship between the two-dimensional feature points and the three-dimensional feature points to obtain multiple three-dimensional feature points, and the multiple three-dimensional feature points are used as detection feature points of the three-dimensional model of the object to be detected.

[0119] Wherein, when the 3D model feature point detection device 400 is used to implement Figure 2 For a more detailed description of the acquisition module 401, the determination module 402, the rendering module 403, the feature point detection module 404 and the feature point conversion module 405, please refer to the above Figure 2 The relevant description about the three-dimensional model feature point detection device in the method embodiment shown is not repeated here.

[0120] Optional, above Figure 4 The acquisition module, determination module, rendering module, feature point detection module and feature point conversion module shown in the figure can all be implemented by software, or can all be implemented by hardware. Exemplarily, the implementation of the determination module is described below by taking the determination module as an example. Similarly, the implementation of the acquisition module, rendering module, feature point detection module and feature point conversion module can refer to the implementation of the determination module, and will not be repeated here.

[0121] When implemented by software, the determination module is used as an example of a software functional unit, and the determination module may be an application or a code block running on a computer device. Among them, the computer device may be at least one of a physical host, a virtual machine, a container and other computing devices. Furthermore, the above-mentioned computer device may be one or more. For example, the determination module may be an application running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the application may be distributed in the same availability zone (AZ) or in different AZs. The multiple hosts / virtual machines / containers used to run the application may be distributed in the same region or in different regions. Among them, usually a region may include multiple AZs.

[0122] Similarly, multiple hosts / virtual machines / containers for running the application can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Typically, a region can include multiple VPCs, and a VPC can include multiple AZs.

[0123] When implemented by hardware, the determination module is taken as an example of a hardware functional unit, and the determination module may include at least one computing device, such as a server, etc. Alternatively, the determination module may also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0124] The multiple computing devices included in the determination module can be distributed in the same AZ or in different AZs. The multiple computing devices included in the determination module can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the determination module can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0125] It should be noted that the steps that the acquisition module, determination module, rendering module, feature point detection module and feature point conversion module are responsible for implementing can be specified as needed. The full functions of the three-dimensional model feature point detection device can be realized by respectively implementing different execution steps of the three-dimensional model feature point detection method through the acquisition module, determination module, rendering module, feature point detection module and feature point conversion module.

[0126] In addition, it should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. The functional modules in the embodiments of the present application can be integrated into one module, or each module can exist physically separately, or two or more modules can be integrated into one module. For example, the acquisition module, the determination module, the rendering module, the feature point detection module and the feature point conversion module can be integrated into one module, or the acquisition module, the determination module, the rendering module, the feature point detection module and the feature point conversion module are the same module. For another example, the determination module, the rendering module, the feature point detection module and the feature point conversion module can be integrated into one module, or the determination module, the rendering module, the feature point detection module and the feature point conversion module are the same module. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0127] Based on the same concept, the present application embodiment also provides a possible computing device for executing the three-dimensional model feature point detection method shown in the above method embodiment. The relevant features can be found in the above method embodiment and will not be repeated here. Figure 5 As shown, computing device 500 includes: bus 501, processor 502, communication interface 503 and memory 504. Processor 502, communication interface 503 and memory 504 communicate with each other through bus 501. Computing device 500 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in computing device 500.

[0128] The bus 501 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The bus 501 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 501 may include a path for transmitting information between various components of the computing device 500 (for example, the memory 504, the processor 502, and the communication interface 503).

[0129] The processor 502 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0130] The memory 504 may include a volatile memory, such as a random access memory (RAM). The processor 502 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0131] The memory 504 stores executable program codes, and the processor 502 executes the executable program codes to respectively implement the functions of the aforementioned 3D model feature point detection device (such as the acquisition module, the determination module, the rendering module, the feature point detection module, and the feature point conversion module), thereby implementing the 3D model feature point detection method provided in the embodiment of the present application. That is, the memory 504 stores computer program instructions for executing the 3D model feature point detection method.

[0132] Alternatively, the memory 504 stores executable code, and the processor 502 executes the executable code to implement the functions of the aforementioned 3D model feature point detection device (such as the acquisition module, the determination module, the rendering module, the feature point detection module, and the feature point conversion module), thereby implementing the 3D model feature point detection method provided in the embodiment of the present application. That is, the memory 504 stores computer program instructions for the 3D model feature point detection device to execute the 3D model feature point detection method provided in the embodiment of the present application.

[0133] The communication interface 503 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 500 and other devices or a communication network.

[0134] Based on the same concept, the embodiment of the present application also provides a possible computing device cluster for executing the method shown in the above method embodiment. The relevant features can be found in the above method embodiment, which will not be repeated here. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop, a laptop, or a smart phone.

[0135] like Figure 6 As shown, the computing device cluster 600 includes at least one computing device 500. The memory 504 in one or more computing devices 500 in the computing device cluster 600 may store the same instructions for executing the three-dimensional model feature point detection method provided in the embodiment of the present application.

[0136] In some possible implementations, the memory 504 of one or more computing devices 500 in the computing device cluster 600 may also respectively store some instructions for executing the 3D model feature point detection method. In other words, the combination of one or more computing devices 500 may jointly execute the instructions for executing the 3D model feature point detection method.

[0137] It should be noted that the memory 504 in different computing devices 500 in the computing device cluster 600 can store different instructions, which are respectively used to execute part of the functions of the three-dimensional model feature point detection device. That is, the instructions stored in the memory 504 in different computing devices 500 can realize the functions of one or more modules among the acquisition module, the determination module, the rendering module, the feature point detection module and the feature point conversion module.

[0138] In some possible implementations, one or more computing devices in the computing device cluster 600 may be connected via a network, which may be a wide area network or a local area network. Figure 7 A possible implementation is shown. Figure 7As shown, two computing devices 500A and 500B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 504 in the computing device 500A stores instructions for executing the functions of the acquisition module 401, the determination module 402, and the rendering module 403. At the same time, the memory 504 in the computing device 500B stores instructions for executing the functions of the feature point detection module 404 and the feature point conversion module 405. In some possible implementations, the memory 504 of one or more computing devices 500 in the computing device cluster may also store partial instructions for executing the three-dimensional model feature point detection method. In other words, a combination of one or more computing devices 500 can jointly execute instructions for executing the three-dimensional model feature point detection method.

[0139] It should be understood that Figure 7 The functions of the computing device 500A shown in FIG. 5A may also be completed by multiple computing devices 500. Similarly, the functions of the computing device 500B may also be completed by multiple computing devices 500.

[0140] Based on the same concept, the embodiment of the present application also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the three-dimensional model feature point detection method provided in the embodiment of the present application.

[0141] Based on the same concept, an embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the three-dimensional model feature point detection method provided in the embodiment of the present application.

[0142] Based on the same concept, an embodiment of the present application further provides a computer chip, which is coupled to a memory and is used to read a computer program stored in the memory to execute the three-dimensional model feature point detection method provided in the embodiment of the present application.

[0143] Based on the same concept, an embodiment of the present application also provides a chip system, which includes a processor for supporting a computer device to implement the three-dimensional model feature point detection method provided in an embodiment of the present application. In a possible design, the chip system also includes a memory, which is used to store the necessary programs and data of the computer device. The chip system can be composed of a computer chip, or it can include a computer chip and other discrete devices.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting feature points of a three-dimensional model, characterized in that: include: Acquire first information of an object to be detected, wherein the first information includes a three-dimensional model of the object to be detected; Determining color information corresponding to the three-dimensional model of the object to be detected; Rendering the three-dimensional model of the object to be detected into a two-dimensional image according to the color information; Performing feature point detection on the two-dimensional image to obtain a plurality of two-dimensional feature points; According to the conversion relationship between the two-dimensional feature points and the three-dimensional feature points, the multiple two-dimensional feature points are converted to obtain multiple three-dimensional feature points, and the multiple three-dimensional feature points are used as detection feature points of the three-dimensional model of the object to be detected.

2. The method according to claim 1, characterized in that Determining color information corresponding to the three-dimensional model of the object to be detected includes: Acquire a texture map corresponding to the three-dimensional model of the object to be detected; and determine color information of multiple points in the three-dimensional model of the object to be detected based on the texture map corresponding to the three-dimensional model of the object to be detected.

3. The method according to claim 1, characterized in that Determining color information corresponding to the three-dimensional model of the object to be detected includes: Parsing the three-dimensional model of the object to be detected to obtain attribute information of the three-dimensional model of the object to be detected; According to the attribute information, color information corresponding to the three-dimensional model of the object to be detected is determined.

4. The method according to claim 3, characterized in that Determining color information corresponding to the three-dimensional model of the object to be detected according to the attribute information includes: When the attribute information includes color information of a plurality of points in the three-dimensional model of the object to be detected, the color information of the plurality of points is used as the color information corresponding to the three-dimensional model of the object to be detected.

5. The method according to claim 3, characterized in that Determining color information corresponding to the three-dimensional model of the object to be detected according to the attribute information includes: When the attribute information does not include color information of a plurality of points in the three-dimensional model of the object to be detected, establishing a point registration relationship between the three-dimensional model of the object to be detected and the three-dimensional model of a standard object; Color information corresponding to the three-dimensional model of the object to be detected is determined according to the point registration relationship and the texture map corresponding to the three-dimensional model of the standard object, wherein the three-dimensional model of the standard object is used as a reference template for the three-dimensional model of the object to be detected.

6. The method according to claim 5, characterized in that Establishing a point registration relationship between the three-dimensional model of the object to be detected and the three-dimensional model of the standard object includes: Displaying information of a plurality of second calibration points of the standard object; Acquire a plurality of first calibration point information of the three-dimensional model of the object to be detected; According to the multiple first calibration points of the three-dimensional model of the object to be detected and the multiple second calibration points of the three-dimensional model of the standard object, the three-dimensional model of the standard object is rigidly transformed using a rigid registration objective function to determine multiple first registration points in the three-dimensional model of the standard object, wherein the multiple first calibration points are obtained by a user performing one-to-one calibration on the three-dimensional model of the object to be detected with respect to the multiple second calibration points; According to the multiple first registration points in the three-dimensional model of the standard object, a non-rigid transformation is performed on the multiple first registration points in the three-dimensional model of the standard object using a non-rigid registration objective function to determine multiple second registration points in the three-dimensional model of the standard object, wherein the multiple second registration points are used as points having a registration relationship with multiple points in the three-dimensional model of the object to be detected; The point registration relationship is determined according to a one-to-one correspondence between the plurality of second registration points and the plurality of points.

7. The method according to claim 6, characterized in that Determining color information corresponding to the three-dimensional model of the object to be detected according to the point registration relationship and the texture map corresponding to the three-dimensional model of the standard object includes: Determining color information of the plurality of second registration points from a texture map corresponding to the three-dimensional model of the standard object; According to the point registration relationship, using the color information of the plurality of second registration points as the color information of the plurality of points corresponding to the three-dimensional model of the object to be detected; The color information of the multiple points is used as the color information corresponding to the three-dimensional model of the object to be detected.

8. The method according to any one of claims 1 to 7, characterized in that: The first information is obtained in response to a user editing operation on the information of the object to be detected in a configuration interface.

9. A three-dimensional model feature point detection device, characterized in that: include: An acquisition module, used for acquiring first information of the object to be detected, wherein the first information includes a three-dimensional model of the object to be detected; A determination module, used to determine color information corresponding to the three-dimensional model of the object to be detected; A rendering module, used for rendering the three-dimensional model of the object to be detected into a two-dimensional image according to the color information; A feature point detection module, used to perform feature point detection on the two-dimensional image to obtain a plurality of two-dimensional feature points; The feature point conversion module is used to convert the multiple two-dimensional feature points according to the conversion relationship between the two-dimensional feature points and the three-dimensional feature points to obtain multiple three-dimensional feature points, and the multiple three-dimensional feature points are used as detection feature points of the three-dimensional model of the object to be detected.

10. The device according to claim 9, characterized in that When determining the color information corresponding to the three-dimensional model of the object to be detected, the determination module is specifically used to: Acquire a texture map corresponding to the three-dimensional model of the object to be detected; and determine color information of multiple points in the three-dimensional model of the object to be detected based on the texture map corresponding to the three-dimensional model of the object to be detected.

11. The device according to claim 9, characterized in that When determining the color information corresponding to the three-dimensional model of the object to be detected, the determination module is specifically used to: Parsing the three-dimensional model of the object to be detected to obtain attribute information of the three-dimensional model of the object to be detected; According to the attribute information, color information corresponding to the three-dimensional model of the object to be detected is determined.

12. The device according to claim 11, characterized in that When the determination module determines the color information corresponding to the three-dimensional model of the object to be detected according to the attribute information, it is specifically used to: When the attribute information includes color information of a plurality of points in the three-dimensional model of the object to be detected, the color information of the plurality of points is used as the color information corresponding to the three-dimensional model of the object to be detected.

13. The device according to claim 11, characterized in that When the determination module determines the color information corresponding to the three-dimensional model of the object to be detected according to the attribute information, it is specifically used to: When the attribute information does not include color information of a plurality of points in the three-dimensional model of the object to be detected, establishing a point registration relationship between the three-dimensional model of the object to be detected and the three-dimensional model of a standard object; The color information corresponding to the three-dimensional model of the object to be detected is determined according to the point registration relationship and the texture map corresponding to the three-dimensional model of the standard object, wherein the three-dimensional model of the standard object is used as a reference template for the three-dimensional model of the object to be detected.

14. The device according to claim 13, characterized in that When establishing the point registration relationship between the three-dimensional model of the object to be detected and the three-dimensional model of the standard object, the determination module is specifically used to: Displaying information of a plurality of second calibration points of the standard object; Acquire information of a plurality of first calibration points of the three-dimensional model of the object to be detected, wherein the plurality of first calibration points are obtained by a user performing one-to-one calibration on the three-dimensional model of the object to be detected with respect to the plurality of second calibration points; According to the plurality of first calibration points of the three-dimensional model of the object to be detected and the plurality of second calibration points of the three-dimensional model of the standard object, the three-dimensional model of the standard object is rigidly transformed using a rigid registration objective function to determine a plurality of first registration points in the three-dimensional model of the standard object; According to the multiple first registration points in the three-dimensional model of the standard object, a non-rigid transformation is performed on the multiple first registration points in the three-dimensional model of the standard object using a non-rigid registration objective function to determine multiple second registration points in the three-dimensional model of the standard object, wherein the multiple second registration points are used as points having a registration relationship with multiple points in the three-dimensional model of the object to be detected; The point registration relationship is determined according to a one-to-one correspondence between the plurality of second registration points and the plurality of points.

15. The device according to claim 14, characterized in that When the determination module determines the color information corresponding to the three-dimensional model of the object to be detected according to the point registration relationship and the texture map corresponding to the three-dimensional model of the standard object, it is specifically used to: Determining color information of the plurality of second registration points from a texture map corresponding to the three-dimensional model of the standard object; According to the point registration relationship, using the color information of the plurality of second registration points as the color information of the plurality of points corresponding to the three-dimensional model of the object to be detected; The color information of the multiple points is used as the color information corresponding to the three-dimensional model of the object to be detected.

16. The device according to any one of claims 9 to 15, characterized in that: The first information is acquired by the acquisition module in response to a user's editing operation on the information of the object to be detected in a configuration interface.

17. A computing device, characterized in that The computing device includes a communication interface, a processor, and a memory; The communication interface is used to receive and send data; The memory is used to store computer program instructions; The processor calls the computer program instructions stored in the memory to execute the method according to any one of claims 1 to 8.

18. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, each computing device includes a processor and a memory; the memory in the at least one computing device is used to store computer program instructions; The processor in the at least one computing device calls the computer program instructions stored in the memory to execute the method according to any one of claims 1 to 8.

19. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device, the computing device performs the method according to any one of claims 1 to 8.

20. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device, the computing device is caused to perform the method as claimed in any one of claims 1 to 8.

Images