Object model rotation method and related device thereof
By performing unbiased Schmitt orthogonalization of the target object information, the object model rotation is directly used to rotate the n-order rotation matrix, which solves the problem of poor rotation estimation performance in the prior art, and achieves accurate object model rotation and model performance promote.
Patent Information
- Application Number
- PCT/CN2024/092219
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-12
- Filing Date
- 2024-05-10
- Publication Date
- 2024-11-21
AI Technical Summary
In the rotation estimation of the prior art, since the neural network model output is a 3×2 matrix, it is necessary to pass Schmitt orthogonalization to obtain a 3×3 rotation matrix, resulting in poor model training performance and the object model cannot be accurately completed. Rotate.
By obtaining the information of the target object, input it into the target model for processing, obtaining the n-order matrix and performing unbiased Schmidt orthogonalization, obtaining the n-order rotation matrix, and directly using this matrix to rotate the object model to avoid orthogonalization processing Problems during model training.
The performance of the target model is improved, the rotation estimation and rotation of the object model can be accurately completed, and problems such as gradient reverse and gradient explosion caused by orthogonalization are avoided, which improves the training efficiency and accuracy of the model.
Smart Images

Figure CN2024092219_21112024_PF_FP_ABST
Abstract
Description
Object model rotation method and related equipment
[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on May 12, 2023, with application number 202310540964.4 and invention name “A method for rotating an object model and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The embodiments of the present application relate to the field of artificial intelligence (AI), and in particular to a method for rotating an object model and related equipment. Background Art
[0003] Among various computer vision tasks, rotation estimation is one of the most important and difficult parts. In rotation estimation, a rotation matrix is usually used to describe the rotation change between the current state of an object and the initial state of the object.
[0004] In related technologies, after obtaining information associated with an object, this information can be input into a neural network model to be processed by the neural network model to obtain a 3×2 matrix. This matrix is then subjected to a gram-schmidt orthogonalization (GS) process to obtain a 3×3 rotation matrix. In this way, the rotation matrix can be used to rotate a three-dimensional (3D) model of the object (representing the object in its initial state) to obtain a rotated 3D model of the object (representing the object in its current state).
[0005] During the training of the aforementioned neural network model, the model output is a 3×2 matrix, which requires GS processing to obtain a 3×3 rotation matrix, which is then used to train the neural network model. However, GS processing is detrimental to the model training process and can result in poor performance of the trained neural network model. Specifically, the neural network model cannot accurately estimate rotations, and thus cannot accurately rotate the object model.
[0006] Summary of the Invention
[0007] The embodiments of the present application provide an object model rotation method and related equipment, which provide a target model that can accurately complete rotation estimation and thus accurately complete the rotation of the object model.
[0008] A first aspect of an embodiment of the present application provides a method for rotating an object model, the method comprising:
[0009] When it is necessary to rotate the object model of the target object, information of the target object in a first state (eg, a current state or a real-time state, etc.) may be obtained first.
[0010] After obtaining the target object information in the first state, the target object information in the first state can be input into the target model, whereupon the target model performs a series of processing on the target object information in the first state, thereby obtaining a first matrix. It should be noted that the first matrix output by the target model is an n-order matrix, that is, a matrix with n rows and n columns.
[0011] Since the first matrix is a pseudo-rotation matrix, not a true rotation matrix, directly using the first matrix to rotate the n-dimensional model of the target object will cause the model to deform. Therefore, an unbiased Gram-Schmidt orthogonalization can be performed on the first matrix to obtain the second matrix. It should be noted that the second matrix is an n-order rotation matrix, that is, a rotation matrix with n rows and n columns.
[0012] After obtaining the second matrix, the preset n-dimensional model of the target object can be directly rotated using the second matrix to obtain the rotated n-dimensional model of the target object. It should be noted that the preset n-dimensional model of the target object is used to indicate the target object in the second state (e.g., the preset state or the initial state), while the rotated n-dimensional model of the target object is used to indicate the target object in the first state. At this point, the object model rotation of the target object is completed.
[0013] The above method demonstrates that when rotating the target object model, information about the target object in its first state can be obtained and input into the target model. Next, the target model processes this information to obtain a first matrix, which is an n-order matrix, where n is a positive integer greater than or equal to 2. The first matrix can then be orthogonalized to obtain a second matrix, which is an n-order rotation matrix. Finally, the pre-set n-dimensional model of the target object can be directly rotated based on the second matrix to obtain a rotated n-dimensional model of the target object. The pre-set n-dimensional model of the target object is used to indicate the target object in its second state, and the rotated n-dimensional model of the target object is used to indicate the target object in its first state. In the above process, the output of the target model is an n×n matrix, which can be considered a pseudo-rotation matrix. Therefore, this matrix can be used directly to complete the training of the target model without undergoing orthogonalization. It can be seen that orthogonalization processing will not occur during the training process of the target model, which can avoid various problems caused by orthogonalization processing during the model training process. Training the target model in this way is conducive to improving the performance of the target model, that is, the target model can accurately complete the rotation estimation, and then accurately complete the object model rotation for the target object.
[0014] In one possible implementation, a first matrix includes n first column vectors, and a second matrix includes n second column vectors. Orthogonalizing the first matrix to obtain a second matrix corresponding to the target object includes: performing a first calculation on the first to nth first column vectors to obtain the first second column vector; performing a second calculation on the first second column vector to the i-1th second column vector, and the i-th first column vector to the n-th first column vector to obtain the i-th second column vector, where i=2, ...n-1; and performing a third calculation on the first to n-1th second column vectors to obtain the n-th second column vector. In the aforementioned implementation, after obtaining the n first column vectors of the first matrix, a first calculation can be performed on the first to nth first column vectors to obtain the first second column vector. After obtaining the first second column vector, a second calculation can be performed on the first second column vector and the second to nth first column vectors to obtain the second second column vector. After obtaining the second second column vector, a second calculation can be performed on the first second column vector to the second second column vector, and on the third first column vector to the nth first column vector to obtain the third second column vector, ..., after obtaining the n-2th second column vector, a second calculation can be performed on the first second column vector to the n-2th second column vector, and on the n-1th first column vector to the nth first column vector to obtain the n-1th second column vector. Finally, since the first second column vector to the n-1th second column vector have been obtained, a third calculation can be performed on the first second column vector to the n-1th second column vector to obtain the nth second column vector. It can be seen that according to the unbiased Schmidt orthogonalization method provided in the application embodiment, the first matrix, which is an n-order ordinary matrix, can be successfully converted into a second matrix, which is an n-order rotation matrix.
[0015] In one possible implementation, after obtaining n first column vectors, a cross product may be performed on the second first column vector to the nth first column vector to obtain the first third column vector. After obtaining the first third column vector, a weighted average may be performed on the first third column vector and the first first column vector to obtain the first fourth column vector. After obtaining the first fourth column vector, the first fourth column vector may be normalized to obtain the first second column vector.
[0016] In one possible implementation, after obtaining the i-1th second column vector, a cross product may be performed on the 1st second column vector to the i-1th second column vector, and on the i+1th first column vector to the nth first column vector, thereby obtaining the i-th third column vector. After obtaining the i-th third column vector, the i-th third column vector and the i-th first column vector may be averaged to obtain the i-th fourth column vector. After obtaining the i-th fourth column vector, the i-th fourth column vector may be projected onto the 1st second column vector to the i-1th second column vector, and the projection results may be added together to obtain the i-th fifth column vector. After obtaining the i-th fifth column vector, the i-th fourth column vector may be subtracted from the i-th fifth column vector to obtain the i-th sixth column vector. After obtaining the i-th sixth column vector, the i-th sixth column vector may be normalized to obtain the i-th second column vector. Since i=2, ..., i-1, the aforementioned process is repeated continuously to eventually obtain the first second column vector to the n-1th second column vector.
[0017] In one possible implementation, since the 1st second column vector to the n-1th second column vector have been obtained, cross product may be performed on the 1st second column vector to the n-1th second column vector to obtain the nth second column vector.
[0018] In one possible implementation, the n second column vectors satisfy at least one of the following conditions: the n second column vectors are mutually orthogonal; the modules of the n second column vectors are all one; and the n second column vectors constitute an n-dimensional coordinate system.
[0019] In one possible implementation, the information about the target object in the first state includes two categories of information: the first category of information is information about the target object in the first state, and the second category of information is information collected by the target object in the first state. The first category of information may refer to n-dimensional data about the target object in the first state, and the second category of information may refer to n-dimensional data collected by the target object in the first state.
[0020] A second aspect of an embodiment of the present application provides a model training method, which includes: obtaining information of a target object in a first state; processing the information through a target model to obtain a first matrix, the first matrix being an n-order matrix, n being a positive integer greater than or equal to 2, the first matrix being used to obtain a rotated n-dimensional model of the target object, and the rotated n-dimensional model being used to indicate the target object in the first state; obtaining a target loss based on the first matrix; and training a model to be trained based on the target loss to obtain a target model.
[0021] The target model trained by the above method has a rotation estimation function. Specifically, when it is necessary to rotate the object model of the target object, the information of the target object in the first state can be obtained first and input into the target model. Then, the information can be processed by the target model to obtain a first matrix, which is an n-order matrix, where n is a positive integer greater than or equal to 2. Then, the first matrix can be orthogonalized to obtain a second matrix, which is an n-order rotation matrix. Finally, the preset n-dimensional model of the target object can be directly rotated based on the second matrix to obtain the rotated n-dimensional model of the target object. The preset n-dimensional model of the target object is used to indicate the target object in the second state, and the rotated n-dimensional model of the target object is used to indicate the target object in the first state. In the above process, the output of the target model is an n×n matrix, which can be regarded as a pseudo-rotation matrix. Therefore, the matrix can be used directly to complete the training of the target model without undergoing orthogonalization. It can be seen that orthogonalization processing will not occur during the training process of the target model, which can avoid various problems caused by orthogonalization processing during the model training process. Training the target model in this way is conducive to improving the performance of the target model, that is, the target model can accurately complete the rotation estimation, and then accurately complete the object model rotation for the target object.
[0022] In one possible implementation, based on the first matrix, obtaining the target loss includes: directly using the first matrix to rotate the preset n-dimensional model of the target object to obtain a rotated n-dimensional model of the target object, and the preset n-dimensional model is used to indicate the target object in the second state; based on the rotated n-dimensional model and the rotated true n-dimensional model of the target object, obtaining the target loss, the rotated true n-dimensional model is used to indicate the target object in the first state, and the target loss is used to indicate the difference between the rotated n-dimensional model and the rotated true n-dimensional model.
[0023] In one possible implementation, obtaining the target loss based on the first matrix includes: obtaining the target loss based on the first matrix and a true matrix, where the target loss is used to indicate the difference between the first matrix and the true matrix, and the true matrix is an n-order matrix.
[0024] In a possible implementation, the information includes at least one of the following: n-dimensional data of the target object in the first state and n-dimensional data collected from the target object in the first state.
[0025] A third aspect of an embodiment of the present application provides an object model rotation device, which includes: an acquisition module for acquiring information of a target object in a first state; a processing module for processing the information through a target model to obtain a first matrix, the first matrix being an n-order matrix, and n being a positive integer greater than or equal to 2; an orthogonalization module for orthogonalizing the first matrix to obtain a second matrix, the second matrix being an n-order rotation matrix; a rotation module for rotating a preset n-dimensional model of the target object based on the second matrix to obtain a rotated n-dimensional model of the target object, the preset n-dimensional model being used to indicate the target object in the second state, and the rotated n-dimensional model being used to indicate the target object in the first state.
[0026] It can be seen from the above device that when it is necessary to rotate the object model of the target object, the information of the target object in the first state can be obtained first, and this information can be input into the target model. Then, the information can be processed by the target model to obtain a first matrix, which is an n-order matrix, and n is a positive integer greater than or equal to 2. Then, the first matrix can be orthogonalized to obtain a second matrix, which is an n-order rotation matrix. Finally, the preset n-dimensional model of the target object can be directly rotated based on the second matrix to obtain the rotated n-dimensional model of the target object. The preset n-dimensional model of the target object is used to indicate the target object in the second state, and the rotated n-dimensional model of the target object is used to indicate the target object in the first state. In the above process, the output of the target model is an n×n matrix, which can be regarded as a pseudo-rotation matrix. Therefore, the matrix can be used directly to complete the training of the target model without undergoing orthogonalization. It can be seen that orthogonalization processing will not occur during the training process of the target model, which can avoid various problems caused by orthogonalization processing during the model training process. Training the target model in this way is conducive to improving the performance of the target model, that is, the target model can accurately complete the rotation estimation, and then accurately complete the object model rotation for the target object.
[0027] In one possible implementation, the first matrix contains n first column vectors, the second matrix contains n second column vectors, and the orthogonalization module is used to: perform a first calculation on the 1st first column vector to the nth first column vector to obtain the 1st second column vector; perform a second calculation on the 1st second column vector to the i-1th second column vector, and on the ith first column vector to the nth first column vector to obtain the ith second column vector, i=2,...n-1; perform a third calculation on the 1st second column vector to the n-1th second column vector to obtain the nth second column vector.
[0028] In one possible implementation, the orthogonalization module is used to: perform a cross product from the second first column vector to the nth first column vector to obtain the first third column vector; perform a weighted average on the first third column vector and the first first column vector to obtain the first fourth column vector; and normalize the first fourth column vector to obtain the first second column vector.
[0029] In one possible implementation, the orthogonalization module is used to: perform cross multiplication on the 1st second column vector to the i-1th second column vector, and the i+1th first column vector to the nth first column vector, to obtain the i-th third column vector; average the i-th third column vector and the i-th first column vector to obtain the i-th fourth column vector; project the i-th fourth column vector to the 1st second column vector to the i-1th second column vector, and add the projection results to obtain the i-th fifth column vector; subtract the i-th fourth column vector from the i-th fifth column vector to obtain the i-th sixth column vector; and normalize the i-th sixth column vector to obtain the i-th second column vector.
[0030] In a possible implementation, the orthogonalization module is configured to perform a cross product on the 1st second column vector to the n-1th second column vector to obtain the nth second column vector.
[0031] In one possible implementation, the n second column vectors satisfy at least one of the following conditions: the n second column vectors are mutually orthogonal; the modules of the n second column vectors are all one; and the n second column vectors constitute an n-dimensional coordinate system.
[0032] In a possible implementation, the information includes at least one of the following: n-dimensional data of the target object in the first state and n-dimensional data collected from the target object in the first state.
[0033] A fourth aspect of an embodiment of the present application provides a model training device, which includes: a first acquisition module for acquiring information of a target object in a first state; a processing module for processing the information through a target model to obtain a first matrix, the first matrix being an n-order matrix, n being a positive integer greater than or equal to 2, the first matrix being used to acquire a rotated n-dimensional model of the target object, and the rotated n-dimensional model being used to indicate the target object in the first state; a second acquisition module for acquiring a target loss based on the first matrix; and a training module for training a model to be trained based on the target loss to obtain a target model.
[0034] The target model obtained by training the above-mentioned device has a rotation estimation function. Specifically, when it is necessary to rotate the object model of the target object, the information of the target object in the first state can be obtained first, and this information can be input into the target model. Then, the information can be processed by the target model to obtain a first matrix, which is an n-order matrix, and n is a positive integer greater than or equal to 2. Then, the first matrix can be orthogonalized to obtain a second matrix, which is an n-order rotation matrix. Finally, the preset n-dimensional model of the target object can be directly rotated based on the second matrix to obtain the rotated n-dimensional model of the target object. The preset n-dimensional model of the target object is used to indicate the target object in the second state, and the rotated n-dimensional model of the target object is used to indicate the target object in the first state. In the above process, the output of the target model is an n×n matrix, which can be regarded as a pseudo-rotation matrix. Therefore, the matrix can be used directly to complete the training of the target model without undergoing orthogonalization. It can be seen that orthogonalization processing will not occur during the training process of the target model, which can avoid various problems caused by orthogonalization processing during the model training process. Training the target model in this way is conducive to improving the performance of the target model, that is, the target model can accurately complete the rotation estimation, and then accurately complete the object model rotation for the target object.
[0035] In one possible implementation, the second acquisition module is used to: directly use the first matrix to rotate the preset n-dimensional model of the target object to obtain the rotated n-dimensional model of the target object, and the preset n-dimensional model is used to indicate the target object in the second state; based on the rotated n-dimensional model and the rotated true n-dimensional model of the target object, obtain the target loss, and the rotated true n-dimensional model is used to indicate the target object in the first state, and the target loss is used to indicate the difference between the rotated n-dimensional model and the rotated true n-dimensional model.
[0036] In one possible implementation, the second acquisition module is used to acquire a target loss based on the first matrix and the true matrix, where the target loss is used to indicate the difference between the first matrix and the true matrix, and the true matrix is an n-order matrix.
[0037] In a possible implementation, the information includes at least one of the following: n-dimensional data of the target object in the first state and n-dimensional data collected from the target object in the first state.
[0038] A fifth aspect of an embodiment of the present application provides an object model rotation device, which includes a memory and a processor; the memory stores code, and the processor is configured to execute the code. When the code is executed, the object model rotation device executes the method described in the first aspect or any possible implementation method of the first aspect.
[0039] A sixth aspect of an embodiment of the present application provides a model training device, which includes a memory and a processor; the memory stores code, and the processor is configured to execute the code. When the code is executed, the model training device executes the method described in the second aspect or any possible implementation method of the second aspect.
[0040] A seventh aspect of an embodiment of the present application provides a circuit system, which includes a processing circuit, and the processing circuit is configured to execute the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.
[0041] An eighth aspect of an embodiment of the present application provides a chip system, which includes a processor for calling a computer program or computer instructions stored in a memory so that the processor executes a method as described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.
[0042] In a possible implementation, the processor is coupled to the memory through an interface.
[0043] In one possible implementation, the chip system further includes a memory in which computer programs or computer instructions are stored.
[0044] A ninth aspect of an embodiment of the present application provides a computer storage medium storing a computer program, which, when executed by a computer, enables the computer to implement the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.
[0045] A tenth aspect of an embodiment of the present application provides a computer program product, which stores instructions that, when executed by a computer, enable the computer to implement the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.
[0046] In an embodiment of the present application, when it is necessary to rotate the object model of the target object, the information of the target object in the first state can be obtained first, and the information can be input into the target model. Then, the information can be processed by the target model to obtain a first matrix, which is an n-order matrix, and n is a positive integer greater than or equal to 2. Then, the first matrix can be orthogonalized to obtain a second matrix, which is an n-order rotation matrix. Finally, the preset n-dimensional model of the target object can be rotated directly based on the second matrix to obtain the rotated n-dimensional model of the target object. The preset n-dimensional model of the target object is used to indicate the target object in the second state, and the rotated n-dimensional model of the target object is used to indicate the target object in the first state. In the above process, the output of the target model is an n×n matrix, which can be regarded as a pseudo-rotation matrix, so the matrix can be used directly to complete the training of the target model without undergoing orthogonalization. It can be seen that orthogonalization processing will not occur during the training process of the target model, which can avoid various problems caused by orthogonalization processing during the model training process. Training the target model in this way is conducive to improving the performance of the target model, that is, the target model can accurately complete the rotation estimation, and then accurately complete the object model rotation for the target object. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 is a schematic diagram of the structure of the artificial intelligence main framework;
[0048] FIG2a is a schematic structural diagram of an object model rotation system provided in an embodiment of the present application;
[0049] FIG2 b is another structural diagram of the object model rotation system provided in an embodiment of the present application;
[0050] FIG2c is a schematic diagram of a device related to object model rotation provided in an embodiment of the present application;
[0051] FIG3 is a schematic diagram of the architecture of the system 100 provided in an embodiment of the present application;
[0052] FIG4 is a flow chart of a method for rotating an object model provided in an embodiment of the present application;
[0053] FIG5 is a schematic diagram of an application example of the object model rotation method provided in an embodiment of the present application;
[0054] FIG6 is a flow chart of a model training method provided in an embodiment of the present application;
[0055] FIG7 is a schematic diagram of an application example of the model training method provided in an embodiment of the present application;
[0056] FIG8 is a schematic diagram of a comparison result provided by an embodiment of the present application;
[0057] FIG9 is another schematic diagram of the comparison results provided by an embodiment of the present application;
[0058] FIG10 is a schematic structural diagram of an object model rotation device provided in an embodiment of the present application;
[0059] FIG11 is a schematic structural diagram of a model training device provided in an embodiment of the present application;
[0060] FIG12 is a schematic diagram of the structure of an execution device provided in an embodiment of the present application;
[0061] FIG13 is a schematic diagram of the structure of a training device provided in an embodiment of the present application;
[0062] FIG14 is a schematic structural diagram of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION
[0063] The embodiments of the present application provide an object model rotation method and related equipment, which provide a target model that can accurately complete rotation estimation and thus accurately complete the rotation of the object model.
[0064] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.
[0065] Among various computer vision tasks, rotation estimation is one of the most important and challenging. In rotation estimation, a rotation matrix is typically used to describe the rotational change between an object's current state and its initial state. For example, in 3D human motion capture, a rotation matrix can be used to describe the rotational change between a 3D model of a person and a 3D model of a standard human body.
[0066] In related technologies, after obtaining information associated with an object, this information can be input into a neural network model for processing, resulting in a 3×2 matrix. This matrix is then subjected to GS processing, thereby restoring the 3×2 matrix into a 3×3 rotation matrix, which is the rotation matrix corresponding to the object. In this way, the rotation matrix can be used to rotate the 3D model of the object, thereby obtaining the rotated 3D model of the object.
[0067] During the training of the aforementioned neural network model, since the model output is a 3×2 matrix, it needs to undergo GS processing to obtain a 3×3 rotation matrix, which is then used to complete the training of the neural network model. However, GS processing is detrimental to the model training process (for example, GS processing can cause various problems such as gradient reversal, gradient numerical errors, and gradient explosion), resulting in poor performance of the trained neural network model. Specifically, the neural network model cannot accurately complete rotation estimation, and therefore cannot accurately complete the rotation of the object model.
[0068] In order to solve the above problems, an embodiment of the present application provides an object model rotation method, which can be implemented in combination with artificial intelligence (AI) technology. AI technology is a technical discipline that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence. AI technology obtains the best results by sensing the environment, acquiring knowledge and using knowledge. In other words, artificial intelligence technology is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Using artificial intelligence for data processing is a common application of artificial intelligence.
[0069] First, we will describe the overall workflow of an AI system. See Figure 1, which illustrates a schematic diagram of the AI framework. This framework will be explained from two perspectives: the "intelligent information chain" (horizontal axis) and the "IT value chain" (vertical axis). The "intelligent information chain" reflects the entire process from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. Throughout this process, data undergoes a condensed journey from "data-information-knowledge-wisdom." The "IT value chain," spanning the underlying infrastructure of human intelligence, information (provided and processed by technology), and the system's industrial ecosystem, reflects the value that AI brings to the information technology industry.
[0070] (1) Infrastructure
[0071] Infrastructure provides computing power for AI systems, enabling communication with the outside world and supporting this through a foundational platform. External communication occurs through sensors; computing power is provided by intelligent chips (CPUs, NPUs, GPUs, ASICs, FPGAs, and other hardware accelerators). The foundational platform includes a distributed computing framework and network-related platform guarantees and support, including cloud storage and computing, and interconnected networks. For example, sensors communicate with the outside world to acquire data, which is then fed into the intelligent chips within the distributed computing system provided by the foundational platform for computation.
[0072] (2) Data
[0073] Data above the infrastructure layer represents data sources for AI. This data includes graphics, images, voice, and text, as well as IoT data from traditional devices. This includes business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.
[0074] (3) Data processing
[0075] Data processing generally includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.
[0076] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.
[0077] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.
[0078] Decision-making refers to the process of making decisions after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.
[0079] (4) General ability
[0080] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.
[0081] (5) Smart products and industry applications
[0082] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart terminals, smart transportation, smart medical care, autonomous driving, smart cities, etc.
[0083] Next, several application scenarios of this application are introduced.
[0084] Figure 2a is a schematic diagram of the structure of an object model rotation system provided in an embodiment of the present application. The object model rotation system includes a user device and a data processing device. The user device includes an intelligent terminal such as a mobile phone, personal computer, or information processing center. The user device is the initiator of the object model rotation request, typically initiated by a user through the user device.
[0085] The data processing device can be a device or server with data processing capabilities, such as a cloud server, network server, application server, or management server. The data processing device receives object model rotation requests from smart terminals via an interactive interface. It then uses its data storage and data processing processor to perform machine learning, deep learning, search, reasoning, and decision-making to perform object model rotation. The memory in the data processing device is a general term that includes local storage and a database that stores historical data. The database can be located on the data processing device or on other network servers.
[0086] In the object model rotation system shown in FIG2a, a user device can receive a user's instruction. For example, the user device can obtain information associated with the target object input / selected by the user, and then initiate a request to a data processing device, causing the data processing device to execute an object model rotation application for the information associated with the target object from the user device, thereby obtaining a rotated model of the target object. For example, the user device can obtain information associated with the target object input by the user (that is, information of the target object in the first state), and then the user device can initiate an object model rotation request to the data processing device, causing the data processing device to perform a series of processing on the information associated with the target object based on the object model rotation request, so as to complete the required rotation of the preset n-dimensional model of the target object (for indicating the target object in the second state) based on the information processing result, thereby obtaining a rotated n-dimensional model of the target object (for indicating the target object in the first state).
[0087] In FIG. 2 a , a data processing device may execute the object model rotation method according to an embodiment of the present application.
[0088] Figure 2b is another structural schematic diagram of the object model rotation system provided in an embodiment of the present application. In Figure 2b, the user device directly serves as a data processing device. The user device can directly obtain input from the user and directly process it by the hardware of the user device itself. The specific process is similar to that of Figure 2a. Please refer to the above description and will not be repeated here.
[0089] In the object model rotation system shown in Figure 2b, the user device can receive user instructions. For example, the user device can obtain information associated with the target object input by the user (that is, information of the target object in the first state). The user device can then perform a series of processing on the information associated with the target object to complete the required rotation of the preset n-dimensional model of the target object (used to indicate the target object in the second state) based on the information processing results, thereby obtaining the rotated n-dimensional model of the target object (used to indicate the target object in the first state).
[0090] In FIG. 2 b , the user equipment itself can execute the object model rotation method of the embodiment of the present application.
[0091] FIG2c is a schematic diagram of a device related to the rotation of an object model provided in an embodiment of the present application.
[0092] The user device in the above Figures 2a and 2b can specifically be the local device 301 or the local device 302 in Figure 2c, and the data processing device in Figure 2a can specifically be the execution device 210 in Figure 2c, wherein the data storage system 250 can store the data to be processed of the execution device 210, and the data storage system 250 can be integrated on the execution device 210, or it can be set on the cloud or other network servers.
[0093] The processors in Figures 2a and 2b can perform data training / machine learning / deep learning through a neural network model or other models (for example, a model based on a support vector machine), and use the model finally trained or learned from the data to perform object model rotation application on the image, thereby obtaining corresponding processing results.
[0094] Figure 3 is a schematic diagram of the system 100 architecture provided in an embodiment of the present application. In Figure 3, the execution device 110 is configured with an input / output (I / O) interface 112 for data interaction with an external device. The user can input data to the I / O interface 112 through the client device 140. The input data can include: various tasks to be scheduled, callable resources and other parameters in the embodiment of the present application.
[0095] When the execution device 110 preprocesses the input data, or when the computing module 111 of the execution device 110 performs calculations and other related processing (such as implementing the functions of the neural network in this application), the execution device 110 can call the data, code, etc. in the data storage system 150 for the corresponding processing, and can also store the data, instructions, etc. obtained from the corresponding processing in the data storage system 150.
[0096] Finally, the I / O interface 112 returns the processing result to the client device 140 so as to provide it to the user.
[0097] It is worth noting that the training device 120 can generate corresponding target models / rules based on different training data for different goals or tasks. The corresponding target models / rules can be used to achieve the above goals or complete the above tasks, thereby providing the user with the desired results. The training data can be stored in the database 130 and come from training samples collected by the data collection device 160.
[0098] In the scenario shown in FIG3 , the user can manually input data, which can be performed through the interface provided by I / O interface 112. Alternatively, client device 140 can automatically send input data to I / O interface 112. If user authorization is required for client device 140 to automatically send input data, the user can set the corresponding permissions in client device 140. The user can view the output of execution device 110 on client device 140, which can be presented in a display, sound, action, or other specific form. Client device 140 can also serve as a data acquisition terminal, collecting input data and output results from I / O interface 112 as new sample data and storing them in database 130. Of course, collection can also be performed without client device 140, with I / O interface 112 directly storing the input data and output results from I / O interface 112 as new sample data in database 130.
[0099] It is worth noting that FIG3 is only a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in FIG3, the data storage system 150 is an external memory relative to the execution device 110. In other cases, the data storage system 150 can also be placed in the execution device 110. As shown in FIG3, a neural network can be trained according to the training device 120.
[0100] The present application also provides a chip including a neural network processor (NPU). The chip can be provided in the execution device 110 shown in FIG3 to complete the computational work of the computation module 111. The chip can also be provided in the training device 120 shown in FIG3 to complete the training work of the training device 120 and output the target model / rule.
[0101] The neural network processor (NPU) is a coprocessor mounted on the host central processing unit (CPU), which assigns tasks to it. The core of the NPU is the arithmetic circuit, which is controlled by a controller to extract data from memory (weight memory or input memory) and perform calculations.
[0102] In some implementations, the arithmetic circuit includes multiple processing engines (PEs). In some implementations, the arithmetic circuit is a two-dimensional systolic array. The arithmetic circuit can also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit is a general-purpose matrix processor.
[0103] For example, consider an input matrix A, a weight matrix B, and an output matrix C. The computational circuit retrieves the corresponding data for matrix B from the weight memory and caches it on each PE within the computational circuit. The computational circuit then performs a matrix operation on the matrix A data from the input memory and matrix B, storing the partial or final matrix results in the accumulator.
[0104] The vector computation unit can further process the output of the arithmetic circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. For example, the vector computation unit can be used for network calculations in non-convolutional / non-FC layers in neural networks, such as pooling, batch normalization, and local response normalization.
[0105] In some implementations, the vector computation unit can store the processed output vectors in a unified buffer. For example, the vector computation unit can apply a nonlinear function to the output of the computation circuit, such as a vector of accumulated values, to generate activation values. In some implementations, the vector computation unit generates normalized values, merged values, or both. In some implementations, the processed output vectors can be used as activation inputs to the computation circuit, such as for use in subsequent layers in a neural network.
[0106] The unified memory is used to store input data and output data.
[0107] The weight data is directly transferred from the external memory to the input memory and / or unified memory through the direct memory access controller (DMAC), the weight data in the external memory is stored in the weight memory, and the data in the unified memory is stored in the external memory.
[0108] The bus interface unit (BIU) is used to implement interaction between the main CPU, DMAC and instruction fetch memory through the bus.
[0109] An instruction fetch buffer connected to the controller, used to store instructions used by the controller;
[0110] The controller is used to call the instructions cached in the memory to control the working process of the computing accelerator.
[0111] Generally, unified memory, input memory, weight memory and instruction fetch memory are all on-chip memories, and external memory is memory outside the NPU, which can be double data rate synchronous dynamic random access memory (DDR SDRAM), high bandwidth memory (HBM) or other readable and writable memory.
[0112] Since the embodiments of the present application involve the application of a large number of neural networks, in order to facilitate understanding, the relevant terms and related concepts such as neural networks involved in the embodiments of the present application are first introduced below.
[0113] (1) Neural Network
[0114] A neural network can be composed of neural units. A neural unit can refer to an operation unit with xs and intercept 1 as input. The output of the operation unit can be:
[0115] Where s = 1, 2, ... n, n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolutional layer. The activation function can be a sigmoid function. A neural network is a network formed by connecting many of the above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field. The local receptive field can be an area composed of several neural units.
[0116] The operation of each layer in a neural network can be described by the mathematical expression y = a(Wx + b). From a physical perspective, the operation of each layer in a neural network can be understood as transforming the input space (a set of input vectors) into the output space (i.e., from the row space to the column space of a matrix) through five operations. These operations include: 1. Dimensionality increase / decrease; 2. Scaling / reduction; 3. Rotation; 4. Translation; and 5. "Bending." Operations 1, 2, and 3 are performed by Wx, 4 by +b, and 5 by a(). The word "space" is used here because the objects being classified are not individual things, but rather a class of things, and space refers to the collection of all individuals within that class. W is the weight vector, each value in which represents the weight of a neuron in that layer of the neural network. This vector W determines the spatial transformation from input space to output space described above. That is, the weights W of each layer control how the space is transformed. The goal of training a neural network is to ultimately obtain the weight matrix for all layers of the trained neural network (a weight matrix formed by the vectors W of many layers). Therefore, the training process of a neural network is essentially about learning how to control spatial transformations, and more specifically, about learning the weight matrix.
[0117] Because we want the output of the neural network to be as close as possible to the value we really want to predict, we can compare the current network's predicted value with the desired target value, and then update the weight vector of each layer of the neural network based on the difference between the two (of course, there is usually an initialization process before the first update, which is to pre-configure the parameters for each layer in the neural network). For example, if the network's predicted value is too high, the weight vector is adjusted to make it predict a lower value, and this adjustment is continued until the neural network can predict the desired target value. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value." This is the loss function or objective function, which is an important equation used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the neural network becomes a process of minimizing this loss as much as possible.
[0118] (2) Backpropagation algorithm
[0119] Neural networks can use the back propagation (BP) algorithm to correct the size of the parameters in the initial neural network model during training, reducing the reconstruction error loss of the neural network model. Specifically, the forward propagation of the input signal to the output generates error loss. This error loss information is then backpropagated to update the parameters in the initial neural network model, thereby converging the error loss. The BP algorithm is a backward propagation movement dominated by error loss, aiming to obtain the optimal parameters of the neural network model, such as the weight matrix.
[0120] The method provided in this application is described below from the training side of the neural network and the application side of the neural network.
[0121] The model training method provided in the embodiment of the present application involves the processing of data sequences, and can be specifically applied to methods such as data training, machine learning, and deep learning, to perform symbolic and formalized intelligent information modeling, extraction, preprocessing, and training on the training data (for example, the information associated with the target object in the model training method provided in the embodiment of the present application), and finally obtain a trained neural network (for example, the target model in the model training method provided in the embodiment of the present application); and the object model rotation method provided in the embodiment of the present application can use the above-mentioned trained neural network to input the input data (for example, the information associated with the target object in the object model rotation method provided in the embodiment of the present application) into the trained neural network to obtain output data (for example, the rotated n-dimensional model of the target object in the object model rotation method provided in the embodiment of the present application). It should be noted that the model training method and the object model rotation method provided in the embodiment of the present application are inventions based on the same concept, and can also be understood as two parts in a system, or two stages of an overall process: such as the model training stage and the model application stage.
[0122] FIG4 is a flow chart of a method for rotating an object model according to an embodiment of the present application. As shown in FIG4 , the giant method includes:
[0123] 401. Obtain information of a target object in a first state.
[0124] In this embodiment, when it is necessary to rotate the object model of a target object, information about the target object in a first state (also referred to as a current state or real-time state) can be first obtained. It should be noted that the target object can be either an information collector or an information recipient. Therefore, the information about the target object in the first state includes two major categories of information: the first category is information about the target object in the first state (collected by other objects), and the second category is information about other objects collected by the target object in the first state.
[0125] Among them, the first type of information can refer to n-dimensional data (n is a positive integer greater than or equal to 2) of the target object in the first state. When n is 2, the 2D data can refer to images, videos, and 2D key points, etc. When n is 3, the 3D data can refer to point clouds and 3D key points, etc. The value of n can also be 4 and 5, etc., which will not be repeated here. For example, if the target object is a person, an image of the person who is currently moving can be obtained. For another example, if the target object is a vehicle, a video of the vehicle that is currently moving can be obtained, etc. For another example, if the target object is a cat, a point cloud of the cat that is currently jumping can be obtained, etc.
[0126] The second type of information can refer to n-dimensional data collected by the target object in the first state. Similarly, when n is 2, the 2D data can refer to images, videos, and 2D key points, etc. When n is 3, the 3D data can refer to point clouds and 3D key points, etc. The value of n can also be 4, 5, etc., which will not be repeated here. For example, if the target object is a camera, the image collected by the camera in a certain position can be obtained. For another example, if the target object is a lidar, the point cloud collected by the lidar in a certain position can be obtained, etc.
[0127] 402. Process the information through the target model to obtain a first matrix, where the first matrix is an n-order matrix, and n is a positive integer greater than or equal to 2.
[0128] After obtaining the information of the target object in the first state, the information of the target object in the first state can be input into the target model (which is a trained neural network model) so that the target model can perform a series of processing (for example, feature extraction, etc.) on the information of the target object in the first state, thereby obtaining a first matrix. It should be noted that the first matrix output by the target model is an n-order matrix, that is, the first matrix is a matrix with n rows and n columns. It can be understood that among the n×n elements contained in the first matrix, the n elements in the first column constitute the first first column vector, the n elements in the second column constitute the second first column vector, the n elements in the third column constitute the third first column vector, ..., the n elements in the nth column constitute the nth first column vector, that is, the first matrix contains n first column vectors. Since the first matrix is a normal matrix, the n first column vectors contained in the first matrix do not need to meet any conditions.
[0129] For example, as shown in Figure 5 (Figure 5 is a schematic diagram of an application example of the object model rotation method provided in an embodiment of the present application), after obtaining an image of a walking user, the user's image can be input into the target model. After processing by the target model, a third-order matrix (pseudo-rotation matrix) can be output, that is, a matrix with 3 rows and 3 columns, which contains column vector a1, column vector a2 and column vector a3, where a1 contains element a11, element a21 and element a31, a2 contains element a12, element a22 and element a32, and a3 contains element a13, element a23 and element a33.
[0130] 403. Orthogonalize the first matrix to obtain a second matrix, where the second matrix is an n-order rotation matrix.
[0131] Since the first matrix is a pseudo-rotation matrix, not a true rotation matrix, if the first matrix is used directly to rotate the n-dimensional model of the target object, the model will be deformed (for example, the bones of the model become longer or shorter). In order to solve this problem, the first matrix can be subjected to unbiased Gram-Schmidt orthogonalization to obtain the second matrix. It should be noted that the second matrix is an n-order rotation matrix, that is, the second matrix is an n-row and n-column rotation matrix. It can be understood that among the n×n elements contained in the second matrix, the n elements in the first column constitute the first second column vector, the n elements in the second column constitute the second second column vector, the n elements in the third column constitute the third second column vector, ..., the n elements in the nth column constitute the nth second column vector, that is, the second matrix contains n second column vectors, that is, the second matrix contains n second column vectors. Since the second matrix is a rotation matrix, the n second column vectors contained in the second matrix can satisfy at least one of the following conditions:
[0132] (a) The n second-column vectors are mutually orthogonal. That is, the vector product of any two of the n second-column vectors is zero. (b) The modulus of the n second-column vectors is unity. That is, for any second-column vector, the square root of the sum of the squares of all its elements is unity. (c) The n second-column vectors form an n-dimensional coordinate system. For example, when n is 3, the three second-column vectors form a right-handed coordinate system.
[0133] Still as shown in the above example, after obtaining the third-order matrix, the third-order matrix can be unbiased Schmidt orthogonalized to obtain a third-order rotation matrix, that is, a rotation matrix with 3 rows and 3 columns, which contains column vector a`1, column vector a`2 and column vector a`3, where a`1 contains elements a`11, a`21 and a`31, a`2 contains elements a`12, a`22 and a`32, and a`3 contains elements a`13, a`23 and a`33.
[0134] Specifically, the second matrix can be obtained in the following way:
[0135] (1) After obtaining n first column vectors, a first calculation may be performed on the first first column vector to the nth first column vector to obtain the first second column vector.
[0136] The first calculation process may include:
[0137] (1.1) After obtaining n first column vectors, we can perform cross multiplication on the second first column vector to the nth first column vector to obtain the first third column vector. The cross multiplication can be implemented by the following formula: n-cross(p2, ..., p n ) (2)
[0138] In the above formula, n-cross(p2, ..., p n ) is the first third column vector, p2 is the second first column vector, p n is the nth first column vector.
[0139] (1.2) After obtaining the first third column vector, a weighted average can be performed on the first third column vector and the first first column vector to obtain the first fourth column vector. The weighted average can be achieved by the following formula (assuming that the weight of each column vector is 0.5 for schematic illustration. In actual applications, the weight of each column vector can be set according to actual needs and is not limited here):
[0140] In the above formula, avg(p1) is the first fourth column vector, and p1 is the first first column vector.
[0141] (1.3) After obtaining the first fourth column vector, the first fourth column vector can be normalized to obtain the first second column vector. The normalization can be achieved by the following formula: r1=N(avg(p1)) (4)
[0142] In the above formula, r1 is the first second column vector.
[0143] (2) After obtaining the first second column vector, a second calculation may be performed on the first second column vector and the second first column vector to the nth first column vector to obtain the second second column vector. After obtaining the second second column vector, a second calculation may be performed on the first second column vector to the second second column vector and the third first column vector to the nth first column vector to obtain the third second column vector, ..., after obtaining the n-2th second column vector, a second calculation may be performed on the first second column vector to the n-2th second column vector and the n-1th first column vector to the nth first column vector to obtain the n-1th second column vector.
[0144] Thus, it can be seen that the process of obtaining the second second column vector to the n-1th second column vector is similar. The following is a schematic introduction using any second column vector, and the second column vector is referred to as the i-th second column vector (i=2, ...n-1). Then, the second calculation process may include:
[0145] (2.1) After obtaining the i-1th second column vector, we can perform cross multiplication on the 1st second column vector to the i-1th second column vector, and on the i+1th first column vector to the nth first column vector, thereby obtaining the i-th third column vector. The cross multiplication can be implemented by the following formula: n-cross(r1, ..., r i-1 ,p i+1 ,...,p n ) (5)
[0146] In the above formula, n-cross(r1,...,p i-1 ,p i+1 ,...,p n ) is the i-th third column vector, r1 is the first second column vector, r i-1 is the second column vector of the i-1th column, p i+1 is the first column vector of the i+1th column, p n is the nth first column vector.
[0147] (2.2) After obtaining the i-th third column vector, the i-th third column vector and the i-th first column vector can be averaged to obtain the i-th fourth column vector. The weighted average can be achieved using the following formula (assuming that the weight of each column vector is 0.5 for schematic illustration. In actual applications, the weight of each column vector can be set according to actual needs and is not limited here):
[0148] In the above formula, avg(p i ) is the fourth column vector of the i-th column, p i is the first column vector of the i-th column.
[0149] (2.3) After obtaining the i-th fourth column vector, the i-th fourth column vector can be projected onto the 1st second column vector to the i-1th second column vector, and the projection results are added together to obtain the i-th fifth column vector. The projection and addition can be implemented using the following formula:
[0150] In the above formula, is the fifth column vector of the i-th column, r j is the jth second column vector (j=1,...,i-1), (r j avg(p i ))r j is the projection result obtained by projecting the i-th fourth column vector onto the j-th second column vector.
[0151] (2.4) After obtaining the i-th fifth column vector, the i-th fourth column vector and the i-th fifth column vector can be subtracted to obtain the i-th sixth column vector. The subtraction can be achieved by the following formula:
[0152] In the above formula, is the sixth column vector of the i-th column.
[0153] (2.5) After obtaining the i-th sixth column vector, the i-th sixth column vector is normalized to obtain the i-th second column vector. The normalization can be achieved by the following formula:
[0154] In the above formula, r i is the ith second column vector. Repeat steps (2.1) to (2.5) to obtain the 2nd to n-1th second column vectors.
[0155] (3) After steps (1) and (2), the first second column vector to the n-1th second column vector can be obtained, so the third calculation can be performed on the first second column vector to the n-1th second column vector to obtain the nth second column vector.
[0156] The second calculation process may include:
[0157] After obtaining the first second column vector to the n-1th second column vector, a cross product can be performed on the first second column vector to the n-1th second column vector to obtain the nth second column vector. The cross product can be implemented by the following formula: n =n-cross(r1, ..., r n-1 ) (10)
[0158] In the above formula, rn is the nth second column vector. So far, we can get the first second column vector to the nth second column vector, that is, the second matrix.
[0159] 404. Based on the second matrix, rotate the preset n-dimensional model of the target object to obtain a rotated n-dimensional model of the target object, wherein the preset n-dimensional model is used to indicate the target object in the second state, and the rotated n-dimensional model is used to indicate the target object in the first state.
[0160] After obtaining the second matrix, the preset n-dimensional model of the target object can be directly rotated using the second matrix to obtain the rotated n-dimensional model of the target object. It should be noted that the preset n-dimensional model of the target object is used to indicate the target object in the second state (also referred to as the preset state or initial state), while the rotated n-dimensional model of the target object is used to indicate the target object in the first state. At this point, the object model rotation of the target object is complete.
[0161] As in the previous example, after obtaining the third-order rotation matrix, we can directly use this rotation matrix to rotate the 3D model (3D points) of the standard human body to obtain the rotated 3D model of the standard human body. The 3D model of the standard human body is used to indicate a user in a stationary state (for example, the user is standing with their hands spread out), while the rotated 3D model of the standard human body is used to indicate a user in an active state (for example, the user is walking).
[0162] In addition, the target model provided in the embodiment of the present application (such as ours in Table 1 and Table 2) can be compared with the models provided by the related art (such as GS, SVD, etc. in Table 1 and Table 2). The comparison results are shown in Table 1 and Table 2:
[0163] Table 1
[0164] Table 2
[0165] Based on Table 1 and Table 2, the performance (accuracy and speed) of the target model provided by the embodiment of the present application is better than that of the model provided by the related art.
[0166] In an embodiment of the present application, when it is necessary to rotate the object model of the target object, the information of the target object in the first state can be obtained first, and the information can be input into the target model. Then, the information can be processed by the target model to obtain a first matrix, which is an n-order matrix, and n is a positive integer greater than or equal to 2. Then, the first matrix can be orthogonalized to obtain a second matrix, which is an n-order rotation matrix. Finally, the preset n-dimensional model of the target object can be rotated directly based on the second matrix to obtain the rotated n-dimensional model of the target object. The preset n-dimensional model of the target object is used to indicate the target object in the second state, and the rotated n-dimensional model of the target object is used to indicate the target object in the first state. In the above process, the output of the target model is an n×n matrix, which can be regarded as a pseudo-rotation matrix, so the matrix can be used directly to complete the training of the target model without undergoing orthogonalization. It can be seen that orthogonalization processing will not occur during the training process of the target model, which can avoid various problems caused by orthogonalization processing during the model training process. Training the target model in this way is conducive to improving the performance of the target model, that is, the target model can accurately complete the rotation estimation, and then accurately complete the object model rotation for the target object.
[0167] The above is a detailed description of the object model rotation method provided by the embodiment of the present application. The following is an introduction to the model training method provided by the embodiment of the present application. Figure 6 is a flow chart of the model training method provided by the embodiment of the present application. As shown in Figure 6, the method includes:
[0168] 601. Obtain information of a target object in a first state.
[0169] In this embodiment, when training a model to be trained (a neural network model to be trained), a batch of training data can be first obtained, where the training data includes information about a target object in a first state. It should be noted that the true n-dimensional model and true matrix of the target object after rotation are known, and the true matrix is an n-order matrix (i.e., an n×n matrix). The true n-dimensional model of the target object after rotation can be used to accurately simulate (indicate) the target object in the first state.
[0170] In a possible implementation, the information of the target object in the first state includes at least one of the following: n-dimensional data of the target object in the first state and n-dimensional data collected from the target object in the first state.
[0171] For an introduction to step 601 , please refer to the relevant description of step 401 in the embodiment shown in FIG. 4 , which will not be repeated here.
[0172] 602. Process the information through the target model to obtain a first matrix, where the first matrix is an n-order matrix, where n is a positive integer greater than or equal to 2. The first matrix is used to obtain a rotated n-dimensional model of the target object, and the rotated n-dimensional model is used to indicate the target object in a first state.
[0173] The information of the target object in the first state is obtained, and the information of the target object in the first state can be input into the model to be trained. Then, the model to be trained can process the information of the target object in the first state (for example, feature extraction, etc.) to obtain a first matrix, where the first matrix is an n-order matrix, and n is a positive integer greater than or equal to 2.
[0174] For an introduction to step 602 , please refer to the relevant description of step 402 in the embodiment shown in FIG. 4 , which will not be repeated here.
[0175] 603. Obtain target loss based on the first matrix.
[0176] After obtaining the first matrix, the first matrix can be further processed to obtain the target loss.
[0177] Specifically, the target loss can be obtained in the following ways:
[0178] (1) After obtaining the first matrix, the preset n-dimensional model of the target object is directly rotated using the first matrix to obtain the rotated n-dimensional model of the target object. It should be noted that the preset n-dimensional model of the target object can be used to accurately simulate the target object in the second state, and the rotated n-dimensional model of the target object can be used to roughly simulate the target object in the first state.
[0179] After obtaining the rotated n-dimensional model of the target object, since the true n-dimensional model of the target object after rotation is known, the rotated n-dimensional model of the target object and the true n-dimensional model of the target object after rotation can be calculated through a preset first loss function to obtain a target loss, which is used to indicate the difference between the rotated n-dimensional model of the target object and the true n-dimensional model of the target object after rotation.
[0180] (2) After obtaining the first matrix, the first matrix and the true matrix are calculated using a preset second loss function to obtain a target loss, which is used to indicate the difference between the first matrix and the true matrix.
[0181] 604. Based on the target loss, the to-be-trained model is trained to obtain a target model.
[0182] After obtaining the target loss, the parameters of the model to be trained can be updated based on the target loss to obtain the model to be trained with updated parameters, and the model to be trained with updated parameters can be further trained using the next batch of training data until the model training conditions are met (for example, the target loss converges, etc.), thereby obtaining the target model in the embodiment shown in Figure 4.
[0183] For example, as shown in FIG7 (FIG. 7 is a schematic diagram of an application example of the model training method provided in an embodiment of the present application, and FIG7 is drawn based on FIG5), after obtaining the training data, the training data (for example, images of active users, etc.) can be input into the model to be trained. The model to be trained can directly output a third-order matrix (pseudo-rotation matrix). Then, the third-order matrix can be directly used to construct the target loss, or the third-order matrix can be used to rotate the 3D model of the standard human body to obtain the rotated 3D model of the standard human body, and the target loss can be directly constructed using the rotated 3D model of the standard human body. Then, the model to be trained is trained based on the target loss until the target model is obtained.
[0184] In addition, the model training method provided in the embodiment of the present application (such as ours in Figures 8 and 9) can be compared with the model training method provided by the related technology (such as 6D, SVD, etc. in Figures 8 and 9). The comparison results are shown in Figures 8 and 9 (Figure 8 is a schematic diagram of the comparison results provided in the embodiment of the present application, and Figure 9 is another schematic diagram of the comparison results provided in the embodiment of the present application). It can be seen that the training process of the target model provided in the embodiment of the present application converges faster and more stably, and has lower errors.
[0185] The target model obtained by training in the embodiment of the present application has a rotation estimation function. Specifically, when it is necessary to rotate the object model of the target object, the information of the target object in the first state can be obtained first, and the information can be input into the target model. Then, the information can be processed by the target model to obtain a first matrix, which is an n-order matrix, and n is a positive integer greater than or equal to 2. Then, the first matrix can be orthogonalized to obtain a second matrix, which is an n-order rotation matrix. Finally, the preset n-dimensional model of the target object can be rotated directly based on the second matrix to obtain the rotated n-dimensional model of the target object. The preset n-dimensional model of the target object is used to indicate the target object in the second state, and the rotated n-dimensional model of the target object is used to indicate the target object in the first state. In the above process, the output of the target model is an n×n matrix, which can be regarded as a pseudo-rotation matrix, so the matrix can be used directly to complete the training of the target model without undergoing orthogonalization. It can be seen that orthogonalization processing will not occur during the training process of the target model, which can avoid various problems caused by orthogonalization processing during the model training process. Training the target model in this way is conducive to improving the performance of the target model, that is, the target model can accurately complete the rotation estimation, and then accurately complete the object model rotation for the target object.
[0186] The above is a detailed description of the object model rotation method and model training method provided in the embodiments of the present application. The object model rotation device and model training device provided in the embodiments of the present application will be introduced below. Figure 10 is a structural schematic diagram of the object model rotation device provided in the embodiments of the present application. As shown in Figure 10, the device includes:
[0187] An acquisition module 1001 is configured to acquire information of a target object in a first state;
[0188] A processing module 1002 is configured to process the information using a target model to obtain a first matrix, where the first matrix is an n-th order matrix, where n is a positive integer greater than or equal to 2;
[0189] An orthogonalization module 1003 is used to orthogonalize the first matrix to obtain a second matrix, where the second matrix is an n-order rotation matrix;
[0190] The rotation module 1004 is used to rotate the preset n-dimensional model of the target object based on the second matrix to obtain the rotated n-dimensional model of the target object. The preset n-dimensional model is used to indicate the target object in the second state, and the rotated n-dimensional model is used to indicate the target object in the first state.
[0191] In an embodiment of the present application, when it is necessary to rotate the object model of the target object, the information of the target object in the first state can be obtained first, and the information can be input into the target model. Then, the information can be processed by the target model to obtain a first matrix, which is an n-order matrix, and n is a positive integer greater than or equal to 2. Then, the first matrix can be orthogonalized to obtain a second matrix, which is an n-order rotation matrix. Finally, the preset n-dimensional model of the target object can be rotated directly based on the second matrix to obtain the rotated n-dimensional model of the target object. The preset n-dimensional model of the target object is used to indicate the target object in the second state, and the rotated n-dimensional model of the target object is used to indicate the target object in the first state. In the above process, the output of the target model is an n×n matrix, which can be regarded as a pseudo-rotation matrix, so the matrix can be used directly to complete the training of the target model without undergoing orthogonalization. It can be seen that orthogonalization processing will not occur during the training process of the target model, which can avoid various problems caused by orthogonalization processing during the model training process. Training the target model in this way is conducive to improving the performance of the target model, that is, the target model can accurately complete the rotation estimation, and then accurately complete the object model rotation for the target object.
[0192] In one possible implementation, the first matrix contains n first column vectors, the second matrix contains n second column vectors, and the orthogonalization module is used to: perform a first calculation on the 1st first column vector to the nth first column vector to obtain the 1st second column vector; perform a second calculation on the 1st second column vector to the i-1th second column vector, and on the ith first column vector to the nth first column vector to obtain the ith second column vector, i=2,...n-1; perform a third calculation on the 1st second column vector to the n-1th second column vector to obtain the nth second column vector.
[0193] In one possible implementation, the orthogonalization module is used to: perform a cross product from the second first column vector to the nth first column vector to obtain the first third column vector; perform a weighted average on the first third column vector and the first first column vector to obtain the first fourth column vector; and normalize the first fourth column vector to obtain the first second column vector.
[0194] In one possible implementation, the orthogonalization module is used to: perform cross multiplication on the 1st second column vector to the i-1th second column vector, and the i+1th first column vector to the nth first column vector, to obtain the i-th third column vector; average the i-th third column vector and the i-th first column vector to obtain the i-th fourth column vector; project the i-th fourth column vector to the 1st second column vector to the i-1th second column vector, and add the projection results to obtain the i-th fifth column vector; subtract the i-th fourth column vector from the i-th fifth column vector to obtain the i-th sixth column vector; and normalize the i-th sixth column vector to obtain the i-th second column vector.
[0195] In a possible implementation, the orthogonalization module is configured to perform a cross product on the 1st second column vector to the n-1th second column vector to obtain the nth second column vector.
[0196] In one possible implementation, the n second column vectors satisfy at least one of the following conditions: the n second column vectors are mutually orthogonal; the modules of the n second column vectors are all one; and the n second column vectors constitute an n-dimensional coordinate system.
[0197] In a possible implementation, the information includes at least one of the following: n-dimensional data of the target object in the first state and n-dimensional data collected from the target object in the first state.
[0198] FIG11 is a schematic diagram of the structure of a model training device provided in an embodiment of the present application. As shown in FIG11 , the device includes:
[0199] A first acquisition module 1101 is configured to acquire information of a target object in a first state;
[0200] a processing module 1102 configured to process the information using the target model to obtain a first matrix, where the first matrix is an n-order matrix, where n is a positive integer greater than or equal to 2, and the first matrix is used to obtain a rotated n-dimensional model of the target object, where the rotated n-dimensional model is used to indicate the target object in a first state;
[0201] A second acquisition module 1103 is configured to acquire a target loss based on the first matrix;
[0202] The training module 1104 is used to train the to-be-trained model based on the target loss to obtain a target model.
[0203] In one possible implementation, the second acquisition module is used to: directly use the first matrix to rotate the preset n-dimensional model of the target object to obtain the rotated n-dimensional model of the target object, and the preset n-dimensional model is used to indicate the target object in the second state; based on the rotated n-dimensional model and the rotated true n-dimensional model of the target object, obtain the target loss, and the rotated true n-dimensional model is used to indicate the target object in the first state, and the target loss is used to indicate the difference between the rotated n-dimensional model and the rotated true n-dimensional model.
[0204] In one possible implementation, the second acquisition module is used to acquire a target loss based on the first matrix and the true matrix, where the target loss is used to indicate the difference between the first matrix and the true matrix, and the true matrix is an n-order matrix.
[0205] In a possible implementation, the information includes at least one of the following: n-dimensional data of the target object in the first state and n-dimensional data collected from the target object in the first state.
[0206] It should be noted that the information interaction, execution process, etc. between the modules / units of the above-mentioned device are based on the same concept as the method embodiment of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown above in the embodiment of the present application, and no further details will be given here.
[0207] The embodiment of the present application also relates to an execution device, and Figure 12 is a structural diagram of the execution device provided by the embodiment of the present application. As shown in Figure 12, the execution device 1200 can be specifically manifested as a mobile phone, a tablet, a laptop computer, an intelligent wearable device, a server, etc., which are not limited here. Among them, the execution device 1200 can be deployed with the object model rotation device described in the embodiment corresponding to Figure 10, for realizing the function of rotating the object model in the embodiment corresponding to Figure 4. Specifically, the execution device 1200 includes: a receiver 1201, a transmitter 1202, a processor 1203 and a memory 1204 (wherein the number of processors 1203 in the execution device 1200 can be one or more, and Figure 12 takes one processor as an example), wherein the processor 1203 may include an application processor 12031 and a communication processor 12032. In some embodiments of the present application, the receiver 1201, the transmitter 1202, the processor 1203 and the memory 1204 can be connected via a bus or other means.
[0208] The memory 1204 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1203. A portion of the memory 1204 may also include non-volatile random access memory (NVRAM). The memory 1204 stores processor and operation instructions, executable modules, or data structures, or subsets or extended sets thereof. The operation instructions may include various operation instructions for implementing various operations.
[0209] Processor 1203 controls the operation of the execution device. In specific applications, the various components of the execution device are coupled together via a bus system. In addition to a data bus, the bus system may also include a power bus, a control bus, and a status signal bus. However, for clarity, all bus systems are referred to as a bus system in the figure.
[0210] The methods disclosed in the above embodiments of the present application can be applied to or implemented by processor 1203. Processor 1203 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in processor 1203. The above processor 1203 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and can further include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor 1203 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of the present application can be directly implemented as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 1204, and processor 1203 reads the information in memory 1204 and, in conjunction with its hardware, completes the steps of the above method.
[0211] Receiver 1201 can be used to receive input digital or character information and generate signal input related to executing device-related settings and function control. Transmitter 1202 can be used to output digital or character information through the first interface. Transmitter 1202 can also be used to send instructions to the disk pack through the first interface to modify data in the disk pack. Transmitter 1202 can also include a display device such as a display screen.
[0212] In an embodiment of the present application, in one case, the processor 1203 is configured to obtain a rotated n-dimensional model of the target object through the target model in the embodiment corresponding to FIG. 4 .
[0213] The present application also relates to a training device. FIG13 is a schematic diagram of the structure of the training device provided by the present application. As shown in FIG13 , the training device 1300 is implemented by one or more servers. The training device 1300 may vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1313 (e.g., one or more processors) and memory 1332, and one or more storage media 1330 (e.g., one or more mass storage devices) storing application programs 1342 or data 1344. The memory 1332 and storage medium 1330 may be temporary storage or permanent storage. The program stored in the storage medium 1330 may include one or more modules (not shown), each module may include a series of instruction operations in the training device. Furthermore, the CPU 1313 may be configured to communicate with the storage medium 1330 to execute a series of instruction operations in the storage medium 1330 on the training device 1300.
[0214] The training device 1300 may also include one or more power supplies 1326, one or more wired or wireless network interfaces 1350, one or more input and output interfaces 1358; or, one or more operating systems 1341, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.
[0215] Specifically, the training device can execute the model training method in the embodiment corresponding to Figure 6 to obtain the target model.
[0216] An embodiment of the present application also relates to a computer storage medium, which stores a program for signal processing. When the computer storage medium is run on a computer, it enables the computer to execute the steps executed by the aforementioned execution device, or enables the computer to execute the steps executed by the aforementioned training device.
[0217] An embodiment of the present application also relates to a computer program product, which stores instructions that, when executed by a computer, enable the computer to execute the steps executed by the aforementioned execution device, or enable the computer to execute the steps executed by the aforementioned training device.
[0218] The execution device, training device or terminal device provided in the embodiments of the present application can specifically be a chip, and the chip includes: a processing unit and a communication unit, the processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, a pin or a circuit, etc. The processing unit can execute the computer execution instructions stored in the storage unit, so that the chip in the execution device executes the data processing method described in the above embodiment, or so that the chip in the training device executes the data processing method described in the above embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit can also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.
[0219] Specifically, see Figure 14 , which is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip may be a neural network processor (NPU) 1400. NPU 1400 is mounted on a host CPU (host CPU) as a coprocessor, with tasks assigned by the host CPU. The core of the NPU is arithmetic circuit 1403, which is controlled by controller 1404 to extract matrix data from memory and perform multiplication operations.
[0220] In some implementations, the arithmetic circuit 1403 includes multiple processing units (PEs). In some implementations, the arithmetic circuit 1403 is a two-dimensional systolic array. The arithmetic circuit 1403 can also be a one-dimensional systolic array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1403 is a general-purpose matrix processor.
[0221] For example, assume there are input matrix A, weight matrix B, and output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from weight memory 1402 and caches it on each PE in the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from input memory 1401 and performs a matrix operation on matrix B. The partial or final matrix result is stored in accumulator 1408.
[0222] Unified memory 1406 is used to store input and output data. Weight data is directly transferred to weight memory 1402 through the Direct Memory Access Controller (DMAC) 1405. Input data is also transferred to unified memory 1406 through the DMAC.
[0223] BIU stands for Bus Interface Unit, i.e., bus interface unit 1413 , which is used for interaction between the AXI bus, DMAC, and instruction fetch buffer (IFB) 1409 .
[0224] The bus interface unit 1413 (BIU) is used for the instruction fetch memory 1409 to obtain instructions from the external memory, and is also used for the storage unit access controller 1405 to obtain the original data of the input matrix A or the weight matrix B from the external memory.
[0225] DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 1406 or transfer weight data to the weight memory 1402 or transfer input data to the input memory 1401.
[0226] The vector calculation unit 1407 includes multiple operation processing units. When necessary, it further processes the output of the operation circuit 1403, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as batch normalization, pixel-level summation, and upsampling of the predicted label plane.
[0227] In some implementations, the vector calculation unit 1407 can store the processed output vector to the unified memory 1406. For example, the vector calculation unit 1407 can apply a linear function or a nonlinear function to the output of the operation circuit 1403, such as linear interpolation of the predicted label plane extracted by the convolution layer, or another example is a vector of accumulated values to generate an activation value. In some implementations, the vector calculation unit 1407 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1403, for example, for use in subsequent layers in a neural network.
[0228] An instruction fetch buffer 1409 connected to the controller 1404 is used to store instructions used by the controller 1404;
[0229] Unified memory 1406, input memory 1401, weight memory 1402, and instruction fetch memory 1409 are all on-chip memories. External memories are private to the NPU hardware architecture.
[0230] The processor mentioned in any of the above places can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the above program.
[0231] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.
[0232] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.
[0233] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.
[0234] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).
Claims
1. A method for rotating an object model, characterized in that: The method comprises: Acquire information of a target object in a first state; Processing the information through a target model to obtain a first matrix, where the first matrix is an n-order matrix, and n is a positive integer greater than or equal to 2; Orthogonalizing the first matrix to obtain a second matrix, where the second matrix is an n-order rotation matrix; Based on the second matrix, the preset n-dimensional model of the target object is rotated to obtain the rotated n-dimensional model of the target object, the preset n-dimensional model is used to indicate the target object in the second state, and the rotated n-dimensional model is used to indicate the target object in the first state.
2. The method according to claim 1, characterized in that The first matrix includes n first column vectors, the second matrix includes n second column vectors, and orthogonalizing the first matrix to obtain a second matrix corresponding to the target object includes: Perform a first calculation on the 1st first column vector to the nth first column vector to obtain the 1st second column vector; Perform a second calculation on the 1st second column vector to the i-1th second column vector, and the i-th first column vector to the n-th first column vector to obtain the i-th second column vector, i=2, ...n-1; A third calculation is performed on the first second column vector to the n-1th second column vector to obtain an nth second column vector.
3. The method according to claim 2, characterized in that The performing a first calculation on the first column vector to the nth first column vector to obtain the first second column vector comprises: Perform cross multiplication on the second first column vector to the nth first column vector to obtain the first third column vector; Performing weighted averaging on the first third column vector and the first first column vector to obtain a first fourth column vector; The first fourth column vector is normalized to obtain the first second column vector.
4. The method according to claim 2 or 3, characterized in that: The performing a second calculation on the 1st second column vector to the i-1th second column vector and the i-th first column vector to the n-th first column vector to obtain the i-th second column vector comprises: Perform cross multiplication on the 1st second column vector to the i-1th second column vector, and the i+1th first column vector to the nth first column vector to obtain the ith third column vector; Averaging the i-th third column vector and the i-th first column vector to obtain the i-th fourth column vector; Project the i-th fourth column vector to the 1st second column vector to the (i-1)th second column vector, and add the projection results to obtain the i-th fifth column vector; Subtract the i-th fourth column vector from the i-th fifth column vector to obtain the i-th sixth column vector; The i-th sixth column vector is normalized to obtain the i-th second column vector.
5. The method according to any one of claims 2 to 4, characterized in that: The performing a third calculation on the first second column vector to the n-1th second column vector to obtain the nth second column vector comprises: Perform a cross product on the first second column vector to the n-1th second column vector to obtain an nth second column vector.
6. The method according to any one of claims 2 to 5, characterized in that: The n second column vectors satisfy at least one of the following conditions: The n second column vectors are mutually orthogonal; The moduli of the n second column vectors are all one; The n second column vectors constitute an n-dimensional coordinate system.
7. The method according to any one of claims 1 to 6, characterized in that: The information includes at least one of the following: n-dimensional data of the target object in the first state and n-dimensional data collected from the target object in the first state.
8. A model training method, characterized in that: The method comprises: Acquire information of a target object in a first state; Processing the information through the target model to obtain a first matrix, where the first matrix is an n-order matrix, where n is a positive integer greater than or equal to 2, and the first matrix is used to obtain a rotated n-dimensional model of the target object, and the rotated n-dimensional model is used to indicate the target object in the first state; Based on the first matrix, obtaining a target loss; Based on the target loss, the model to be trained is trained to obtain a target model.
9. The method according to claim 8, characterized in that The obtaining the target loss based on the first matrix includes: directly using the first matrix to rotate the preset n-dimensional model of the target object to obtain the rotated n-dimensional model of the target object, wherein the preset n-dimensional model is used to indicate the target object in the second state; Based on the rotated n-dimensional model and the rotated real n-dimensional model of the target object, a target loss is obtained, wherein the rotated real n-dimensional model is used to indicate the target object in the first state, and the target loss is used to indicate the difference between the rotated n-dimensional model and the rotated real n-dimensional model.
10. The method according to claim 8, characterized in that The acquiring the target loss based on the first matrix includes: Based on the first matrix and the true matrix, a target loss is obtained, where the target loss is used to indicate the difference between the first matrix and the true matrix, and the true matrix is an n-order matrix.
11. The method according to any one of claims 8 to 10, characterized in that: The information includes at least one of the following: n-dimensional data of the target object in the first state and n-dimensional data collected from the target object in the first state.
12. An object model rotating device, characterized in that: The device comprises: An acquisition module, used for acquiring information of a target object in a first state; A processing module, used for processing the information through a target model to obtain a first matrix, where the first matrix is an n-order matrix, and n is a positive integer greater than or equal to 2; An orthogonalization module, used for orthogonalizing the first matrix to obtain a second matrix, where the second matrix is an n-order rotation matrix; A rotation module is used to rotate the preset n-dimensional model of the target object based on the second matrix to obtain the rotated n-dimensional model of the target object, wherein the preset n-dimensional model is used to indicate the target object in the second state, and the rotated n-dimensional model is used to indicate the target object in the first state.
13. The device according to claim 12, characterized in that The first matrix includes n first column vectors, the second matrix includes n second column vectors, and the orthogonalization module is used to: Perform a first calculation on the 1st first column vector to the nth first column vector to obtain the 1st second column vector; Perform a second calculation on the 1st second column vector to the i-1th second column vector, and the i-th first column vector to the n-th first column vector to obtain the i-th second column vector, i=2, ...n-1; A third calculation is performed on the first second column vector to the n-1th second column vector to obtain an nth second column vector.
14. The device according to claim 13, characterized in that The orthogonalization module is used for: Perform cross multiplication on the second first column vector to the nth first column vector to obtain the first third column vector; Performing weighted averaging on the first third column vector and the first first column vector to obtain a first fourth column vector; The first fourth column vector is normalized to obtain a first second column vector.
15. The device according to claim 13 or 14, characterized in that The orthogonalization module is used for: Perform cross multiplication on the 1st second column vector to the i-1th second column vector, and the i+1th first column vector to the nth first column vector to obtain the ith third column vector; Averaging the i-th third column vector and the i-th first column vector to obtain the i-th fourth column vector; Project the i-th fourth column vector to the 1st second column vector to the (i-1)th second column vector, and add the projection results to obtain the i-th fifth column vector; Subtract the i-th fourth column vector from the i-th fifth column vector to obtain the i-th sixth column vector; The i-th sixth column vector is normalized to obtain the i-th second column vector.
16. The device according to any one of claims 13 to 15, characterized in that The orthogonalization module is used to perform cross multiplication on the first second column vector to the n-1th second column vector to obtain the nth second column vector.
17. The device according to any one of claims 13 to 16, characterized in that The n second column vectors satisfy at least one of the following conditions: The n second column vectors are mutually orthogonal; The moduli of the n second column vectors are all one; The n second column vectors constitute an n-dimensional coordinate system.
18. The device according to any one of claims 12 to 17, characterized in that The information includes at least one of the following: n-dimensional data of the target object in the first state and n-dimensional data collected from the target object in the first state.
19. A model training device, characterized in that: The device comprises: A first acquisition module, used to acquire information of a target object in a first state; a processing module, configured to process the information through a target model to obtain a first matrix, wherein the first matrix is an n-order matrix, where n is a positive integer greater than or equal to 2, and the first matrix is used to obtain a rotated n-dimensional model of the target object, and the rotated n-dimensional model is used to indicate the target object in the first state; A second acquisition module, used for acquiring a target loss based on the first matrix; A training module is used to train the model to be trained based on the target loss to obtain a target model.
20. The device according to claim 19, characterized in that The second acquisition module is used to: directly using the first matrix to rotate the preset n-dimensional model of the target object to obtain the rotated n-dimensional model of the target object, wherein the preset n-dimensional model is used to indicate the target object in the second state; Based on the rotated n-dimensional model and the rotated real n-dimensional model of the target object, a target loss is obtained, wherein the rotated real n-dimensional model is used to indicate the target object in the first state, and the target loss is used to indicate the difference between the rotated n-dimensional model and the rotated real n-dimensional model.
21. The device according to claim 19, characterized in that The second acquisition module is used to acquire a target loss based on the first matrix and a true matrix, where the target loss is used to indicate the difference between the first matrix and the true matrix, and the true matrix is an n-order matrix.
22. The device according to any one of claims 19 to 21, characterized in that The information includes at least one of the following: n-dimensional data of the target object in the first state and n-dimensional data collected from the target object in the first state.
23. An object model rotating device, characterized in that: The device includes a memory and a processor; the memory stores codes, and the processor is configured to execute the codes. When the codes are executed, the object model rotation device performs the method according to any one of claims 1 to 11.
24. A computer storage medium, characterized in that The computer storage medium stores one or more instructions, which, when executed by one or more computers, enable the one or more computers to implement the method of any one of claims 1 to 11.
25. A computer program product, characterized in that The computer program product stores instructions, which, when executed by a computer, enable the computer to implement the method according to any one of claims 1 to 11.
Citation Information
Patent Citations
Human head posture recognition method and device, human head posture model construction method and device, equipment and storage medium
CN113158708A
Virtual model, construction method thereof, interaction method and electronic equipment
CN113344770A
Iterative 6D pose estimation method and device based on deep learning
CN114119999A
Calculation method, device and equipment of coordinate transformation matrix and readable storage medium
CN114186189A
Object model rotating method and related equipment thereof
CN116681596A
