Depth regression method and device based on angle compensation, equipment and medium

Abstract

The invention discloses a depth regression method and device based on angle compensation, equipment and a medium, and the method comprises the steps: creating a basic regression model which comprises an encoder, a projection layer and a prediction layer, training the basic regression model through an angle compensation loss function, and obtaining a depth regression model; and finally, inputting verification set data to the deep regression model to verify the model performance, thereby solving the technical problems that a large amount of training data is generally needed and the effect is often lower than that of other task types due to the fact that the sample distribution of the regression task is generally continuous and unbalanced, such as age, power consumption, physiological indexes and the like.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a depth regression method, device, equipment and medium based on angle compensation. Background Art

[0002] Regression Task is a supervised learning task in machine learning, whose goal is to predict a continuous numerical output based on the input feature value. In regression tasks, common applications include scenarios such as house price prediction and stock price prediction, and the goal of these tasks is to predict a continuous numerical value. For example, in the house price prediction task, the input features may include information such as the house area and the number of rooms, and the output is the price of the house. Regression tasks can be implemented using a variety of algorithms, the most common of which are linear regression and logistic regression. Linear regression is one of the simplest regression algorithms, which describes the relationship between the independent variable and the dependent variable by fitting a straight line. Although logistic regression is usually used for classification problems, it can also be used for regression tasks in some cases, especially when dealing with binary classification problems. In addition to linear regression and logistic regression, there are many other regression algorithms to choose from, such as ridge regression, LASSO regression, Elastic Net, polynomial regression, multilayer perceptron, random forest regression, and support vector machine. Therefore, the application areas of regression tasks are very wide, covering stock price and credit score prediction in the financial field, disease and drug response prediction in the medical field, sales and customer value prediction in marketing, and many other fields, such as meteorology, transportation planning, energy management, logistics and human resources, all of which require various continuous numerical predictions.

[0003] Deep regression is a method for solving regression tasks through deep learning models. Deep learning is favored for its powerful ability to handle large-scale, complex data sets and learn implicit patterns in the data. However, since the sample distribution of regression tasks is usually continuous and unbalanced, such as age, electricity consumption, physiological indicators, etc., a large amount of training data is usually required, and its effect is often lower than other task types. In order to deal with this problem, a variety of methods have been proposed, such as data augmentation, adding regularization terms, using more appropriate loss functions, and choosing better network architectures. Although these methods can improve the effect, there is still room for improvement in deep regression problems. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a deep regression method, device, equipment and medium based on angle compensation that can be applied to machine learning or other related fields, aiming to solve the problem that the overall effect of the regression task is poor due to the fact that the sample distribution of the existing regression task is usually continuous and unbalanced.

[0005] The technical solution of the present invention is as follows:

[0006] The present invention provides a depth regression method based on angle compensation, which comprises:

[0007] Create a basic regression model consisting of an encoder, projection layer, and prediction layer;

[0008] The basic regression model is trained by an angle compensation loss function to obtain a depth regression model;

[0009] The validation set data is input into the deep regression model to verify the model performance.

[0010] Optionally, the step of creating a basic regression model including an encoder, a projection layer, and a prediction layer comprises:

[0011] Using a neural network model to create the encoder that converts raw input data into a high-dimensional vector;

[0012] Creating the projection layer that maps the high-dimensional vector to a low-dimensional vector;

[0013] Creating the prediction layer that maps the low-dimensional vector to a final regression value;

[0014] The basic regression model is generated according to the encoder, the projection layer and the prediction layer.

[0015] Optionally, the step of training the basic regression model by using an angle compensation loss function to obtain a depth regression model includes:

[0016] Correct the negative samples of the original input data through the angle compensation algorithm and calculate the corresponding angle compensation loss function;

[0017] The basic regression model is trained using the angle compensation loss function to obtain the depth regression model.

[0018] Optionally, the step of correcting the negative samples of the original input data by an angle compensation algorithm and calculating the corresponding angle compensation loss function includes:

[0019] Calculating the cosine similarity corresponding to the negative sample, and correcting the negative sample;

[0020] The cosine similarity and the pre-calculated positive sample similarity are calculated to obtain an angle compensation contrast loss function.

[0021] Optionally, the step of using the angle compensation loss function to train the basic regression model to obtain the depth regression model includes:

[0022] Calculate the total loss function by using the mean square error of the basic regression model and the angle compensation contrast loss function;

[0023] The basic regression model is trained by the total loss function to obtain the deep regression model.

[0024] Optionally, the step of training the basic regression model by using the total loss function to obtain the deep regression model includes:

[0025] Using a gradient descent algorithm to minimize the total loss function and train the basic regression model;

[0026] Repeat at least 2 rounds of training until the optimal deep regression model is obtained.

[0027] Optionally, the step of creating a basic regression model including an encoder, a projection layer, and a prediction layer also includes:

[0028] Get the original input data and generate data training pairs;

[0029] The data training pairs are preprocessed, and the data training pairs are divided into a training set, a validation set, and a test set according to a preset ratio.

[0030] On the other hand, this embodiment further provides a depth regression device based on angle compensation, comprising:

[0031] The basic regression model module is used to create a basic regression model including an encoder, a projection layer, and a prediction layer;

[0032] A regression model training module, used to train the basic regression model through an angle compensation loss function to obtain a deep regression model;

[0033] The deep regression model verification module is used to input the verification set data into the deep regression model to verify the model performance.

[0034] Optionally, the basic regression model module includes:

[0035] An encoder submodule, for creating the encoder that converts raw input data into a high-dimensional vector using a neural network model;

[0036] A projection layer submodule, used to create the projection layer that maps the high-dimensional vector to a low-dimensional vector;

[0037] A prediction layer submodule, for creating the prediction layer that maps the low-dimensional vector to a final regression value;

[0038] A basic regression model submodule is used to generate the basic regression model according to the encoder, the projection layer, and the prediction layer.

[0039] Optionally, the regression model training module includes:

[0040] Angle compensation loss function calculation submodule, used to correct the negative samples of the original input data through the angle compensation algorithm and calculate the corresponding angle compensation loss function;

[0041] The regression model training submodule is used to train the basic regression model using the angle compensation loss function to obtain the depth regression model.

[0042] Optionally, the angle compensation loss function calculation submodule is specifically used to calculate the cosine similarity corresponding to the negative sample, correct the negative sample, calculate the cosine similarity and the similarity of the pre-calculated positive sample, and obtain the angle compensation contrast loss function.

[0043] Optionally, the regression model training submodule is specifically used to calculate the total loss function through the mean square error of the basic regression model and the angle compensation contrast loss function, and train the basic regression model through the total loss function to obtain the depth regression model.

[0044] Optionally, the regression model training submodule is further used to minimize the total loss function using a gradient descent algorithm and train the basic regression model, repeating at least 2 rounds of training until an optimal deep regression model is obtained.

[0045] Optionally, it also includes:

[0046] An acquisition module is used to obtain original input data and generate data training pairs;

[0047] The preprocessing module is used to preprocess the data training pairs and divide the data training pairs into a training set, a validation set and a test set according to a preset ratio.

[0048] On the other hand, this embodiment further provides a depth regression device based on angle compensation, the device comprising at least one processor; and

[0049] a memory communicatively connected to the at least one processor; wherein,

[0050] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above-mentioned depth regression method based on angle compensation.

[0051] On the other hand, this embodiment further provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors can execute the above-mentioned depth regression method based on angle compensation.

[0052] Beneficial effect: The present invention discloses a deep regression method, device, equipment and medium based on angle compensation. Compared with the prior art, the present invention creates a basic regression model including an encoder, a projection layer and a prediction layer, and then trains the basic regression model through an angle compensation loss function to obtain a deep regression model. Finally, the validation set data is input into the deep regression model to verify the model performance. This solves the existing technical problem that the sample distribution of regression tasks is usually continuous and unbalanced, such as age, electricity consumption, physiological indicators, etc., and therefore usually requires a large amount of training data, and the effect is often lower than that of other task types. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0054] Figure 1 A flow chart of a depth regression method based on angle compensation provided by an embodiment of the present invention;

[0055] Figure 2 A schematic diagram of the functional modules of the device for automatically generating test data provided by an embodiment of the present invention;

[0056] Figure 3 A schematic diagram of the hardware structure of a device for automatically generating test data provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The embodiments of the present invention are described below in conjunction with the accompanying drawings.

[0058] The angle compensation training regression model mentioned in the present invention is a method using angle compensation supervised contrast loss, which optimizes the performance of the model by considering the angle difference between samples, so that positive samples are closer in the feature space, while negative samples from different categories are farther away. Therefore, the angle compensation supervised contrast loss is used to achieve this goal by considering the angle difference between sample feature vectors.

[0059] The current regression models are: Linear Regression: The most basic regression model, assuming that there is a linear relationship between the input features and the output variables. Polynomial Regression: An extension of linear regression that allows the model to learn high-order terms and interaction terms of features. Ridge Regression: Dealing with the overfitting problem in linear regression by introducing the L2 regularization term. Lasso Regression: By introducing the L1 regularization term, it can not only reduce overfitting but also achieve feature selection. Elastic Net: Combining the characteristics of Ridge Regression and Lasso Regression, the model complexity is controlled by adjusting the ratio of L1 and L2 regularization terms. Decision Tree Regression: Uses a tree graph model to predict continuous values. Random Forest Regression: Ensembles multiple decision trees to improve the accuracy and stability of predictions by voting or averaging. Gradient Boosting Trees: Minimizes the loss function by iteratively adding decision trees. Support Vector Regression (SVR): A regression version of the support vector machine that predicts continuous values ​​by maximizing the margin. Neural Network Regression: Uses a multi-layer neural network to learn complex nonlinear relationships. Their regression task evaluation usually uses the following metrics: Mean Squared Error (MSE): The average of the squares of the difference between the predicted value and the actual value. Root Mean Squared Error (RMSE): The square root of the MSE, expressing the error in units of the actual value. Mean Absolute Error (MAE): The average of the absolute values ​​of the difference between the predicted value and the actual value. R-squared (R-squared, R 2 ): It indicates the proportion of the variation explained by the model to the total variation.

[0060] It is precisely because in the regression task, when there is a large difference between different negative samples and anchor points, such as the anchor point corresponds to the age of 20, then 21 and 81 will be considered negative samples, but the distance between them and the anchor point is very different. The above regression model has an overall poor effect in processing regression tasks.

[0061] In order to solve the above-mentioned problems, the present invention proposes a depth regression method based on angle compensation, see Figure 1 , Figure 1The present invention provides a flowchart of an embodiment of a depth regression method based on angle compensation.

[0062] The angle-compensated depth regression method provided in this embodiment is applied to mobile devices, fixed terminal devices, etc. based on artificial intelligence, and its terminal operating system may include a handheld device operating system (iPhone operating system, iOS system), an Android system, or other operating systems, including but not limited to smart phones, tablet computers, portable computers, and desktop servers, etc. Figure 1 As shown, the method specifically comprises the following steps:

[0063] S101, creating a basic regression model including an encoder, a projection layer and a prediction layer;

[0064] This embodiment can be applied in multiple fields such as computer vision, natural language processing, medical artificial intelligence, etc. When it is necessary to solve the large differences between different negative samples and anchor points, the original input data is first obtained, and a data training pair is generated, the data training pair is preprocessed, and the data training pair is divided into a training set, a validation set, and a test set according to a preset ratio.

[0065] It is understandable that for different application scenarios and different application requirements, data sets are created specifically to form data training pairs such as (x, y). After all data are preprocessed accordingly, they are divided into training sets, validation sets, and test sets according to a certain ratio.

[0066] Further, create and process data sets for machine learning model training: Data collection: Financial field: collect data such as stock prices, trading volume, macroeconomic indicators, etc. Medical field: collect patient medical records, genetic data, drug dosage and other information. Marketing: collect sales data, customer information, market activity data, etc. Other fields: collect corresponding data according to field requirements, such as meteorological data, traffic flow, energy consumption, etc. Then data cleaning is required: Handle missing values: fill or delete missing data. Identify and handle outliers: identify and handle abnormal data points through statistical analysis or visualization methods. Data type conversion: ensure that all data is of the correct data type, such as date and time format, encoding of categorical variables, etc. Feature engineering: Feature selection: select the features that are most useful for predicting the target. Feature construction: create new features, such as year, month, day extracted from the date. Feature scaling: standardize or normalize numerical features to eliminate the impact of dimension. Data set division: Training set: data used to train the model. Validation set: data used for model selection and hyperparameter adjustment. Test set: data used to finally evaluate the performance of the model. For example, the ratio can be 70% training set, 15% validation set, and 15% test set, but this ratio can be adjusted according to specific circumstances.

[0067] The aforementioned data preprocessing: Encode categorical variables, such as One-Hot Encoding. Normalize or standardize continuous variables to have zero mean and unit variance.

[0068] In actual operations, adjustments may be required based on specific circumstances. For example, in the financial field, the characteristics of time series data may need to be considered; in the medical field, data privacy and security may need to be considered. In addition, data in different fields may require specific preprocessing techniques, such as convolutional neural network (CNN) preprocessing of image data or word embedding processing of text data.

[0069] After data preprocessing, it is necessary to create a basic regression model consisting of an encoder, a projection layer, and a prediction layer.

[0070] Specifically, the steps are as follows:

[0071] Using a neural network model to create the encoder that converts raw input data into a high-dimensional vector;

[0072] Creating the projection layer that maps the high-dimensional vector to a low-dimensional vector;

[0073] Creating the prediction layer that maps the low-dimensional vector to a final regression value;

[0074] The basic regression model is generated according to the encoder, the projection layer, and the prediction layer.

[0075] It can be understood that the model mainly consists of three parts: encoder f θ (·), projection layer Proj(·), and prediction layer Pred(·). The encoder encodes the input data into a high-dimensional vector. Its architecture varies depending on the input. Common encoders include ResNet (Residual Network), Bert model (Bidirectional Encoder Representations from Transformers, pre-trained language representation model), etc. The projection layer Proj(·) maps the high-dimensional features extracted by the encoder into a low-dimensional vector, usually a linear layer, and sometimes a multi-layer perceptron. The prediction layer Pred(·) maps the extracted features into the required estimated regression values, usually also a linear layer.

[0076] Encoder: Function: The encoder is the first part of the model, and its task is to convert the raw input data (such as images, text, or sensor data) into a high-dimensional feature representation. This feature representation should capture the key information and patterns of the input data. Architecture: The encoder can adopt different neural network architectures, depending on the type of input data. For example: For image data, commonly used encoders include convolutional neural networks (CNNs), such as ResNet models (Residual Networks), etc. For text data, commonly used encoders include BERT models or other pre-trained language models.

[0077] Parameters: The parameters θ of the encoder, which represent the weights and biases in the network, are learned during training.

[0078] Projection layer: Function: The purpose of the projection layer (also called dimensionality reduction layer or bottleneck layer) is to map the high-dimensional feature vector produced by the encoder to a lower-dimensional space. This helps reduce the complexity of the data while retaining the most important information. Implementation: The projection layer is usually one or more fully connected layers (linear layers), and sometimes a multi-layer perceptron (MLP), depending on the degree of dimensionality reduction required and the model design. Purpose: Dimensionality reduction can improve the generalization ability of the model, reduce the risk of overfitting, and reduce computational complexity.

[0079] Prediction layer: Function: The prediction layer is the last part of the model, which maps the low-dimensional feature vector output by the projection layer to the final regression value. In regression tasks, this regression value is the predicted output of the model, such as house prices, temperature, sales, etc. Implementation: The prediction layer is usually one or more fully connected layers, and the final output layer usually has only one neuron and uses a linear activation function (for continuous value prediction). Output: The output of the prediction layer is the model's predicted value for the input data, which is obtained by learning the patterns and relationships in the data.

[0080] S102, training the basic regression model through an angle compensation loss function to obtain a depth regression model;

[0081] The basic regression model is trained by the angle compensation loss function to obtain the deep regression model. The negative samples of the original input data can be corrected by the angle compensation algorithm, and the corresponding angle compensation loss function is calculated; the basic regression model is trained using the angle compensation loss function to obtain the deep regression model.

[0082] It can be understood that the aforementioned correction of the negative samples of the original input data by the angle compensation algorithm and the calculation of the corresponding angle compensation loss function are further as follows:

[0083] Calculating the cosine similarity corresponding to the negative sample, and correcting the negative sample;

[0084] The cosine similarity and the pre-calculated positive sample similarity are calculated to obtain an angle compensation contrast loss function.

[0085] Cosine similarity for negative samples The correction is calculated as follows:

[0086]

[0087] where y i represents the label of sample i, max(Y) represents the maximum value of the label, min(Y) represents the minimum value of the label, z i represents the output vector of sample i after the aforementioned projection layer, represents the output vector of the data set with different labels from sample i in the same training batch after the projection layer, is the vector z i and The angle between φ and φ is an auxiliary angle used to adjust or correct The estimate, Indicates the cosine value of the auxiliary angle φ, y m and i is a measure of two samples or observations.

[0088] The function of this formula is to convert y m and i The difference between is normalized to the interval [0, 1], and then mapped to the interval [0, π]. This mapping can be used to process periodic data, such as angles or time (hours, months, etc.), such as in angle estimation, time series analysis, or any scenario where data needs to be mapped to a fixed range. In this way, it can be ensured that the periodic characteristics of the data are properly handled, resulting in better performance in subsequent analysis or model training.

[0089] In machine learning, especially in tasks involving angle estimation, negative samples usually refer to those samples that do not match the target category. In angle estimation, It indicates the predicted angle The cosine value of , where i represents the sample index and m represents a specific angle or direction.

[0090] When training a model, we usually want the model to give high confidence predictions for positive samples (i.e. samples that match the target class) and low confidence predictions for negative samples. In angle estimation, this may mean: for positive samples, The value of is close to 1, indicating that the predicted angle is very close to the true angle. For negative samples, The value of is close to -1 or far away from 1, which means that the predicted angle is far from the true angle.

[0091] When calculating the loss function, we need to consider how to use The value of is used to distinguish positive and negative samples, thereby guiding the model to learn the correct perspective. For example, a loss function based on cosine similarity may impose a larger penalty on negative samples to encourage the model to distinguish them from positive samples.

[0092] A simple example is the use of contrastive loss, which explicitly penalizes the model for predicting negative samples that are too close to positive samples.

[0093] The aforementioned Contrastive Loss is a loss function used in the field of machine learning and computer vision, especially in unsupervised learning and self-supervised learning tasks. Its goal is to bring similar samples closer together while pushing dissimilar samples further apart. This loss function helps the model learn feature representations that distinguish different samples during training. Contrastive loss can be used in the following scenarios: Metric Learning: Learning the distance metric between samples so that similar samples are closer in feature space and dissimilar samples are farther apart. Clustering: Clustering samples by learning feature representations without explicit labels. Face Recognition: Identifying or verifying identities by comparing feature representations of face images. Recommendation System: Recommending related items by learning the similarity between users and items. The application of Contrastive Loss in supervised learning usually relies on the definition of positive and negative pairs. Positive pairs refer to pairs of samples belonging to the same category, while negative pairs refer to pairs of samples belonging to different categories. By minimizing the distance between positive pairs and maximizing the distance between negative pairs, Contrastive Loss helps the model learn feature representations that distinguish different categories.

[0094] In practical applications, contrastive loss can be combined with other types of loss functions, such as classification loss (such as cross entropy loss), to ensure that the model can achieve good classification performance while improving the quality of feature representation. The introduction of contrastive loss provides an effective feature learning method for supervised learning tasks, which helps to improve the performance of the model on various complex tasks.

[0095] In practical applications, how to define and handle negative samples depends on the specific task and dataset. For example, when training a head pose estimation model, if the goal is to estimate the angle of the head relative to the body, then the negative samples may include all non-target head angles. In this case, the loss function needs to be designed to distinguish the target head angle from other angles.

[0096] Then, the cosine similarity and the pre-calculated positive sample similarity are calculated to obtain an angle compensation contrast loss function, the calculation formula of which can be:

[0097]

[0098] Among them, It represents the data set with the same label as sample i in the same training batch, which is the positive sample set of sample i, that is, the sample set similar to or of the same type as sample i. It represents the data set with different labels from sample i in the same training batch, z i represents the output vector of sample i after the projection layer described in the previous section, and τ represents the temperature coefficient, which is a pre-set hyperparameter. i is the loss for sample i. p and z k is the feature vector of a sample in the set p(i). is the estimated angle between sample i and negative sample m.

[0099] It is understandable that the above formula Represents the similarity between sample i and positive sample p, which is adjusted by the dot product of the feature vector and the temperature parameter τ. The sum of similarities between sample i and all positive samples. It is the sum of the similarities between sample i and all negative samples, using cosine similarity to measure the angle. The negative logarithmic softmax form of the entire score encourages the model to increase the similarity of positive samples p while reducing the similarity of negative samples N(i).

[0100] The training goal is to minimize the loss L for each sample i i , by adjusting the model parameters to maximize the similarity between positive samples and minimize the similarity between negative samples. This is usually achieved through gradient descent or its variants.

[0101] This loss function is particularly suitable for tasks that require automatic learning of feature representations from data, such as clustering, anomaly detection, or recommendation systems. In this way, the model can learn the intrinsic structure of the data without explicit label information.

[0102] It should be noted that the loss function based on contrastive learning is used to learn the feature representation of samples so that similar positive samples are closer in the feature space and dissimilar negative samples are farther away. This type of loss function is used in self-supervised learning or unsupervised learning scenarios.

[0103] Therefore, in addition to the contrast loss function, in self-supervised learning or unsupervised learning scenarios, triplet loss can also be used: used for metric learning, especially in face recognition and image retrieval tasks. It consists of an anchor sample, a positive sample (similar to the anchor), and a negative sample (dissimilar to the anchor). Triplet loss encourages the distance between the anchor and the positive sample to be smaller than the distance between the anchor and the negative sample. Clustering Loss: used for clustering tasks such as K-means or spectral clustering. In self-supervised learning, loss functions can be designed to encourage the model to group data points into meaningful clusters. Reconstruction Loss: used for generative models such as autoencoders. In unsupervised learning, the model is trained to reconstruct the input data, and the reconstruction loss (such as mean squared error) is used to measure the difference between the reconstructed data and the original data. Feature Matching Loss: In self-supervised learning, feature matching loss can be used to encourage the model to learn task-related feature representations. For example, in visual tasks, loss functions can be designed by comparing feature maps under different views or enhancements. Prediction Loss: In some self-supervised learning tasks, certain attributes of the data or future states can be predicted. For example, in video prediction tasks, the model needs to predict the image content of the next frame. Adversarial Loss: In generative adversarial networks (GANs), adversarial losses are used to train generators and discriminators to generate realistic data. Information Loss: Used for contrastive learning, especially when dealing with data points within a batch. It encourages the model to increase the similarity of positive sample pairs and reduce the similarity of negative sample pairs. Entropy Loss: In some self-supervised learning tasks, the loss function can be designed to minimize the entropy of the prediction, thereby encouraging the model to learn more deterministic feature representations.

[0104] Furthermore, the aforementioned step of using the angle compensation loss function to train the basic regression model to obtain the depth regression model may be:

[0105] Calculate the total loss function by using the mean square error of the basic regression model and the angle compensation contrast loss function;

[0106] The basic regression model is trained by the total loss function to obtain the deep regression model.

[0107] It can be understood that the loss function and the mean square error loss used to optimize the regression effect together constitute the overall loss function of the model as follows:

[0108] L=Lmse +γL scl ;

[0109] Where γ represents the weight coefficient of the loss function, L mse It represents the mean square error between the model prediction result and the actual result, L scl L represents the above angle compensation supervised contrast loss. mse Mean square error loss, Among them, y i is the true value of sample i, is the model prediction value for sample i, and n is the number of samples.

[0110] It can be further understood that the aforementioned step of training the basic regression model through the total loss function to obtain the deep regression model in this embodiment includes: using a gradient descent algorithm to minimize the total loss function and training the basic regression model; repeating at least 2 rounds of training until the optimal deep regression model is obtained.

[0111] The model can be trained using the gradient descent method. During the training process, after each round of training, the model is tested on the validation set, and the model with the best performance on the validation set is selected as the final model.

[0112] Gradient descent is an optimization algorithm used to train models in machine learning and deep learning. It minimizes the loss function by iteratively adjusting model parameters. Initialization parameters:

[0113] Randomly initialize the model parameters (weights and biases), which are usually represented as vectors. Forward propagation: Calculate the predicted output of the model. During training, this involves passing the input data through the various layers of the model to calculate the final prediction. Calculate loss:

[0114] The difference between the model prediction and the true label is calculated using a loss function. The choice of loss function depends on the specific task (for example, regression tasks may use mean squared error). The loss calculation has been described in detail above and will not be repeated here. Backpropagation: Calculate the gradient of the loss function with respect to each parameter. This process is implemented using the chain rule, starting from the output layer and calculating backward layer by layer until the input layer. The gradient is the derivative of the loss function on the parameter and points to the direction where the loss increases fastest. Update parameters: Update the parameters of the model based on the calculated gradient and a pre-set learning rate (a positive scaling factor). Iteration process: Repeat the above method until the stopping condition is met, such as reaching the maximum number of iterations, the loss value is below a certain threshold, or the loss no longer decreases significantly. During the training process, the learning rate may need to be adjusted. For example, the learning rate can be reduced over time, or an adaptive learning rate algorithm (such as Adam) can be used. Convergence: When the model parameter update no longer significantly reduces the loss function value, the model can be considered to have converged.

[0115] It is understood that the gradient descent method also includes variants as follows: Batch Gradient Descent: Use the entire training set for each update. Stochastic Gradient Descent (SGD): Use only one training example for each update. Mini-batch Gradient Descent: Use a small batch of training examples for each update.

[0116] In practical applications, the gradient descent method faces problems such as local minima, saddle points or gradient vanishing / exploding. To solve these problems, strategies such as using momentum, adaptive learning rate algorithms (such as AdaGrad, RMSProp, Adam) and regularization techniques can be used to solve them.

[0117] In one embodiment, in addition, during the training process, the model will undergo multiple data augmentations Aug(·) to calculate the loss function. By applying a series of random transformations to the training data set during the training process, new data points are generated, thereby increasing the diversity and quantity of the data set, helping the model learn more generalized features, and improving the generalization ability and robustness of the model.

[0118] The main purpose of data augmentation is to generate new training samples by transforming or perturbing the original data without increasing the actual amount of data, thereby improving the generalization ability and robustness of the model. Data augmentation methods can be divided into the following categories: Unsupervised data augmentation methods: GAN (Generative Adversarial Networks): Learn the distribution of data through the model and randomly generate pictures that are consistent with the distribution of the training data set. AutoAugment: Learn a data augmentation method suitable for the current task through the model. Standard data augmentation: refers to some commonly used data augmentation methods in the early stage or earlier of deep learning, such as image transformation and image cropping. Image transformation: A set of transformation combinations searched based on NAS. Image cropping: Data augmentation methods similar to dropout, etc. GridMask: Generate a mask with the same resolution as the original image, randomly flip the mask, and multiply it with the original image to obtain the augmented image. Graph data augmentation: A data augmentation technique used in graph neural networks that increases the amount of training data by creating reasonable variants of existing data. NLP data augmentation methods: In natural language processing tasks, data augmentation methods include paraphrase, noise, and sampling. Diffusion model-based data augmentation: Combine the traditional empirical risk minimization method with the diffusion model-based data augmentation method to achieve better domain generalization effect.

[0119] The choice of data augmentation technology should be determined according to the actual application scenario. After data augmentation, the augmented data should be mixed with the original data, otherwise the model may overfit during testing.

[0120] In practical applications, data augmentation techniques can help training models better perceive different parts of an image and improve the model's generalization ability on unseen data. At the same time, data augmentation techniques can also be used together with regularization techniques to provide additional stability for the model.

[0121] In short, data augmentation is an important means to improve model performance and generalization ability in deep learning. By rationally selecting and applying data augmentation techniques, the training effect and application of the model can be effectively improved.

[0122] S103, inputting validation set data into the deep regression model to verify model performance.

[0123] After the model is trained, the obtained model is tested on the validation set. Different test conditions are set according to the actual usage scenario. When the target of interest is the absolute majority of samples, such as age estimation, then the test set can be directly validated. However, if the target of interest is samples in a specific range, such as the business target of interest is outlier detection, such as prediction of certain physiological indicators, balanced sampling and post-testing are required.

[0124] During the training and testing of machine learning models, it is indeed necessary to set the test conditions according to the needs of the actual application scenario. The following are test strategies for different targets of interest: Testing for most samples: If the goal is to evaluate the performance of the model on most samples, such as age estimation, you can test it directly on the validation set. All available validation data should be used to evaluate the overall performance of the model, including indicators such as accuracy, recall, F1 score, and mean square error. Testing for samples in a specific range: If the goal of interest is samples in a specific range, such as outlier detection or prediction of specific physiological indicators, additional steps may be needed to ensure that the test data is representative of these specific samples. Balanced sampling: Since outliers or samples under specific conditions may be rare in the dataset, oversampling or undersampling may be required to balance the dataset in order to better evaluate the performance of the model on these specific samples. Synthetic data: In some cases, generative models (such as GANs) can be used to synthesize outlier samples to increase the proportion of these samples in the test set. Setting of test conditions: Data preprocessing: Ensure that the preprocessing method of the test data is consistent with the training data, including normalization, standardization, etc. Evaluation metric selection: Choose the appropriate evaluation metric based on business needs. For example, for anomaly detection, you may pay more attention to the recall rate of the model rather than the precision rate, because missing a true anomaly may be more serious than misclassifying a normal sample as an anomaly. Model interpretability: For a specific range of samples, the interpretability of the model is also important. You can use model interpretability tools to understand why the model makes specific predictions for certain samples. Post-test analysis: Performance analysis: Analyze the performance of the model on different types of samples and identify the strengths and weaknesses of the model. Error analysis: Conduct in-depth analysis of samples that the model predicts incorrectly to understand the cause of the error, whether it is due to data quality issues, insufficient model training, or insufficient features. Model adjustment: Adjust the model based on the test results, which may include retraining, adjusting the model structure, adding or reducing features, etc. Monitoring after model deployment: Continuous monitoring: Even after the model is deployed, it is necessary to continuously monitor the performance of the model to ensure that it remains stable in actual applications. Feedback loop: Establish a feedback mechanism to feed back the performance of the model in actual applications to the model iteration process so as to continuously optimize the model.

[0125] In short, when testing a model, it is necessary to consider the specific needs of the actual application scenario and design a test strategy and evaluate the performance of the model based on these needs. For samples in a specific range, special sampling and evaluation methods are required to ensure the reliability and effectiveness of the model.

[0126] The deep regression method based on angle compensation provided in this embodiment creates a basic regression model including an encoder, a projection layer and a prediction layer, and then trains the basic regression model through an angle compensation loss function to obtain a deep regression model. Finally, verification set data is input into the deep regression model to verify the model performance. This solves the existing technical problem that the sample distribution of regression tasks is usually continuous and unbalanced, such as age, electricity consumption, physiological indicators, etc., which usually requires a large amount of training data and the effect is often lower than that of other task types.

[0127] At the same time, the cosine similarity is calculated with the pre-calculated positive sample similarity to obtain the angle compensation contrast loss function, and the negative samples are corrected, so that the samples with large differences are further eliminated, so that the positive samples of the training model have high accuracy.

[0128] Furthermore, by calculating the total loss function through the mean square error of the basic regression model and the angle compensation contrast loss function, the contrast loss and the regression loss can be optimized at the same time.

[0129] Furthermore, by calculating the cosine similarity corresponding to the negative sample and correcting the negative sample, it is possible to improve the original data regression effect by introducing the proposed angle compensation supervised contrast loss on the original basis without changing the main structure of the original regression model. And it can be applied to a variety of regression tasks, including but not limited to natural language processing and computer vision.

[0130] Another embodiment of the present invention provides a depth regression device based on angle compensation, such as Figure 2 As shown, the device comprises:

[0131] A basic regression model module 21, used to create a basic regression model including an encoder, a projection layer and a prediction layer;

[0132] A regression model training module 22, used to train the basic regression model through an angle compensation loss function to obtain a depth regression model;

[0133] The deep regression model verification module 23 is used to input the verification set data into the deep regression model to verify the model performance.

[0134] The angle-compensated deep regression device provided in this embodiment creates a basic regression model including an encoder, a projection layer, and a prediction layer, and then trains the basic regression model through an angle compensation loss function to obtain a deep regression model. Finally, verification set data is input into the deep regression model to verify the model performance. This solves the existing technical problem that the sample distribution of regression tasks is usually continuous and unbalanced, such as age, electricity consumption, physiological indicators, etc., and therefore usually requires a large amount of training data, and the effect is often lower than that of other task types.

[0135] At the same time, the cosine similarity is calculated with the pre-calculated positive sample similarity to obtain the angle compensation contrast loss function, and the negative samples are corrected, so that the samples with large differences are further eliminated, so that the positive samples of the training model have high accuracy.

[0136] Furthermore, by calculating the total loss function through the mean square error of the basic regression model and the angle compensation contrast loss function, the contrast loss and the regression loss can be optimized at the same time.

[0137] Furthermore, by calculating the cosine similarity corresponding to the negative sample and correcting the negative sample, it is possible to improve the original data regression effect by introducing the proposed angle compensation supervised contrast loss on the original basis without changing the main structure of the original regression model. And it can be applied to a variety of regression tasks, including but not limited to natural language processing and computer vision.

[0138] Another embodiment of the present invention provides a depth regression device based on angle compensation, such as Figure 3 As shown, the device 10 includes:

[0139] One or more processors 110 and memory 120, Figure 3 A processor 110 is used as an example for description. The processor 110 and the memory 120 may be connected via a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0140] The processor 110 is used to complete various control logics of the device 10, and it can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISCMachine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of these components. In addition, the processor 110 can also be any traditional processor, microprocessor or state machine. The processor 110 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0141] The memory 120, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions corresponding to the depth regression method based on angle compensation in the embodiment of the present invention. The processor 110 executes various functional applications and data processing of the device 10 by running the non-volatile software programs, instructions and units stored in the memory 120, that is, implementing the depth regression method based on angle compensation in the above method embodiment.

[0142] The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created according to the use of the device 10, etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 120 may optionally include a memory remotely arranged relative to the processor 110, and these remote memories may be connected to the device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0143] One or more units are stored in the memory 120, and when executed by one or more processors 110, the following steps are implemented:

[0144] Create a basic regression model consisting of an encoder, projection layer, and prediction layer;

[0145] The basic regression model is trained by an angle compensation loss function to obtain a depth regression model;

[0146] The validation set data is input into the deep regression model to verify the model performance.

[0147] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the above steps S101 to S103 are implemented.

[0148] As an example, non-volatile storage media can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) as external cache memory. By way of illustration and not limitation, RAM can be obtained in many forms such as synchronous RAM (SRAM), dynamic RAM, (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The disclosed memory components or memories of the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.

[0149] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium, and the computer program can include the processes of the above-mentioned method embodiments when executed. The storage medium can be a memory, a disk, a floppy disk, a flash memory, an optical storage device, etc.

[0150] Compared with the existing technology, the present invention creates a basic regression model including an encoder, a projection layer and a prediction layer, and then trains the basic regression model through an angle compensation loss function to obtain a deep regression model. Finally, the validation set data is input into the deep regression model to verify the model performance. This solves the existing technical problems that the sample distribution of regression tasks is usually continuous and unbalanced, such as age, electricity consumption, physiological indicators, etc., and therefore usually requires a large amount of training data, and the effect is often lower than that of other task types.

[0151] At the same time, the cosine similarity is calculated with the pre-calculated positive sample similarity to obtain the angle compensation contrast loss function, and the negative samples are corrected, so that the samples with large differences are further eliminated, so that the positive samples of the training model have high accuracy.

[0152] Furthermore, by calculating the total loss function through the mean square error of the basic regression model and the angle compensation contrast loss function, the contrast loss and the regression loss can be optimized at the same time.

[0153] Furthermore, by calculating the cosine similarity corresponding to the negative sample and correcting the negative sample, it is possible to improve the original data regression effect by introducing the proposed angle compensation supervised contrast loss on the original basis without changing the main structure of the original regression model. And it can be applied to a variety of regression tasks, including but not limited to natural language processing and computer vision.

[0154] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A depth regression method based on angle compensation, characterized in that: include: Create a basic regression model consisting of an encoder, projection layer, and prediction layer; The basic regression model is trained by an angle compensation loss function to obtain a depth regression model; The validation set data is input into the deep regression model to verify the model performance.

2. The depth regression method based on angle compensation according to claim 1, characterized in that: The steps to create a basic regression model consisting of an encoder, projection layer, and prediction layer include: Using a neural network model to create the encoder that converts raw input data into a high-dimensional vector; Creating the projection layer that maps the high-dimensional vector to a low-dimensional vector; Creating the prediction layer that maps the low-dimensional vector to a final regression value; The basic regression model is generated according to the encoder, the projection layer, and the prediction layer.

3. The depth regression method based on angle compensation according to claim 1, characterized in that: The step of training the basic regression model through the angle compensation loss function to obtain the depth regression model includes: Correct the negative samples of the original input data through the angle compensation algorithm and calculate the corresponding angle compensation loss function; The basic regression model is trained using the angle compensation loss function to obtain the depth regression model.

4. The depth regression method based on angle compensation according to claim 3, characterized in that: The steps of correcting the negative samples of the original input data by the angle compensation algorithm and calculating the corresponding angle compensation loss function include: Calculating the cosine similarity corresponding to the negative sample, and correcting the negative sample; The cosine similarity and the pre-calculated positive sample similarity are calculated to obtain an angle compensation contrast loss function.

5. The depth regression method based on angle compensation according to claim 4, characterized in that: The step of training the basic regression model using the angle compensation loss function to obtain the depth regression model comprises: Summing the mean square error of the basic regression model and the angle compensation contrast loss function to obtain a total loss function; The basic regression model is trained by the total loss function to obtain the deep regression model.

6. The depth regression method based on angle compensation according to claim 5, characterized in that: The step of training the basic regression model by using the total loss function to obtain the deep regression model includes: Using a gradient descent algorithm to minimize the total loss function and train the basic regression model; Repeat at least 2 rounds of training until the optimal deep regression model is obtained.

7. The depth regression method based on angle compensation according to claim 1, characterized in that: The steps to create a basic regression model including encoder, projection layer and prediction layer also include: Get the original input data and generate data training pairs; The data training pairs are preprocessed, and the data training pairs are divided into a training set, a validation set, and a test set according to a preset ratio.

8. A depth regression device based on angle compensation, characterized in that: include: The basic regression model module is used to create a basic regression model including an encoder, a projection layer, and a prediction layer; A regression model training module, used to train the basic regression model through an angle compensation loss function to obtain a deep regression model; The deep regression model verification module is used to input the verification set data into the deep regression model to verify the model performance.

9. A depth regression device based on angle compensation, characterized in that: The device comprises at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the angle-compensated depth regression method described in any one of claims 1-7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by one or more processors, the one or more processors can execute the angle-compensated depth regression method described in any one of claims 1-7.

Images