Object positioning model processing, object positioning method, device and computer equipment

Abstract

The application relates to an object positioning model processing method and device, computer equipment and a storage medium, and an object positioning method and device, computer equipment and a storage medium. The object positioning model processing method comprises the following steps: determining a regression region based on the image features of a training sample image and a regression network of an object positioning model; calculating regression accuracy based on a target object label region corresponding to the training sample image, calculating a regression loss based on the regression accuracy; determining a classification confidence based on the image features of the training sample image and a classification network of the object positioning model, calculating a classification loss based on the classification confidence; updating the classification loss based on the regression accuracy, and updating the regression loss based on the classification confidence; training the object positioning model according to the updated classification loss and the updated regression loss to obtain a trained object positioning model; and the trained object positioning model is used for object positioning of an input image. The method can improve positioning accuracy.

Description

[0001] The application claims priority to the Chinese patent application No. 2021100429568, filed on January 13, 2021, and entitled "Object positioning model processing, object positioning method, device and computer equipment", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of image processing, in particular to an object positioning model processing method and device, computer equipment and storage medium, and an object positioning method and device, computer equipment and storage medium. BACKGROUND

[0003] With the rapid development of computer technology and image processing technology, more and more fields involve target object positioning in image processing to perform subsequent applications according to the positioned target object. For example, target recognition technology has very important applications in automatic driving, smart city and smart home.

[0004] In the traditional technology, the classification network and the regression network of the object positioning model are independent of each other, and the region with the highest classification confidence is not always the region with the highest regression accuracy, resulting in low positioning accuracy. SUMMARY

[0005] Therefore, it is necessary to provide an object positioning model processing method and device, computer equipment and storage medium capable of improving positioning accuracy, and an object positioning method and device, computer equipment and storage medium.

[0006] An object positioning model processing method, the method comprising:

[0007] obtaining a training sample image comprising a target object;

[0008] determining a regression region corresponding to the training sample image based on image features of the training sample image and a regression network of an object positioning model;

[0009] calculating a regression accuracy of the regression region based on a target object annotation region corresponding to the training sample image, and calculating a regression loss based on the regression accuracy;

[0010] determining a classification confidence of the regression region based on the image features of the training sample image and a classification network of the object positioning model, and calculating a classification loss based on the classification confidence;

[0011] updating the classification loss based on the regression accuracy, and updating the regression loss based on the classification confidence;

[0012] train the object localization model according to the updated classification loss and the updated regression loss until a training stop condition is met, to obtain a trained object localization model.

[0013] The trained object localization model is configured to perform object localization on an input image.

[0014] An object localization model processing apparatus, the apparatus comprising:

[0015] a training sample acquisition module configured to acquire a training sample image comprising a target object;

[0016] a regression region determination module configured to determine a regression region corresponding to the training sample image based on image features of the training sample image and a regression network of an object localization model;

[0017] a regression loss calculation module configured to calculate a regression accuracy of the regression region based on a target object annotation region corresponding to the training sample image, and calculate a regression loss based on the regression accuracy;

[0018] a classification loss calculation module configured to determine a classification confidence of the regression region based on the image features of the training sample image and a classification network of the object localization model, and calculate a classification loss based on the classification confidence;

[0019] an updating module configured to update the classification loss based on the regression accuracy, and update the regression loss based on the classification confidence;

[0020] a training module configured to train the object localization model according to the updated classification loss and the updated regression loss until a training stop condition is met, to obtain a trained object localization model; wherein the trained object localization model is configured to perform object localization on an input image.

[0021] In one embodiment, the determination of the regression region corresponding to the training sample image based on the image features of the training sample image and the regression network of the object localization model comprises:

[0022] determination of a predicted center point position corresponding to the target object in the training sample image and a predicted size corresponding to the target object based on the image features of the training sample image and the regression network of the object localization model;

[0023] determination of the regression region according to the predicted center point position and the predicted size.

[0024] In one embodiment, the determination of the regression region corresponding to the training sample image based on the image features of the training sample image and the regression network of the object localization model comprises:

[0025] obtaining a position of an anchor box corresponding to the training sample image;

[0026] determining an offset corresponding to the anchor box based on an image feature of the training sample image and a regression network of the object positioning model;

[0027] determining a regression region based on the position of the anchor box and the offset of the anchor box.

[0028] In one of the embodiments, before the object to be recognized is located from the image to be recognized according to the classification confidence of each regression region of the image to be recognized, the method further comprises:

[0029] inputting the first cross-correlation recognition feature into a regression accuracy prediction network of the object positioning model to obtain a regression accuracy prediction value of each regression region of the image to be recognized;

[0030] the object to be recognized is located from the image to be recognized according to the classification confidence of each regression region of the image to be recognized comprises:

[0031] multiplying the regression accuracy prediction value of each regression region of the image to be recognized with the classification confidence to obtain a target positioning score of each regression region of the image to be recognized;

[0032] locating the object to be recognized from the image to be recognized according to the target positioning score of each regression region of the image to be recognized.

[0033] A computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the object positioning model processing method when executing the computer program.

[0034] A computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the object positioning model processing method.

[0035] The aforementioned object localization model, method, apparatus, computer equipment, and storage medium first acquire training sample images including target objects. Based on the image features of the training sample images and the regression network of the object localization model, a regression region corresponding to the training sample images is determined. The regression accuracy of the regression region is calculated based on the target object labeled region corresponding to the training sample images. The regression loss is calculated based on the regression accuracy. The classification confidence of the regression region is determined based on the image features of the training sample images and the classification network of the object localization model. The classification loss is calculated based on the classification confidence. Further, the classification loss is updated based on the regression accuracy, and the regression region is updated based on the classification confidence. The object localization model is then trained using the updated classification and regression losses until the training stopping condition is met. This process involves updating the losses of both the regression and classification networks during training. The regression loss now incorporates classification confidence, while the classification loss incorporates regression accuracy. Therefore, training the object localization model with updated losses reduces the inconsistency between the classification and regression networks. This results in higher classification confidence for more accurate regression regions, and more accurate regression regions, ultimately improving localization accuracy.

[0036] An object location method, the method comprising:

[0037] Obtain the input image;

[0038] A trained object localization model is obtained; the object localization model includes a classification network and a regression network; the object localization model is trained using the target regression loss of the regression network and the target classification loss of the classification network; the target regression loss is obtained by updating the initial regression loss using the classification confidence of the regression region; the target classification loss is obtained by updating the initial classification loss using the regression accuracy of the regression region; the regression region is determined based on the image features of the training sample images and the regression network of the object localization model; the regression accuracy of the regression region is calculated based on the target object annotation region corresponding to the training sample images; the classification confidence of the regression region is determined based on the image features of the training sample images and the classification network; the initial classification loss is calculated based on the classification confidence; the initial regression loss is calculated based on the regression accuracy.

[0039] Based on the image features of the input image and the regression network, multiple regression regions of the input image are determined;

[0040] determine a classification confidence of each regression region of the input image based on image features of the input image and the classification network;

[0041] perform object localization on the input image according to the classification confidence of each regression region of the input image to obtain a region where the target object is located.

[0042] An object localization device, the device comprising:

[0043] an image acquisition module configured to acquire an input image;

[0044] a model acquisition module configured to acquire a trained object localization model; the object localization model comprising a classification network and a regression network; the object localization model being trained by a target regression loss of the regression network and a target classification loss of the classification network; the target regression loss being obtained by updating an initial regression loss by a classification confidence of a regression region; the target classification loss being obtained by updating an initial classification loss by a regression accuracy of the regression region; the regression region being determined based on image features of the training sample image and the regression network of the object localization model; the regression accuracy of the regression region being calculated based on a target object annotation region corresponding to the training sample image; the classification confidence of the regression region being determined based on image features of the training sample image and the classification network; the initial classification loss being calculated based on the classification confidence; the initial regression loss being calculated based on the regression accuracy;

[0045] a regression region determination module configured to determine a plurality of regression regions of the input image based on image features of the input image and the regression network;

[0046] a confidence determination module configured to determine a classification confidence of each regression region of the input image based on image features of the input image and the classification network;

[0047] a localization module configured to perform object localization on the input image according to the classification confidence of each regression region of the input image to obtain a region where the target object is located.

[0048] A computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the above object localization method when executing the computer program.

[0049] A computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the above object localization method.

[0050] The object localization method, apparatus, computer equipment, and storage medium described above, since the object localization model is trained using the target regression loss of the regression network and the target classification loss of the classification network, and during the training process, the losses of the regression network and the classification network are updated respectively, so that the regression loss of the regression network considers the classification confidence, and the classification loss of the classification network considers the regression accuracy. Therefore, after training the target localization model with the updated loss, the inconsistency between the classification network and the regression network is reduced in the trained target localization model, so that the classification confidence of the regression region with higher regression accuracy is higher, and the regression region with higher classification confidence is more accurately regressed, thereby improving the accuracy of localization. Attached Figure Description

[0051] Figure 1 This is a diagram illustrating the object localization model processing method and the application environment of the object localization method in one embodiment.

[0052] Figure 2 This is a flowchart illustrating an object location model processing method in one embodiment;

[0053] Figure 3 This is a schematic diagram of the structure of an object localization model in one embodiment;

[0054] Figure 4 This is a schematic diagram showing the location of the target object's labeled region and a certain regression region in one embodiment;

[0055] Figure 5 This is a schematic diagram of the center point and dimensions in one embodiment;

[0056] Figure 6 This is a flowchart illustrating an object location method in one embodiment;

[0057] Figure 7 This is a flowchart illustrating the object location method in another embodiment;

[0058] Figure 8 This is a schematic diagram illustrating the output of an object localization model used for target recognition in one embodiment.

[0059] Figure 9 This is an overall flowchart of the object location method in one embodiment;

[0060] Figure 10 This is a schematic diagram illustrating the result of human body recognition in one embodiment.

[0061] Figure 11 This is a structural block diagram of an object localization model processing device in one embodiment;

[0062] Figure 12A structural block diagram of an object positioning device in one embodiment;

[0063] Figure 13 An internal structural diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0065] Artificial intelligence (AI) is to use digital computers or digital computer controlled machine simulation, extension and expansion of human intelligence, perception of the environment, acquisition of knowledge and use of knowledge to obtain the best results of theory, method, technology and application system. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making.

[0066] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning and other major directions.

[0067] Computer vision technology (CV) Computer vision is a science that studies how to make machines "see". Further, it refers to using cameras and computers to replace human eyes to identify, identify and measure targets and other machine vision, and further process graphics, so that computer processing becomes more suitable for human eye observation or image transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, trying to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, etc. It also includes common face recognition, fingerprint recognition and other biometric identification technologies.

[0068] Machine Learning (ML) is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is a specialized study of how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structure, and continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rule-based learning.

[0069] With the research and progress of artificial intelligence technology, artificial intelligence technology is researched and applied in many fields, such as common smart home, smart wearable device, virtual assistant, smart speaker, smart marketing, unmanned vehicle, autonomous vehicle, unmanned aerial vehicle, robot, smart medical treatment, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0070] The scheme provided by the embodiments of the present application relates to computer vision and other technologies of artificial intelligence, which is specifically explained by the following embodiments:

[0071] The object positioning model processing method provided by the present application can be applied in the application environment as shown in Figure 1 The terminal and the server can individually execute the object positioning model processing method of the present application, and the terminal and the server can also cooperatively execute the object positioning model processing method of the present application. For example, the terminal first acquires a training sample image containing a target object, sends the training sample image to the server, and the server trains an object positioning model according to the training sample image.

[0072] The terminal 102 communicates with the server 104 through a network. The terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices, and the server 104 can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud storage, network service, cloud communication, big data and artificial intelligence platform, etc. The terminal and the server can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application.

[0073] The object positioning method provided by the present application can also be applied in the application environment as shown in Figure 1The terminal and the server can each independently perform the object positioning model processing method of the present application, or the terminal and the server can cooperatively perform the object positioning model processing method of the present application. For example, the server can send the trained object positioning model to the terminal according to a request of the terminal, and the terminal performs object positioning on the input image based on the trained object positioning model.

[0074] In one embodiment, as shown in Figure 2 An object positioning model processing method is provided, which is taken as an example of being applied to a computer device, which is the terminal 102 or the server 104 in Figure 1 The object positioning model processing method comprises the following steps:

[0075] Step 202, obtaining a training sample image comprising a target object.

[0076] The target object is an object that can be positioned in an image, and can be an independent living being or object, such as a natural person, an animal, a vehicle, a virtual character, etc., or can be a specific part, such as a face, a hand, etc. The training sample image is used for supervised training of the object positioning model. Since it is supervised training, the training sample image has a target object annotation region and an annotation category. The annotation category refers to the category of the target object.

[0077] It should be noted that positioning in the present application refers to determining the region where the target object is located from an image. In one embodiment, positioning can be detecting the target object from a single image, in which case the target object is a detection object. In another embodiment, positioning can be recognizing the target object from a video frame sequence, in which case the target object is a to-be-recognized object.

[0078] Specifically, when the target object is a detection object, the computer device can obtain an image comprising the target object by taking a photo of the target object, and the image comprising the target object can be used as a training sample image after the region where the target object is located is manually annotated. The computer device can also obtain an image comprising the target object and having an annotated region where the target object is located from a third-party computer device through wired or wireless means as a training sample image.

[0079] When the target object is a to-be-recognized object, the computer device can obtain a video frame sequence comprising the target object by video shooting of the target object, and the video frame sequence comprising the target object can be used as a training sample image after the region where the target object is located is manually annotated. The computer device can also obtain a video frame sequence comprising the target object and having an annotated region where the target object is located from a third-party computer device through wired or wireless means, and select a frame of image from the video frame sequence as a training sample image.

[0080] In step 204, the regression network of the object localization model is used to determine the regression region corresponding to the training sample image based on the image features of the training sample image.

[0081] The object localization model refers to a machine learning model that can be used to locate a target object in a given image or video frame sequence. The object localization model includes a regression network and a classification network. The regression network is used to perform boundary regression to determine the regression region corresponding to the training sample image. The regression region refers to the region obtained by performing boundary regression. The regression region can be one or more. The plurality refers to at least two. The classification network is used to classify the image content in the regression region. The classification result can be "existence of target object" or "non-existence of target object". The classification network can be a general network that can be used for classification, such as SVM (support vector machines).

[0082] The image features of the training sample image can include texture features, color features, gradient features, spatial relationship features, etc. The texture features describe the surface properties of each target in the image. The color features describe the color of each target in the image. The gradient features describe the shape and structure of each target in the image. The spatial relationship features refer to the spatial position or relative direction relationship between multiple targets segmented from the image. These relationships can be classified as connection / adjacency relationship, intersection / overlap relationship, and inclusion / containment relationship, etc.

[0083] Specifically, the computer device can extract the image features of the training sample image, and perform boundary regression based on the image features of the training sample image and the regression network of the object localization model, thereby determining the regression region corresponding to the training sample image.

[0084] In one embodiment, the object localization model further includes a feature extraction network. The computer device inputs the training sample image into the feature extraction network of the object localization model to obtain the image features of the training sample image.

[0085] In one embodiment, when the target object is a detection object, the computer device can input the image features into the regression network of the object localization model after extracting the image features of the training sample image to determine the regression region corresponding to the training sample image.

[0086] In another embodiment, when the target object is the to-be-recognized object, the computer device, when acquiring the training sample image including the target object, also simultaneously acquires a training reference image including the same target object as the training sample image, determines a first mutual correlation training feature between the training sample image and the training reference image based on an image feature of the training sample image and an image feature of the training reference image, and inputs the first mutual correlation training feature into the regression network to determine the regression region corresponding to the training sample image.

[0087] In step 206, the regression accuracy of the regression region is calculated based on the target object annotation region corresponding to the training sample image, and the regression loss is calculated based on the regression accuracy.

[0088] The target object annotation region (ground truth, GT) is a region of the target object annotated on the training sample image. The regression accuracy is used to represent the position accuracy of the regression region, and the regression accuracy and the position accuracy are positively correlated, that is, the greater the regression accuracy, the more accurate the position of the regression region. The regression loss is used to represent the difference between the target object annotation region and the regression region, and the greater the difference between the target object annotation region and the regression region, the greater the regression loss.

[0089] Specifically, the computer device can determine the regression accuracy of the regression region based on the coincidence degree between the regression region and the target object annotation region, and further calculate the regression loss based on the regression accuracy.

[0090] It can be understood that when the training sample image corresponds to multiple regression regions, the computer device can determine the regression accuracy of each regression region based on the coincidence degree between each regression region and the target object annotation region. Accordingly, the computer device can calculate the regression loss of each regression region based on the regression accuracy of each regression region.

[0091] In one embodiment, the computer device can acquire an intersection region between the target object annotation region and the regression region, and simultaneously acquire a union region between the target object annotation region and the regression region, and determine the regression accuracy according to the ratio of the intersection region to the union region corresponding to the regression region.

[0092] In one embodiment, the computer device can calculate the regression loss by (1-regression accuracy) after calculating the regression accuracy.

[0093] In step 208, the classification confidence of the regression region is determined based on the image feature of the training sample image and the classification network of the object positioning model, and the classification loss is calculated based on the classification confidence.

[0094] The classification confidence is used to represent the possibility of the target object existing in the regression region. The higher the classification confidence of the regression region is, the greater the possibility of the target object existing in the regression region is, and the more accurate the classification of the regression box is. The classification loss is used to represent the classification accuracy. The classification loss and the classification accuracy are negatively correlated, that is, the higher the classification accuracy is, the smaller the classification loss is.

[0095] Specifically, the computer device extracts the image features of the training sample image, determines the classification confidence of each regression region based on the image features of the training sample image and the classification network of the object positioning model, and further judges whether the regression region is a positive sample or a negative sample. When the regression region is a positive sample, the classification loss is calculated based on the classification confidence and the positive sample label. When the regression region is a negative sample, the classification loss is calculated based on the classification confidence and the negative sample label.

[0096] In one embodiment, when the target object is a detection object, the computer device can input the image features of the training sample image into the classification network of the object positioning model to determine the classification confidence of each regression region corresponding to the training sample image.

[0097] In another embodiment, when the target object is a to-be-recognized object, as described in the above embodiment, the computer device obtains the training reference image including the same target object as the training sample image when obtaining the training sample image including the target object. Then, the computer device can determine the second cross-correlation training features between the training sample image and the training reference image based on the image features of the training sample image and the image features of the training reference image, and input the second cross-correlation training features into the classification network to determine the classification confidence of each regression region corresponding to the training sample image. It can be understood that the second cross-correlation training features and the first cross-correlation training features in the above embodiment are usually different features, and the pixel points at the same positions of the first cross-correlation training features and the second cross-correlation training features correspond to the same features in the training sample image.

[0098] In step 210, the classification loss is updated based on the regression accuracy, and the regression loss is updated based on the classification confidence.

[0099] Specifically, the computer device can multiply the regression accuracy of each regression region with the classification loss of each regression region respectively to update the classification loss of each regression region, thereby establishing regression auxiliary connection, so that the classification confidence of the regression region with higher regression accuracy is higher, and the computer device can multiply the classification confidence of each regression region with the regression loss of each regression region respectively to update the regression loss of each regression region, thereby establishing classification auxiliary connection, so that the regression of the regression region with higher classification confidence is more accurate. Through the regression auxiliary connection and the classification auxiliary connection, the inconsistency between the classification network and the regression network is reduced.

[0100] For example, assuming that a training sample image corresponds to two regression regions A and B, for the regression regions A and B, the loss is updated as shown in Table 1.

[0101] Table 1

[0102] Regression area A Regression area B Regression accuracy X1 X2 Regression loss V1 V2 Classification confidence Y1 Y2 Classification loss W1 W2 Updated regression loss Y1*V1 Y2*V2 Updated classification loss X1*W1 X2*W2

[0103] Step 212, training the object positioning model according to the updated classification loss and the updated regression loss until a training stop condition is met, and obtaining the trained object positioning model.

[0104] Specifically, the computer device can superimpose the updated classification loss and the updated regression loss to obtain a comprehensive loss, and then train the object positioning model according to the comprehensive loss until a training stop condition is met, and obtain the trained object positioning model. In the training process, a stochastic gradient descent algorithm, an Adagrad (Adaptive Gradient) algorithm, an Adadelta (an improvement of the AdaGrad algorithm), an RMSprop (an improvement of the AdaGrad algorithm), an Adam (Adaptive Moment Estimation) algorithm, etc. can be used to adjust the network parameters of the object positioning model. When the training stop condition is met, the training is completed, and the trained object positioning model is obtained. The training stop condition can be that the network parameters no longer change, the loss reaches a minimum value, the number of training reaches a maximum iteration number, etc.

[0105] ​​​​​​The trained object positioning model is used for object positioning on an input image. In an embodiment, after obtaining the input image, the computer device determines a plurality of regression regions of the input image based on image features of the input image and a regression network of the trained object positioning model, and determines classification confidence of each regression region based on image features of the input image and a classification network of the trained object positioning model. The computer device selects a regression region with the highest classification confidence as a target region corresponding to a target object in the input image, that is, a region where the target object is predicted to be located.

[0106] In the object positioning model processing method, first, a training sample image including a target object is obtained, a regression region corresponding to the training sample image is determined based on image features of the training sample image and a regression network of the object positioning model, a regression accuracy of the regression region is calculated based on a target object annotation region corresponding to the training sample image, a regression loss is calculated based on the regression accuracy, a classification confidence of the regression region is determined based on image features of the training sample image and a classification network of the object positioning model, a classification loss is calculated based on the classification confidence, further, the classification loss is updated based on the regression accuracy, and the regression loss is updated based on the classification confidence, finally, the object positioning model is trained according to the updated classification loss and the updated regression loss until a training stop condition is met, and the trained object positioning model is obtained. Since the losses of the regression network and the classification network are updated during the training, the classification confidence is considered in the regression loss of the regression network, and the regression accuracy is considered in the classification loss of the classification network. Therefore, after the object positioning model is trained by the updated losses, the inconsistency between the classification network and the regression network is reduced, the classification confidence of the regression region with high regression accuracy is higher, the regression region with high classification confidence is more accurate, and the positioning accuracy is improved.

[0107] In an embodiment, an object positioning model processing method is provided, including steps 1.1-1.8:

[0108] 1.1, obtaining a training sample image including a target object.

[0109] 1.2, determining a regression region corresponding to the training sample image based on image features of the training sample image and a regression network of the object positioning model.

[0110] 1.3, calculating a regression accuracy of the regression region based on a target object annotation region corresponding to the training sample image, and calculating a regression loss based on the regression accuracy.

[0111] 1.4, determining a classification confidence of the regression region based on image features of the training sample image and a classification network of the object positioning model, and calculating a classification loss based on the classification confidence.

[0112] 1.5, update the classification loss based on the regression accuracy, and update the regression loss based on the classification confidence.

[0113] 1.6, the regression accuracy prediction network based on the regression accuracy of the object positioning model determines the regression accuracy prediction value of the regression region.

[0114] Specifically, in the embodiment, the object positioning model further includes a regression accuracy prediction network, which is configured to predict the regression accuracy of each regression region.

[0115] In one embodiment, when the target object is a detection object, the computer device can input the image features of the training sample image into the regression accuracy prediction network to obtain the regression accuracy prediction value of each regression region.

[0116] In another embodiment, when the target object is an object to be identified, as known from the above embodiment, the computer device can determine the first cross-correlation training features between the training sample image and the training reference image based on the image features of the training sample image and the image features of the training reference image. Then, the computer device can input the first cross-correlation training features into the regression accuracy prediction network to obtain the regression accuracy prediction value of each regression region.

[0117] 1.7, determine the regression accuracy loss based on the regression accuracy prediction value of the regression region and the regression accuracy of the regression region.

[0118] The regression accuracy loss is used to represent the difference between the regression accuracy prediction value and the actual regression accuracy of the regression region. The greater the difference, the greater the regression accuracy loss, and vice versa. The smaller the difference, the smaller the regression accuracy loss.

[0119] 1.8, train the object positioning model according to the regression accuracy loss, the updated classification loss and the updated regression loss.

[0120] Specifically, the computer device can superimpose the regression accuracy loss, the updated classification loss and the updated regression loss to obtain a comprehensive loss, and train the object positioning model based on the comprehensive loss.

[0121] In one embodiment, after obtaining the input image, the computer device can determine a plurality of regression regions of the input image based on the image features of the input image and the regression network of the trained object positioning model, determine regression accuracy prediction values of each regression region based on the image features of the input image and the regression accuracy prediction network of the trained object positioning model, and determine classification confidence of each regression region based on the image features of the input image and the classification network of the trained object positioning model. The classification confidence of each regression region is multiplied by the corresponding regression accuracy prediction value to obtain a target positioning score of each regression region. The regression region with the highest target positioning score is taken as the target region corresponding to the target object in the input image, that is, the predicted region where the target object is located.

[0122] In the above embodiment, the computer device trains the object positioning model by the regression accuracy loss, the updated classification loss and the updated regression loss. On the one hand, the regression loss of the regression network considers the classification confidence, and the classification loss of the classification network considers the regression accuracy, which reduces the inconsistency between the classification network and the regression network. On the other hand, the regression accuracy loss of the regression accuracy prediction network is added to the loss, and the trained object positioning model further considers the regression accuracy, thereby further improving the positioning accuracy.

[0123] In one embodiment, an object positioning model processing method is provided. In this embodiment, the target object is a to-be-identified object, and the object positioning model processing method specifically includes the following steps 2.1-2.11.

[0124] 2.1, obtaining a training sample image including a target object and a training reference image corresponding to the training sample image.

[0125] The training reference image and the training sample image include the same target object.

[0126] In one embodiment, the computer device can obtain an image frame sequence of the region where the labeled target object is located, obtain a frame image in front of time from the image frame sequence as a training reference image, and obtain a frame image after the training reference image as a training sample image. The image frame sequence is a set of a series of images in time sequence, which can be a video frame sequence in a video or a plurality of continuous image frames collected by an image collection device.

[0127] 2.2, obtaining image features of the training sample image and image features of the training reference image based on a feature extraction network of the object positioning model.

[0128] The feature extraction network is a machine learning model trained to have image feature extraction capability. A general machine learning model with image feature extraction capability, such as a ResNet-50 model, a VGG16, a MobileNetV2, etc., can be used as the feature extraction network of the object positioning method provided by the present application, because the image features extracted by the general machine learning model with image feature extraction capability meet the requirements of the object positioning method provided by the present application.

[0129] In one embodiment, since the computer device needs to extract features from the training sample image and the training reference image at the same time, the feature extraction network can adopt a Siamese Network, and in this case, the feature extraction network in the object positioning model is two feature extraction networks with the same model structure and sharing model parameters. In this embodiment, step 2.2 specifically comprises: inputting the training sample image and the training reference image into the two feature extraction networks respectively; and outputting the image features of the training sample image and the image features of the training reference image in parallel and respectively by the two feature extraction networks.

[0130] In one specific embodiment, the image features of the training sample image can be a first image feature map, and the training reference image can be a second image feature map. The resolution of the first image feature map is different from the resolution of the training sample image, and the pixel points in the first image feature map correspond to the pixel points in the training sample image, and the correspondence is related to the feature extraction parameters (such as the size of the convolution kernel, the step, etc.) used by the feature extraction network. Similarly, the resolution of the second image feature map is different from the resolution of the training reference image, and the pixel points in the second image feature map correspond to the pixel points in the training reference image, and the correspondence is also related to the feature extraction parameters used by the feature extraction network.

[0131] Specifically, two feature extraction networks are set, and the two feature extraction networks are completely consistent in model structure and model parameters. The computer device inputs the training sample image and the training reference image into the two feature extraction networks respectively, so that the two feature extraction networks extract the image features of the training sample image and the second image features of the training reference image respectively.

[0132] For example, referring to Figure 3 , Figure 3 is a structural schematic diagram of the object positioning model in one embodiment. It can be seen that the training sample image 302 and the training reference image 304 are input into two feature extraction networks respectively, and the two feature extraction networks output the image features of the training sample image and the image features of the training reference image respectively.

[0133] In this embodiment, two feature extraction networks are used to extract image features in parallel, which improves the efficiency of image feature extraction and further improves the efficiency of target object positioning.

[0134] 2.3, determining the first cross-correlation training feature between the training sample image and the training reference image based on the image feature of the training sample image and the image feature of the training reference image.

[0135] The first cross-correlation training feature is also referred to as the first cross-correlation feature in the above embodiment. Since it is a feature determined in the process of training the object positioning model, it is referred to as the first cross-correlation training feature in this embodiment. Similarly, the second cross-correlation feature mentioned in the above embodiment is referred to as the second cross-correlation training feature in this embodiment. The cross-correlation feature combines the image feature of the training sample image and the image feature of the training reference image, and can be used to represent the similarity between the training sample image and the training reference image.

[0136] In one embodiment, step 2.3 specifically includes: performing first convolution operations on the image feature of the training sample image and the image feature of the training reference image based on the object positioning model, respectively, to obtain the first training intermediate feature of the training sample image and the first reference intermediate feature of the training reference image; and performing a cross-correlation operation on the first training intermediate feature and the first reference intermediate feature to obtain the first cross-correlation training feature between the training sample image and the training reference image.

[0137] The image feature of the training sample image and the image feature of the training reference image can form a feature map. The first convolution operation can specifically be to input the feature map of the training sample image and the feature map of the training reference image into two independent convolution layers in the target object positioning network, respectively, to reduce the dimensions of the feature map of the training sample image and the feature map of the training reference image through the two convolution layers, so as to reduce the size of the feature map and obtain the first training intermediate feature of the training sample image and the first reference intermediate feature of the training reference image. It can be understood that the first training intermediate feature and the first reference intermediate feature are both in the form of a feature map.

[0138] The cross-correlation operation refers to a correlation convolution operation with a specific quantization range. Specifically, after the computer device obtains the first training intermediate feature of the training sample image and the first reference intermediate feature of the training reference image, it performs a cross-correlation convolution on the first training intermediate feature and the first reference intermediate feature channel by channel to obtain the first cross-correlation training feature between the training sample image and the training reference image.

[0139] 2.4, inputting the first cross-correlation training feature into the regression network to determine the regression region corresponding to the training sample image.

[0140] 2.5, calculate a regression accuracy of the regression region based on the target object annotation region corresponding to the training sample image, and calculate a regression loss based on the regression accuracy.

[0141] In one embodiment, the IoU (Intersection-over-Union) between the regression region and the target object annotation region corresponding to the training sample image can be calculated as the regression accuracy of the regression region. The IoU can reflect the degree of overlap between the target object annotation region and the regression region, and the higher the degree of overlap, the higher the regression accuracy, so the IOU can be used as the regression accuracy of the regression region. Specifically, the computer device obtains the intersection region between the target object annotation region and the regression region; obtains the union region between the target object annotation region and the regression region; and determines the regression accuracy according to the area ratio of the intersection region and the union region corresponding to the regression region. The area ratio is the IoU, and the specific formula (1) is as follows:

[0142] IoU = area (A∩B) / area (A∪B) formula (1)

[0143] Where area represents the area, A represents the target object annotation region, and B represents the regression region. The intersection is denoted by “∩”, and the union is denoted by “∪”.

[0144] As shown in FIG. 1, a specific embodiment of the position of the target object annotation region and a certain regression region is shown. Box A represents the position of the target object annotation region in the training sample image. Box B represents the position of the regression region in the training sample image. Figure 4 Figure 4 Each grid represents a pixel point, and Figure 4 As can be seen, the intersection of A and B, i.e., the intersection, occupies a total of 6 pixel points (pixel points from the 5th row, 4th column to the 6th column, and the 6th row, 4th column to the 6th column), and the union of A and B occupies a total of 18 pixel points. Therefore, the IoU is 6 / 18 = 0.33.

[0145] After calculating the IoU between the regression region and the target object annotation region corresponding to the training sample image, the computer device can calculate the IoU loss based on the IoU. In one embodiment, the regression loss can be calculated by -ln(IoU). In another embodiment, the regression loss can also be calculated by 1-IoU.

[0146] 2.6, input the first cross-correlation training feature into the regression accuracy prediction network to determine the regression accuracy prediction value of the regression region.

[0147] 2.7, determine the regression accuracy loss based on the regression accuracy prediction value of the regression region and the regression accuracy of the regression region.

[0148] Continuing to refer to​Figure 3 After the computer device obtains the image features of the training sample image and the image features of the training reference image, the computer device inputs the image features of the training sample image into a first convolution layer and inputs the image features of the training reference image into a second convolution layer, the first convolution layer performs convolution operation on the image features of the training sample image to obtain first training intermediate features of the training sample image, the second convolution layer performs convolution operation on the image features of the training reference image to obtain first reference intermediate features of the training reference image, and then cross-correlation operation is performed on the first training intermediate features and the first reference intermediate features to obtain first cross-correlation training features, which are input into the regression network and the regression accuracy prediction network respectively.

[0149] 2.8, determining second cross-correlation training features between the training sample image and the training reference image based on the image features of the training sample image and the image features of the training reference image.

[0150] In one embodiment, the computer device can perform second convolution operation on the image features of the training sample image and the image features of the training reference image based on the object positioning model to obtain second training intermediate features of the training sample image and second reference intermediate features of the training reference image, and perform cross-correlation operation on the second training intermediate features and the second reference intermediate features to obtain second cross-correlation training features between the training sample image and the training reference image.

[0151] The second convolution operation can specifically be that the feature map of the training sample image and the feature map of the training reference image are input into another two independent convolution layers in the target object positioning network, and the feature map of the training sample image and the feature map of the training reference image are reduced in dimension by the two convolution layers to reduce the size of the feature map to obtain the second training intermediate features of the training sample image and the second reference intermediate features of the training reference image. It can be understood that the second training intermediate features and the second reference intermediate features here are in the form of feature maps. It can also be understood that the parameters of the other two independent convolution layers and the two independent convolution layers in the above embodiment can be the same or different, and the obtained features can also be the same or different.

[0152] After the computer device obtains the second training intermediate features of the training sample image and the second reference intermediate features of the training reference image, the computer device performs cross-correlation convolution on the second training intermediate features and the second reference intermediate features channel by channel to obtain second cross-correlation training features between the training sample image and the training reference image.

[0153] With reference to the above Figure 3The computer device inputs the image features of the training sample image into a third convolutional layer and inputs the image features of the training reference image into a fourth convolutional layer, the third convolutional layer performs convolutional operation on the image features of the training sample image to obtain second training intermediate features of the training sample image, and the fourth convolutional layer performs convolutional operation on the image features of the training reference image to obtain second reference intermediate features of the training reference image. Then, the second training intermediate features and the second reference intermediate features are subjected to cross-correlation operation to obtain second cross-correlation training features, and the second cross-correlation training features are input into the classification network.

[0154] 2.9, the second cross-correlation training features are input into the classification network to determine a classification confidence of the regression region, and a classification loss is calculated based on the classification confidence.

[0155] In an embodiment, the computer device can calculate Focal Loss as the classification loss based on the classification confidence. The calculation formula of the Focal Loss is as follows, where a is a balance factor for balancing positive and negative samples, y is a sample label, and y' is a classification confidence:

[0156]

[0157] Based on the above formula, it is first necessary to determine whether the regression region is a positive sample or a negative sample in the embodiment. In the embodiment, the computer device can calculate the IoU between the regression region and the target object label region, that is, obtain the intersection region between the target object label region and the regression region, obtain the union region between the target object label region and the regression region, and obtain the area ratio of the intersection region corresponding to the regression region to the union region (for details, refer to the IOU calculation formula in the above embodiment), which is the IoU between the regression region and the target object label region.

[0158] Further, for each regression region, when the corresponding IoU is greater than a first preset threshold, it can be determined that the sample is a positive sample, and the classification loss is determined according to the positive sample label corresponding to the training sample image and the classification confidence, the positive sample label is 1, and therefore the classification loss of the positive sample can be specifically referred to the formula of y = 1 in the above Focal Loss calculation formula (2); for each regression region, when the corresponding IoU is less than a second preset threshold, it can be determined that the sample is a negative sample, and the classification loss is determined according to the negative sample label corresponding to the training sample image and the classification confidence, the negative sample label is 0, and therefore the classification loss of the negative sample can be specifically referred to the formula of y = 0 in the above Focal Loss calculation formula (2). The first preset threshold and the second preset threshold can be set according to experience, for example, the first preset threshold is set to 0.7, and the second preset threshold is set to 0.3.

[0159] In one embodiment, when the classification confidence of a certain regression region is greater than the second preset threshold and less than the first preset threshold, the regression region is a sample with ambiguous classification, and it is meaningless to train the classification network for such a sample. In order to improve the training efficiency and accuracy, the classification loss of the sample can not be calculated.

[0160] 2.10, update the classification loss based on the regression accuracy and update the regression loss based on the classification confidence.

[0161] In one embodiment, the regression accuracy loss can be a binary cross entropy loss (BCE Loss). The calculation formula of the BCE Loss is as follows: formula (3), wherein N is the number of training sample images in a batch, y i is the label of the training sample image i, 1 for positive class and 0 for negative class, p i is the probability of the training sample image i being predicted as positive:

[0162]

[0163] The BCE Loss is usually used with a sigmoid function, and the sigmoid function is shown in formula (4):

[0164]

[0165] With reference to Figure 3 It can be seen that the computer device inputs the first cross-correlation training feature into the regression network and the regression accuracy prediction network respectively, and inputs the second cross-correlation training feature into the classification network. Then, the computer device calculates the Focal Loss as the classification loss according to the classification confidence of the classification branch, and updates the Focal Loss through the classification auxiliary connection A using the classification confidence of the classification branch. The computer device calculates the Focal Loss as the classification loss according to the classification confidence of the classification branch, and updates the Focal Loss through the classification auxiliary connection A using the classification confidence of the classification branch. The computer device calculates the binary cross entropy loss as the regression accuracy loss according to the regression accuracy of the regression branch and the regression accuracy of the regression accuracy prediction network.

[0166] 2.11, train the object positioning model according to the regression accuracy loss, the updated classification loss and the updated regression loss until the training stop condition is met, and obtain the trained object positioning model.

[0167] In the above embodiment, when the target object is the to-be-identified object, features are extracted from the training sample image and the training reference image respectively by the twin network, and the cross-correlation features between the training sample image and the training reference image are obtained. The cross-correlation features obtained by fusing the features of the training sample image and the training reference image can reflect the similarity between the two, and therefore, the target recognition problem can be converted into a similarity comparison problem. The target object positioning network trained in this way can quickly and accurately locate the target object from the input image when performing target positioning.

[0168] In one embodiment, determining the regression region corresponding to the training sample image based on the image features of the training sample image and the regression network of the object positioning model comprises: determining a predicted center point position corresponding to the target object in the training sample image and a predicted size corresponding to the target object based on the image features of the training sample image and the regression network of the object positioning model; and determining the regression region according to the predicted center point position and the predicted size.

[0169] Specifically, in this embodiment, the regression region is determined based on an anchor-free strategy. The computer device can determine each pixel point position in the feature map input into the regression network as a predicted center point position. For each predicted center point position, the computer device further determines a predicted size corresponding to the target object based on the predicted center point position, and determines a bounding box constructed by the predicted center point position and the predicted size corresponding thereto as the regression region.

[0170] The predicted center point position corresponding to the target object is the position of the center point of the predicted target object. The center point of the target object can be the center point of the smallest bounding box that can be surrounded by the target object. The bounding box can be a polygon, a circle, etc. The center point of the bounding box can be, for example, the intersection point of the diagonals of a polygon, the center of a circle, etc. For example, referring to Figure 5 Taking a virtual object as an example, the center point can be the intersection point 502 of the diagonals of the smallest quadrilateral that can be surrounded by the virtual object.

[0171] The predicted size corresponding to the target object can be the size of the smallest bounding box that can be surrounded by the target object, such as the width and height of a quadrilateral, the diameter of a circle, etc. Continuing to refer to Figure 5 As can be seen, the size can be the width 504 and the height 506 of the smallest quadrilateral that can be surrounded by the virtual object.

[0172] In the above embodiment, by using the anchor-free strategy, the cumbersome step of setting the prior box is avoided, the efficiency of determining the regression region is improved, and therefore the efficiency of model training is improved.

[0173] In an embodiment, determining the regression region corresponding to the training sample image based on the image features of the training sample image and the regression network of the object positioning model comprises: obtaining the position of an anchor box corresponding to the training sample image; determining an offset corresponding to the anchor box based on the image features of the training sample image and the regression network of the object positioning model; and determining the regression region based on the position of the anchor box and the offset of the anchor box.

[0174] In the embodiment, an anchor-based strategy is used to determine the regression region. A plurality of anchor boxes are set in the training sample image in advance, and an offset corresponding to each anchor box is determined based on the image features of the training sample image and the regression network of the object positioning model. Since the position of each anchor box is fixed and known, the computer device can determine a plurality of regression regions based on the position of each anchor box and the offset of each anchor box.

[0175] In an embodiment, the method further comprises a target identification step, which specifically comprises:

[0176] 1. Obtain a video to be identified, determine a first frame of video including an object to be identified in the video to be identified, obtain a reference image to be identified, and sequentially obtain video frames from a next frame of video of the first frame of video as images to be identified.

[0177] The video to be identified can be recorded video data or real-time recorded video data.

[0178] Specifically, the first frame of video of the video to be identified is taken as the reference image, and the object to be identified is sequentially positioned in each subsequent frame of video to achieve identification of the object to be identified in the video to be identified.

[0179] 2. Input the image to be identified and the reference image to be identified into the object positioning model, and obtain first cross-correlation identification features and second cross-correlation identification features between the image to be identified and the reference image to be identified based on the object positioning model.

[0180] The first cross-correlation identification features and the second cross-correlation identification features are cross-correlation features. In order to distinguish from the cross-correlation features obtained when training the object positioning model in the above embodiment, they are referred to as cross-correlation identification features here. The specific acquisition method of the cross-correlation identification features can be referred to the description in the above embodiment, which will not be described herein.

[0181] 3. Input the first cross-correlation identification features into the regression network of the object positioning model to determine a plurality of regression regions of the image to be identified.

[0182] 4. Input the second cross-correlation identification features into the classification network of the object positioning model to determine the classification confidence of each regression region of the image to be identified.

[0183] 5. Locating the object to be recognized from the image to be recognized according to the classification confidence of each regression region of the image to be recognized.

[0184] In one embodiment, the computer device can select the regression region with the maximum classification confidence as the region where the object to be recognized is located in the image to be recognized, thereby locating the object to be recognized from the image to be recognized.

[0185] In another embodiment, the method further includes a target recognition step, which specifically includes:

[0186] 1. Obtaining a video to be recognized, determining a first frame of video including the object to be recognized in the video to be recognized, obtaining a reference image to be recognized, and sequentially obtaining video frames from a next frame of video of the first frame of video as images to be recognized.

[0187] 2. Inputting the image to be recognized and the reference image to be recognized into the object locating model respectively, and obtaining first cross-correlation recognition features and second cross-correlation recognition features between the image to be recognized and the reference image to be recognized based on the object locating model.

[0188] 3. Inputting the first cross-correlation recognition features into a regression network of the object locating model to determine a plurality of regression regions of the image to be recognized.

[0189] 4. Inputting the first cross-correlation recognition features into a regression accuracy prediction network of the object locating model to obtain regression accuracy prediction values corresponding to each regression region of the image to be recognized.

[0190] 5. Inputting the second cross-correlation recognition features into a classification network of the object locating model to determine classification confidence of each regression region of the image to be recognized.

[0191] 6. Correspondingly multiplying the regression accuracy prediction values and the classification confidence of each regression region of the image to be recognized to obtain target locating scores corresponding to each regression region of the image to be recognized.

[0192] For example, assuming that the image to be recognized corresponds to two regression regions, namely a regression region A and a regression region B, the regression accuracy prediction value of the regression region A is A1, the classification confidence is A2, the regression accuracy prediction value of the regression region B is B1, and the classification confidence is B2, then the locating score of the regression region A is A1*A2, and the locating score of the regression region B is B1*B2.

[0193] 7. Locating the object to be recognized from the image to be recognized according to the target locating scores corresponding to each regression region of the image to be recognized.

[0194] In one embodiment, as Figure 6As shown, a target object positioning method is provided, which is taken as an example of being applied to a computer device, which can be a terminal 102 or a server 104 in Figure 1 The method specifically includes the following steps:

[0195] Step 602, obtaining an input image.

[0196] Step 604, obtaining a trained target object positioning model.

[0197] The trained target object positioning model is obtained, the target object positioning model includes a classification network and a regression network, the target object positioning model is trained through a target regression loss of the regression network and a target classification loss of the classification network, the target regression loss is obtained by updating an initial regression loss through a classification confidence of a regression region, the target classification loss is obtained by updating an initial classification loss through a regression accuracy of the regression region, the regression region is determined based on an image feature of a training sample image and the regression network of the target object positioning model, the regression accuracy of the regression region is calculated based on a target object annotation region corresponding to the training sample image, the classification confidence of the regression region is determined based on the image feature of the training sample image and the classification network, the initial classification loss is calculated based on the classification confidence, and the initial regression loss is calculated based on the regression accuracy.

[0198] Step 606, determining a plurality of regression regions of the input image based on an image feature of the input image and the regression network.

[0199] Step 608, determining a classification confidence of each regression region of the input image based on the image feature of the input image and the classification network.

[0200] Step 610, positioning a target object from the input image according to the classification confidence of each regression region of the input image to obtain a region where the target object is located.

[0201] The related explanations of steps 602-610 above can refer to the descriptions in the embodiments above, which will not be repeated here.

[0202] The target object positioning method above, since the target object positioning model is trained through the target regression loss of the regression network and the target classification loss of the classification network, the losses of the regression network and the classification network are updated respectively in the training process, so that the classification confidence is considered in the regression loss of the regression network, and the regression accuracy is considered in the classification loss of the classification network, then after the target positioning model is trained through the updated loss, the trained target positioning model obtained has reduced inconsistency between the classification network and the regression network, so that the classification confidence of the regression region with high regression accuracy is higher, and the regression region with high classification confidence is more accurate, thereby improving the positioning accuracy.

[0203] In one embodiment, the target object is an object to be recognized; the obtaining the input image comprises: obtaining the input image and a reference image corresponding to the input image; the reference image is an image including the target object in one frame of an image frame sequence before the input image; the determining the plurality of regression regions of the input image based on the image feature of the input image and the regression network comprises: determining first cross-correlation features between the input image and the reference image based on the image feature of the input image and the image feature of the reference image; inputting the first cross-correlation features into the regression network to determine the plurality of regression regions of the input image; the determining the classification confidence of each regression region of the input image based on the image feature of the input image and the classification network comprises: determining second cross-correlation features between the input image and the reference image based on the image feature of the input image and the image feature of the reference image; inputting the second cross-correlation features into the classification network to determine the classification confidence of each regression region of the input image.

[0204] The current image and the input image are both one frame of images in an image frame sequence. The image frame sequence is a collection of a series of images with a time sequence, which can be a sequence of video frames in a video, or a plurality of images continuously collected by an image collection device. The target object is included in both the current image and the input image, and the target object in the current image is to be positioned, and the target object in the input image has been positioned. The target object is an object to be recognized in the image frame sequence, which can be an independent living body or object, such as a natural person, an animal, a vehicle, a virtual character, etc., or a specific part, such as a face, a hand, etc.

[0205] It can be understood that, since the current image and the input image come from the same image frame sequence, and the input image is a frame of image processed before the current image, the target object to be recognized can be selected through the input image, and the target object is positioned in the subsequent image frame, so as to realize the recognition of the target object in the image frame sequence.

[0206] Specifically, the input image can be the first frame of image of the image frame sequence, or can be the middle frame of image of the image frame sequence. When the input image is the first frame of image, the target object can be positioned based on a user operation, so that the subsequent image frame can obtain the target object to be recognized through the input image. When the input image is the middle frame of image, the input image can be the previous frame of image of the current image, so that the current image can obtain the target object to be recognized through the target object positioned in the previous frame of image.

[0207] In one specific embodiment, the input image can be a complete image frame in the image frame sequence, or a subject region extracted from the input image based on the position of the target object. The subject region is centered on the position of the target object and has a specified range, so that the target object to be recognized can be quickly obtained from the subsequent image frame.

[0208] In one specific embodiment, the input image can be a complete image frame in the image frame sequence, or a subject region selected according to the recognition result of the previous image frame. Considering that the position of the target object changes little between the previous and subsequent image frames, the subject region is obtained by centering on the position of the target object in the previous image frame and extracting the input image according to a specified search range, so as to reduce the search range of the current image.

[0209] In one embodiment, a target object positioning application is running on the computer device, and the computer device can start the target object positioning application according to user operation. The target object positioning application obtains the image frame sequence and extracts the input image and the current image from the image frame sequence.

[0210] In another embodiment, an object positioning method is provided. In this embodiment, the object positioning model includes a regression network, a classification network, and a regression accuracy prediction network. The object positioning model is trained by a regression accuracy loss of the regression accuracy prediction network, a target regression loss of the regression network, and a target classification loss of the classification network. The target regression loss of the regression network and the target classification loss of the classification network are obtained in the same way as in the above embodiment, and the regression accuracy loss is determined based on the regression accuracy prediction value of the regression region and the regression accuracy of the regression region; the regression accuracy prediction value is determined based on the image features of the training sample image and the regression accuracy prediction network.

[0211] Reference Figure 7 The embodiment specifically includes the following steps:

[0212] Step 702, obtaining an input image and a reference image corresponding to the input image; the reference image is an image including a target object in one of the image frame sequences before the input image.

[0213] Step 704, determining a first cross-correlation feature between the input image and the reference image based on image features of the input image and image features of the reference image.

[0214] Step 706, inputting the first cross-correlation feature into the regression network to determine a plurality of regression regions of the input image.

[0215] Step 708, inputting the first cross-correlation feature into the regression accuracy prediction network to determine a regression accuracy prediction value of each regression region of the input image.

[0216] At step 710, a second cross-correlation feature between the input image and the reference image is determined based on the image feature of the input image and the image feature of the reference image.

[0217] At step 712, the second cross-correlation feature is input into a classification network to determine a classification confidence of each regression region of the input image.

[0218] At step 714, a target positioning score of each regression region of the input image is obtained by corresponding multiplication of a regression accuracy prediction value and the classification confidence of the regression region.

[0219] At step 716, object positioning is performed on the input image according to the target positioning score of each regression region of the input image to obtain a region where the target object is located.

[0220] For example, refer to FIG. 6, which is a schematic diagram of an output result of the object positioning model used for target recognition. Refer to FIG. 7, which is a schematic diagram of a target positioning result of the object positioning model used for target recognition. Figure 8 For example, refer to FIG. 6, which is a schematic diagram of an output result of the object positioning model used for target recognition. Refer to FIG. 7, which is a schematic diagram of a target positioning result of the object positioning model used for target recognition. Figure 8 For example, refer to FIG. 6, which is a schematic diagram of an output result of the object positioning model used for target recognition. Refer to FIG. 7, which is a schematic diagram of a target positioning result of the object positioning model used for target recognition.

[0221] As shown in FIG. 8, which is a whole flowchart of the object positioning method in an embodiment. Refer to FIG. 9, which is a whole flowchart of the object positioning method in an embodiment. Figure 9 As shown in FIG. 8, which is a whole flowchart of the object positioning method in an embodiment. Refer to FIG. 9, which is a whole flowchart of the object positioning method in an embodiment. Figure 9 As shown in FIG. 8, which is a whole flowchart of the object positioning method in an embodiment. Refer to FIG. 9, which is a whole flowchart of the object positioning method in an embodiment.

[0222] As shown in FIG. 8, which is a whole flowchart of the object positioning method in an embodiment. Refer to FIG. 9, which is a whole flowchart of the object positioning method in an embodiment. Figure 10Fig. 2 shows a result of human body recognition in an embodiment. Referring to Fig. 2, the first image frame 110 is a frame of a video captured by a camera of an autonomous driving device. The target object 111 is a human body in the first image frame 110. The server determines the region 112 where the target object 111 is located in the first image frame 110. The second image frame 120 is a frame of the video captured by the camera of the autonomous driving device after the first image frame 110. The third image frame 130 is a frame of the video captured by the camera of the autonomous driving device after the second image frame 120. The target object 111 is still a human body in the second image frame 120 and the third image frame 130. The server determines the region 121 where the target object 111 is located in the second image frame 120 and the region 131 where the target object 111 is located in the third image frame 130. Figure 10 In the first image frame 110, the region 112 where the target object 111 is located is detected. In subsequent image frames of the first image frame 110, the target object 111 is to be recognized by the object positioning method provided in the embodiments of the present application. For example, in the second image frame 120, the region 121 where the target object 111 is located is positioned. In the third image frame 130, the region 131 where the target object 111 is located is positioned. The same applies to the subsequent image frames.

[0223] It should be noted that the first image frame 110, the second image frame 120, and the third image frame 130 can be three image frames that are adjacent in sequence. The first image frame 110, the second image frame 120, and the third image frame 130 can also be three image frames that are arranged in sequence with an interval between each two frames.

[0224] The present application also provides an application scenario. The application scenario applies the object positioning model processing method and the object positioning method described above. In the application scenario, the video captured by the camera of the autonomous driving device needs to be recognized for the human body to be recognized. Specifically, the application of the object positioning model processing method and the object positioning method in the application scenario is as follows.

[0225] First, the server obtains the training sample image and the training reference image including the human body. Then, the server obtains the trained object positioning model by performing steps 2.2-2.11 in the embodiments described above. The server sends the object positioning model to the terminal. After obtaining the video captured by the camera containing the human body, the terminal determines the first frame of the video in which the human body appears as the reference image. From the next frame of the video, the video frames are sequentially obtained as the to-be-recognized images. For each to-be-recognized image, the terminal positions the region where the human body is located by performing the following steps to realize the human body to be recognized in the captured video.

[0226] 1. Determine the first cross-correlation feature between the to-be-recognized image and the reference image based on the image features of the to-be-recognized image and the image features of the reference image.

[0227] 2. Input the first cross-correlation feature into the regression network to determine the multiple regression regions of the to-be-recognized image.

[0228] 3. Input the first cross-correlation feature into the regression accuracy prediction network to determine the regression accuracy prediction value of each regression region of the to-be-recognized image.

[0229] 4. Determine the second cross-correlation feature between the to-be-recognized image and the reference image based on the image features of the to-be-recognized image and the image features of the reference image.

[0230] 5. inputting the second cross-correlation feature into a classification network to determine a classification confidence of each regression region of the image to be recognized.

[0231] 6. multiplying the regression accuracy of each regression region of the image to be recognized with the classification confidence to obtain a target positioning score of each regression region of the image to be recognized.

[0232] 7. positioning a target object from the image to be recognized according to the target positioning score of each regression region of the image to be recognized to obtain a region where the target object is located.

[0233] It should be understood that, although Figures 1-10 each step in the flowchart is shown in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, Figures 1-10 at least part of the steps in the flowchart can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0234] In an embodiment, as shown in Figure 11 , an object positioning model processing apparatus 1100 is provided, which can be a part of a computer device in the form of a software module or a hardware module, and specifically includes:

[0235] The training sample acquisition module 1102 is configured to acquire a training sample image including a target object.

[0236] The regression region determination module 1104 is configured to determine a regression region corresponding to the training sample image based on the image feature of the training sample image and a regression network of the object positioning model.

[0237] The regression loss calculation module 1106 is configured to calculate a regression accuracy of the regression region based on a target object label region of the training sample image, and calculate a regression loss based on the regression accuracy.

[0238] The classification loss calculation module 1108 is configured to determine a classification confidence of the regression region based on the image feature of the training sample image and a classification network of the object positioning model, and calculate a classification loss based on the classification confidence.

[0239] The update module 1110 is configured to update the classification loss based on the regression accuracy, and update the regression loss based on the classification confidence.

[0240] The training module 1112 is configured to train the object positioning model according to the updated classification loss and the updated regression loss until a training stop condition is met, to obtain a trained object positioning model; the trained object positioning model is configured to perform object positioning on an input image.

[0241] In an embodiment, the apparatus further includes a regression accuracy loss determination module configured to determine a regression accuracy prediction value of the regression region based on the regression accuracy prediction network of the object positioning model, and determine a regression accuracy loss based on the regression accuracy prediction value of the regression region and the regression accuracy of the regression region; and the training module is further configured to train the object positioning model according to the regression accuracy loss, the updated classification loss and the updated regression loss.

[0242] In an embodiment, the target object is an object to be recognized; the training sample acquisition module is further configured to acquire a training sample image including the target object and a training reference image corresponding to the training sample image; the training reference image and the training sample image include the same target object; the regression region determination module is further configured to determine a first cross-correlation training feature between the training sample image and the training reference image based on image features of the training sample image and image features of the training reference image, and input the first cross-correlation training feature into the regression network to determine the regression region corresponding to the training sample image; the regression accuracy loss determination module is further configured to input the first cross-correlation training feature into the regression accuracy prediction network to determine the regression accuracy prediction value of the regression region; and the classification loss calculation module is further configured to determine a second cross-correlation training feature between the training sample image and the training reference image based on the image features of the training sample image and the image features of the training reference image, and input the second cross-correlation training feature into the classification network to determine the classification confidence of the regression region; the first cross-correlation training feature and the second cross-correlation training feature are used to represent the similarity between the training sample image and the training reference image.

[0243] In an embodiment, the apparatus further includes a feature extraction module configured to acquire image features of the training sample image and image features of the training reference image based on the object positioning model; and the regression region determination module is further configured to perform convolution operations on the image features of the training sample image and the image features of the training reference image based on the object positioning model, to obtain a first training intermediate feature of the training sample image and a first reference intermediate feature of the training reference image, and perform cross-correlation operations on the first training intermediate feature and the first reference intermediate feature based on the object positioning model, to obtain the first cross-correlation training feature between the training sample image and the training reference image.

[0244] In one embodiment, the object positioning model comprises two feature extraction networks which have the same model structure and share model parameters; the feature extraction module is configured to input the training sample image and the training reference image into the two feature extraction networks respectively; and the feature extraction module is configured to output image features of the training sample image and image features of the training reference image in parallel and respectively through the two feature extraction networks.

[0245] In one embodiment, the regression loss calculation module is configured to obtain an intersection region between the target object annotation region and the regression region; obtain a union region between the target object annotation region and the regression region; and determine the regression accuracy according to an area ratio of the intersection region and the union region corresponding to the regression region.

[0246] In one embodiment, the classification loss calculation module is configured to obtain an intersection region between the target object annotation region and the regression region; obtain a union region between the target object annotation region and the regression region; obtain an area ratio of the intersection region and the union region corresponding to the regression region; when the area ratio is greater than a first preset threshold, determine the classification loss according to the positive sample label corresponding to the training sample image and the classification confidence; and when the area ratio is less than a second preset threshold, determine the classification loss according to the negative sample label corresponding to the training sample image and the classification confidence.

[0247] In one embodiment, the regression region determination module is configured to determine a predicted center point position corresponding to the target object and a predicted size corresponding to the target object in the training sample image based on the image features of the training sample image and the regression network of the object positioning model; and determine the regression region according to the predicted center point position and the predicted size.

[0248] In one embodiment, the regression region determination module is configured to obtain a position of an anchor frame corresponding to the training sample image; determine an offset corresponding to the anchor frame based on the image features of the training sample image and the regression network of the object positioning model; and determine the regression region based on the position of the anchor frame and the offset of the anchor frame.

[0249] In one embodiment, the target object is an object to be identified; the apparatus further includes a target recognition module for acquiring a video to be identified; determining the first video frame containing the object to be identified in the video to be identified to obtain a reference image to be identified; acquiring video frames sequentially from the next video frame after the first video frame as images to be identified; inputting the image to be identified and the reference image to be identified into an object localization model respectively; acquiring a first cross-correlation recognition feature and a second cross-correlation recognition feature between the image to be identified and the reference image to be identified based on the object localization model; inputting the first cross-correlation recognition feature into the regression network of the object localization model to determine multiple regression regions of the image to be identified; inputting the second cross-correlation recognition feature into the classification network of the object localization model to determine the classification confidence of each regression region of the image to be identified; and locating the object to be identified from the image to be identified based on the classification confidence of each regression region of the image to be identified.

[0250] In one embodiment, the target recognition module inputs the first cross-correlation recognition features into the regression accuracy prediction network of the object localization model to obtain the regression accuracy prediction value of each regression region of the image to be recognized; multiplies the regression accuracy prediction value of each regression region of the image to be recognized with the classification confidence to obtain the target localization score of each regression region of the image to be recognized; and locates the object to be recognized from the image to be recognized based on the target localization score of each regression region of the image to be recognized.

[0251] In one embodiment, such as Figure 12 As shown, an object positioning device 1200 is provided. This device can be a software module, a hardware module, or a combination of both as part of a computer device. Specifically, the device includes:

[0252] Image acquisition module 1202 is used to acquire input images;

[0253] The model acquisition module 1204 is used to acquire the trained object localization model. The object localization model includes a classification network and a regression network. The object localization model is trained using the target regression loss of the regression network and the target classification loss of the classification network. The target regression loss is obtained by updating the initial regression loss using the classification confidence of the regression region. The target classification loss is obtained by updating the initial classification loss using the regression accuracy of the regression region. The regression region is determined based on the image features of the training sample images and the regression network of the object localization model. The regression accuracy of the regression region is calculated based on the target object annotation region corresponding to the training sample images. The classification confidence of the regression region is determined based on the image features of the training sample images and the classification network. The initial classification loss is calculated based on the classification confidence. The initial regression loss is calculated based on the regression accuracy.

[0254] The regression region determination module 1206 is configured to determine a plurality of regression regions of the input image based on the image feature of the input image and the regression network.

[0255] The confidence determination module 1208 is configured to determine a classification confidence of each regression region of the input image based on the image feature of the input image and the classification network.

[0256] The positioning module 1210 is configured to perform object positioning on the input image according to the classification confidence of each regression region of the input image, to obtain a region where the target object is located.

[0257] In an embodiment, the target object is a to-be-recognized object; the image acquisition module is further configured to acquire an input image and a reference image corresponding to the input image; the reference image is an image including the target object in one frame of an image frame sequence before the input image; the regression region determination module is further configured to determine a first cross-correlation feature between the input image and the reference image based on an image feature of the input image and an image feature of the reference image; and input the first cross-correlation feature into the regression network, to determine a plurality of regression regions of the input image; and the confidence determination module is further configured to determine a second cross-correlation feature between the input image and the reference image based on the image feature of the input image and the image feature of the reference image; and input the second cross-correlation feature into the classification network, to determine a classification confidence of each regression region of the input image.

[0258] In an embodiment, the object positioning model further includes a regression accuracy prediction network; the object positioning model is trained by a regression accuracy loss of the regression accuracy prediction network, a target regression loss of the regression network, and a target classification loss of the classification network; the regression accuracy loss is determined based on a regression accuracy prediction value of the regression region and a regression accuracy of the regression region; the regression accuracy prediction value is determined based on an image feature of a training sample image and the regression accuracy prediction network; the method further includes a regression accuracy prediction module configured to input the first cross-correlation feature into the regression accuracy prediction network, to determine the regression accuracy prediction value of each regression region of the input image; and the positioning module is further configured to multiply the regression accuracy and the classification confidence of each regression region of the input image correspondingly, to obtain a target positioning score of each regression region of the input image; and perform positioning on the target object from the input image according to the target positioning score of each regression region of the input image, to obtain a region where the target object is located.

[0259] The specific definitions of the object positioning model processing apparatus and the object positioning apparatus can refer to the definitions of the object positioning model processing method and the object positioning method, which are not repeated here. Each module in the object positioning model processing apparatus and the object positioning apparatus can be implemented by software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0260] In an embodiment, a computer device, which can be a server, has an internal structure as shown in Figure 13 The computer device includes a processor, a memory, and a network interface connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device can be used to store training sample image data. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement an object positioning model processing method or an object positioning method.

[0261] Those skilled in the art can understand that Figure 13 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0262] In an embodiment, a computer device is also provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in each method embodiment.

[0263] In an embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps in each method embodiment.

[0264] In an embodiment, a computer program product or a computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium. The processor executes the computer instructions to cause the computer device to perform the steps in each method embodiment.

[0265] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0266] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0267] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method of object localization model processing, the method comprising: The method comprises: obtaining a training sample image comprising a target object; determining a regression region corresponding to the training sample image based on image features of the training sample image and a regression network of an object positioning model; calculating regression accuracy of the regression region based on a target object annotation region corresponding to the training sample image, calculating regression loss based on the regression accuracy; determining classification confidence of the regression region based on image features of the training sample image and a classification network of the object positioning model, calculating classification loss based on the classification confidence; updating the classification loss based on the regression accuracy, and updating the regression loss based on the classification confidence; training the object positioning model according to the updated classification loss and the updated regression loss until a training stop condition is met, to obtain a trained object positioning model; wherein the trained object positioning model is used for object positioning of an input image.

2. The method of claim 1, wherein, Before the training of the object positioning model according to the updated classification loss and the updated regression loss, the method further comprises: determining a regression accuracy prediction value of the regression region based on a regression accuracy prediction network of the object positioning model; determining regression accuracy loss based on the regression accuracy prediction value of the regression region and the regression accuracy of the regression region; the training of the object positioning model according to the updated classification loss and the updated regression loss comprises: training the object positioning model according to the regression accuracy loss, the updated classification loss and the updated regression loss.

3. The method of claim 2, wherein, The target object is a to-be-identified object; the obtaining of the training sample image comprising the target object comprises: obtaining a training sample image comprising a target object and a training reference image corresponding to the training sample image; the training reference image and the training sample image comprise the same target object; the determination of the regression region corresponding to the training sample image based on the image features of the training sample image and the regression network of the object positioning model comprises: determining first cross-correlation training features between the training sample image and the training reference image based on image features of the training sample image and image features of the training reference image; inputting the first cross-correlation training features into the regression network to determine the regression region corresponding to the training sample image; the determination of the regression accuracy prediction value of the regression region based on the regression accuracy prediction network of the object positioning model comprises: inputting the first cross-correlation training features into the regression accuracy prediction network to determine the regression accuracy prediction value of the regression region; the determination of the classification confidence of the regression region based on the image features of the training sample image and the classification network of the object positioning model comprises: determining second cross-correlation training features between the training sample image and the training reference image based on image features of the training sample image and image features of the training reference image; inputting the second cross-correlation training features into the classification network to determine the classification confidence of the regression region; The first cross-correlation training feature and the second cross-correlation training feature are used to represent the similarity between the training sample image and the training reference image.

4. The method of claim 3, wherein, Before determining the first cross-correlation training feature between the training sample image and the training reference image based on the image feature of the training sample image and the image feature of the training reference image, the method further comprises: obtaining the image feature of the training sample image and the image feature of the training reference image based on the object positioning model; determining the first cross-correlation training feature between the training sample image and the training reference image based on the image feature of the training sample image and the image feature of the training reference image comprises: performing convolution operations on the image feature of the training sample image and the image feature of the training reference image based on the object positioning model, to obtain the first training intermediate feature of the training sample image and the first reference intermediate feature of the training reference image; performing cross-correlation operations on the first training intermediate feature and the first reference intermediate feature based on the object positioning model, to obtain the first cross-correlation training feature between the training sample image and the training reference image.

5. The method of claim 4, wherein, The object positioning model comprises two feature extraction networks which have the same model structure and share model parameters; and obtaining the image feature of the training sample image and the image feature of the training reference image based on the object positioning model comprises: inputting the training sample image and the training reference image into the two feature extraction networks respectively; outputting the image feature of the training sample image and the image feature of the training reference image in parallel and respectively through the two feature extraction networks.

6. The method of claim 1, wherein, The calculation of the regression accuracy of the regression region based on the target object annotation region corresponding to the training sample image comprises: obtaining an intersection region between the target object annotation region and the regression region; obtaining a union region between the target object annotation region and the regression region; determining the regression accuracy according to the area ratio of the intersection region corresponding to the regression region to the union region.

7. The method of claim 1, wherein, The calculation of the classification loss based on the classification confidence comprises: obtaining an intersection region between the target object annotation region and the regression region; obtaining a union region between the target object annotation region and the regression region; obtaining the area ratio of the intersection region corresponding to the regression region to the union region; when the area ratio is greater than a first preset threshold, determining the classification loss according to the positive sample label corresponding to the training sample image and the classification confidence; when the area ratio is less than a second preset threshold, determining the classification loss according to the negative sample label corresponding to the training sample image and the classification confidence.

8. The method of claim 2, wherein, The target object is a to-be-recognized object; and the method further comprises: obtaining a to-be-recognized video; determining a first video frame comprising the to-be-recognized object in the to-be-recognized video, to obtain a to-be-recognized reference image; sequentially obtaining video frames as to-be-recognized images from a next video frame of the first video frame; and inputting the to-be-identified image and the to-be-identified reference image into the object positioning model respectively; obtaining first cross-correlation identification features and second cross-correlation identification features between the to-be-identified image and the to-be-identified reference image based on the object positioning model; inputting the first cross-correlation identification features into a regression network of the object positioning model to determine a plurality of regression regions of the to-be-identified image; inputting the second cross-correlation identification features into a classification network of the object positioning model to determine classification confidence of each regression region of the to-be-identified image; locating the to-be-identified object from the to-be-identified image according to the classification confidence of each regression region of the to-be-identified image.

9. A method of locating an object, characterized by The method comprises: obtaining an input image; obtaining a trained object positioning model; the object positioning model comprises a classification network and a regression network; the object positioning model is trained by a target regression loss of the regression network and a target classification loss of the classification network; the target regression loss is obtained by updating an initial regression loss by classification confidence of a regression region; the target classification loss is obtained by updating an initial classification loss by regression accuracy of the regression region; the regression region is determined based on image features of a training sample image and the regression network of the object positioning model; the regression accuracy of the regression region is calculated based on a target object annotation region corresponding to the training sample image; the classification confidence of the regression region is determined based on the image features of the training sample image and the classification network; the initial classification loss is calculated based on the classification confidence; the initial regression loss is calculated based on the regression accuracy; determining a plurality of regression regions of the input image based on image features of the input image and the regression network; determining classification confidence of each regression region of the input image based on the image features of the input image and the classification network; locating an object in the input image according to the classification confidence of each regression region of the input image, to obtain a region where a target object is located.

10. The method of claim 9, wherein, The target object is a to-be-identified object; the obtaining of the input image comprises: obtaining an input image and a reference image corresponding to the input image; the reference image is an image including a target object in a sequence of image frames in which the input image is located and before the input image; The determining of the plurality of regression regions of the input image based on the image features of the input image and the regression network comprises: determining first cross-correlation features between the input image and the reference image based on the image features of the input image and the reference image; inputting the first cross-correlation features into the regression network to determine the plurality of regression regions of the input image; The determining of the classification confidence of each regression region of the input image based on the image features of the input image and the classification network comprises: determining second cross-correlation features between the input image and the reference image based on the image features of the input image and the reference image; input the second cross-correlation feature into the classification network to determine the classification confidence of each regression region of the input image.

11. The method of claim 10, wherein, The object positioning model further comprises a regression accuracy prediction network; the object positioning model is trained by a regression accuracy loss of the regression accuracy prediction network, a target regression loss of the regression network, and a target classification loss of the classification network; the regression accuracy loss is determined based on a regression accuracy prediction value of the regression region and a regression accuracy of the regression region; the regression accuracy prediction value is determined based on an image feature of the training sample image and the regression accuracy prediction network; Before the target object is positioned from the input image according to the classification confidence of each regression region of the input image, the method further comprises: inputting the first cross-correlation feature into the regression accuracy prediction network to determine the regression accuracy prediction value of each regression region of the input image; positioning the target object from the input image according to the classification confidence of each regression region of the input image, to obtain a region where the target object is located, comprising: correspondingly multiplying the regression accuracy prediction value of each regression region of the input image and the classification confidence to obtain a target positioning score of each regression region of the input image; positioning the target object from the input image according to the target positioning score of each regression region of the input image, to obtain a region where the target object is located.

12. An object localization model processing apparatus, characterized by, The device comprises: a training sample acquisition module configured to acquire a training sample image comprising a target object; a regression region determination module configured to determine a regression region corresponding to the training sample image based on an image feature of the training sample image and a regression network of an object positioning model; a regression loss calculation module configured to calculate a regression accuracy of the regression region based on a target object label region of the training sample image, and calculate a regression loss based on the regression accuracy; a classification loss calculation module configured to determine a classification confidence of the regression region based on the image feature of the training sample image and a classification network of the object positioning model, and calculate a classification loss based on the classification confidence; an update module configured to update the classification loss based on the regression accuracy, and update the regression loss based on the classification confidence; a training module configured to train the object positioning model according to the updated classification loss and the updated regression loss until a training stop condition is met, to obtain a trained object positioning model; wherein the trained object positioning model is configured to position an input image.

13. An object positioning apparatus, characterized by comprising: The device comprises: an image acquisition module configured to acquire an input image; The model obtaining module is configured to obtain a trained object positioning model; the object positioning model comprises a classification network and a regression network; the object positioning model is trained by a target regression loss of the regression network and a target classification loss of the classification network; the target regression loss is obtained by updating an initial regression loss by a classification confidence of a regression region; the target classification loss is obtained by updating an initial classification loss by a regression accuracy of the regression region; the regression region is determined based on an image feature of a training sample image and the regression network of the object positioning model; the regression accuracy of the regression region is calculated based on a target object label region corresponding to the training sample image; the classification confidence of the regression region is determined based on the image feature of the training sample image and the classification network; the initial classification loss is calculated based on the classification confidence; and the initial regression loss is calculated based on the regression accuracy. The regression region determining module is configured to determine a plurality of regression regions of the input image based on an image feature of the input image and the regression network. The confidence determining module is configured to determine a classification confidence of each regression region of the input image based on an image feature of the input image and the classification network. The positioning module is configured to perform object positioning on the input image according to the classification confidence of each regression region of the input image to obtain a region where a target object is located.

14. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 11.

15. A computer readable storage medium storing a computer program, wherein the computer program comprises program instructions configured to cause a processor to perform the method according to any one of claims 1 to 14. The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 11.

Images