Model optimization method, device, computer equipment and storage medium

Abstract

The embodiments of the present application provide a model optimization method, apparatus, computer equipment and storage medium, wherein the method comprises: obtaining a sample image, wherein the sample image comprises a sample object; calling an object detection model to perform object detection on the sample image, and obtaining position detection information of a target key point of the sample object, and position detection information of a reference object of the target key point; obtaining a position loss function of the object detection model, wherein the position loss function is constructed based on a target position relationship that needs to be satisfied between the target key point and the reference object; using the position loss function, calculating a target loss value based on the position detection information of the target key point and the position detection information of the reference object; optimizing the model parameters of the object detection model based on the target loss value, and obtaining an optimized object detection model, thereby effectively improving the detection accuracy and robustness of the object detection model.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, in particular to the field of computer technology, and specifically to a model optimization method, a model optimization device, a computer device, and a storage medium. Background Art

[0002] With the continuous development of internet technology, object detection is finding increasingly widespread application. Object detection uses computers to process, analyze, and understand images in order to identify the attributes of objects within them. Currently, object detection is typically achieved using object detection models. The accuracy of object detection results is closely tied to the performance of the object detection model. Consequently, optimizing model training to achieve high-performance object detection models has become a research hotspot. Summary of the Invention

[0003] The embodiments of the present application provide a model optimization method, apparatus, computer device, and storage medium that can effectively improve the detection accuracy and robustness of object detection models.

[0004] In one aspect, an embodiment of the present application provides a model training method, the method comprising:

[0005] Acquire a sample image, where the sample image includes a sample object;

[0006] Calling the object detection model to perform object detection on the sample image to obtain the position detection information of the target key points of the sample object and the position detection information of the reference object of the target key points;

[0007] Obtain the position loss function of the object detection model. The position loss function is constructed based on the target position relationship that must be satisfied between the target key points and the reference object;

[0008] The position loss function is used to calculate the target loss value based on the position detection information of the target key points and the position detection information of the reference object;

[0009] The model parameters of the object detection model are optimized according to the target loss value to obtain an optimized object detection model.

[0010] On the other hand, an embodiment of the present application provides a model training device, comprising:

[0011] an acquiring unit, configured to acquire a sample image, where the sample image includes a sample object;

[0012] a processing unit, configured to call an object detection model to perform object detection on a sample image, and obtain position detection information of a target key point of the sample object and position detection information of a reference object of the target key point;

[0013] The acquisition unit is also used to obtain the position loss function of the object detection model. The position loss function is constructed based on the target position relationship that needs to be satisfied between the target key point and the reference object;

[0014] The processing unit is further configured to calculate a target loss value based on the position detection information of the target key point and the position detection information of the reference object using a position loss function;

[0015] The processing unit is further used to optimize the model parameters of the object detection model according to the target loss value to obtain an optimized object detection model.

[0016] In another aspect, an embodiment of the present application provides a computer device, the computer device including an input device, an output device, and the computer device further including:

[0017] a processor adapted to implement one or more instructions; and

[0018] A computer storage medium storing one or more instructions adapted to be loaded by a processor and executed by a processor:

[0019] Acquire a sample image, where the sample image includes a sample object;

[0020] Calling the object detection model to perform object detection on the sample image to obtain the position detection information of the target key points of the sample object and the position detection information of the reference object of the target key points;

[0021] Obtain the position loss function of the object detection model. The position loss function is constructed based on the target position relationship that must be satisfied between the target key points and the reference object;

[0022] The position loss function is used to calculate the target loss value based on the position detection information of the target key points and the position detection information of the reference object;

[0023] The model parameters of the object detection model are optimized according to the target loss value to obtain an optimized object detection model.

[0024] In another aspect, an embodiment of the present application provides a computer storage medium storing one or more instructions, wherein the one or more instructions are suitable for being loaded by a processor and executing the following steps:

[0025] Acquire a sample image, where the sample image includes a sample object;

[0026] Calling the object detection model to perform object detection on the sample image to obtain the position detection information of the target key points of the sample object and the position detection information of the reference object of the target key points;

[0027] Obtain the position loss function of the object detection model. The position loss function is constructed based on the target position relationship that must be satisfied between the target key points and the reference object;

[0028] The position loss function is used to calculate the target loss value based on the position detection information of the target key points and the position detection information of the reference object;

[0029] The model parameters of the object detection model are optimized according to the target loss value to obtain an optimized object detection model.

[0030] On the other hand, an embodiment of the present application provides a computer program product, which includes a computer program; the computer program is stored in a computer-readable storage medium, and when the computer program is executed by a processor of a computer device, the above-mentioned model optimization method is executed.

[0031] In an embodiment of the present application, after a computer device calls an object detection model to perform object detection on a sample image to obtain position detection information of target key points and position detection information of a reference object, it can use a position loss function to calculate a target loss value based on the position detection information of the target key points and the position detection information of the reference object; and optimize the model parameters in the object detection model based on the target loss value to obtain an optimized object detection model. By calculating the target loss value based on the position detection information of the target key points and the position detection information of the reference object, a relatively accurate target loss value can be obtained, and the model can be optimized based on the target loss value, effectively improving the detection accuracy and robustness of the object detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0033] Figure 1a This is a schematic diagram of the architecture of a model optimization system provided in an embodiment of the present application;

[0034] Figure 1b This is a schematic diagram of the architecture of another model optimization system provided in an embodiment of the present application;

[0035] Figure 2 This is a flow chart of a model optimization method provided in an embodiment of the present application;

[0036] Figure 3 This is a schematic diagram of the structure of an object detection model provided in an embodiment of the present application;

[0037] Figure 4 This is a flow chart of a model optimization method provided in an embodiment of the present application;

[0038] Figure 5a This is a schematic diagram of the positions of target key points and object detection frames provided in an embodiment of the present application;

[0039] Figure 5b is a schematic diagram of vertex vectors of an irregular detection frame provided in an embodiment of the present application;

[0040] Figure 5c This is a schematic diagram of calculating a reference vertex vector by traversing the vertices of an irregular detection box provided by an embodiment of the present application;

[0041] Figure 6 This is a flow chart of a model optimization method provided in an embodiment of the present application;

[0042] Figure 7a is a schematic diagram of a first baseline vector and a first reference vector provided in an embodiment of the present application;

[0043] Figure 7b is a schematic diagram of a second baseline vector and a second reference vector provided in an embodiment of the present application;

[0044] Figure 8 This is a flow chart of a model optimization method provided in an embodiment of the present application;

[0045] Figure 9 This is a schematic structural diagram of a model optimization device provided in an embodiment of the present application;

[0046] Figure 10 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0049] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0050] Computer vision (CV) is the science of making machines "see." Specifically, it refers to using cameras and computers to replace the human eye in identifying, tracking, and measuring objects, and further processing them to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0051] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning / deep learning typically includes techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0052] Based on the computer vision technology and machine learning technology mentioned in the above-mentioned artificial intelligence technology, the embodiment of the present application proposes a model optimization scheme to better optimize the model; in a specific implementation, the model optimization scheme can be executed by a computer device, and the general principle of the scheme is as follows: in the process of optimizing the model, a position loss function can be designed for the object detection model, and the position loss function can be used to characterize at least one of the following positional relationships: the positional relationship between key points and key points in the sample object, and the positional relationship between key points and object detection frames; then the computer device uses the position loss function to determine the loss value of the object detection model, and optimizes the object detection model according to the loss value of the object detection model; wherein, the key point of the object refers to a unique feature that can reflect the object; for example, the key point of a face can be the left eye key point, the nose tip key point, etc. wherein, the object detection model can be any object detection model based on deep learning, such as the object detection model can be a face detection model, an animal detection model, etc.

[0053] To facilitate understanding of the entire solution, the model optimization solution proposed in this application is further elaborated below, taking the object detection model as an example of the face detection model; specifically, the computer device may first obtain a picture or a video, wherein the picture or video includes a face; then the computer device calls the face detection model to perform face detection on the image in the obtained picture or video, and obtains the position information of the key points of the face and the position information of the face detection frame; then the position loss function is used to calculate the loss value of the face detection model according to the position information of the key points of the face and the position information of the face detection frame; then the face detection model is optimized according to the loss value, so that the optimized face detection model is used to detect faces in online videos or pictures, as well as related information about the face; so that subsequent face-related services are performed based on the related information of the detected face. For example, the face-related services may be face recognition, face attributes, face posture estimation and other services.

[0054] It should be noted that the above-mentioned computer device can be a terminal or a server; when the computer device is a server, the embodiment of the present application provides a model optimization system, such as Figure 1aAs shown, the model optimization system includes at least one terminal 101 and at least one server 102; the terminal 101 refers to any device equipped with a camera, and the terminal 101 can use the camera to collect sample images and upload the collected sample images to the server 102 (i.e., a computer device), so that the computer device optimizes the object detection model according to the sample images to obtain an optimized object detection model. Among them, the terminal 101 can be a smart phone, tablet computer, laptop computer, desktop computer, car computer, smart home, wearable electronic device, VR / AR device, etc.; the terminal 101 and the server 102 can be directly or indirectly connected via wired or wireless communication; the server 102 here can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms, etc.

[0055] It should be noted that Figure 1a This is just an example to describe the system architecture of the model optimization system, and does not limit the specific architecture of the model optimization system. Figure 1a The terminal and server in the example can be located outside the blockchain network; however, in other embodiments, the model optimization system can also be deployed based on the blockchain network, that is, both the terminal and the server can be deployed in the blockchain network, or the terminal can be deployed outside the blockchain network and the server can be deployed in the blockchain network, etc. When the server is deployed in the blockchain network, and the server is a server cluster or distributed system composed of multiple physical servers, each physical server can serve as a node in the blockchain network, for example Figure 1b In this example, terminal 101 is deployed outside the blockchain network, while server 102 is deployed within it. The blockchain, as mentioned here, is a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. It is essentially a decentralized database, a series of data blocks linked together using cryptographic methods.

[0056] Based on the relevant description of the above-mentioned model optimization scheme, it can be seen that in the embodiment of the present application, the object detection model is optimized by utilizing the position loss function to associate key points with key points and key points with detection frames in the sample object, which can effectively improve the detection accuracy and robustness of the object detection model, so that the subsequent optimized object detection model can more accurately detect the target object in the online video or picture, thereby improving the service quality of the business related to the target object.

[0057] Based on the model optimization solution provided above, the embodiment of the present application provides a model optimization method, which can be executed by the computer device mentioned above, which can be the terminal or server mentioned above. Figure 2 , the model optimization method may include the following steps S201-S205:

[0058] S201: Acquire a sample image, where the sample image includes a sample object.

[0059] The sample image can be any static image including one or more objects, any frame image including one or more objects in a dynamic image, or any frame image including one or more objects in a video. Each object in a sample image can be referred to as a sample object, i.e., the number of sample objects in a sample image can be one or more. For ease of explanation, the following description will take a sample image including one sample object as an example. The sample objects mentioned here can be human faces, animals, machines, etc., and are not limited in the embodiments of this application.

[0060] In a specific implementation, the computer device can acquire images in real time through its configured camera components (such as a camera carried by the computer device, an external camera device, etc.) to obtain a sample image; or the sample image can be obtained from an image resource or video resource pre-saved in the local space. Among them, the specific implementation method of obtaining a sample image from an image resource can be: if the image resource is a static image containing an object, the image resource can be directly used as a sample image; if the image resource is a dynamic image containing an object, the object recognition can be performed on each frame image in the dynamic image, and a frame image containing the object can be selected from the image resource as a sample image based on the recognition result. It should be understood that the specific implementation method of obtaining a sample image from a video resource is similar to the specific implementation method of obtaining a sample image from a dynamic image, and will not be repeated here.

[0061] S202: Calling an object detection model to perform object detection on a sample image, and obtaining position detection information of target key points of the sample object and position detection information of reference objects of the target key points.

[0062] The object detection model can be any object detection model based on deep learning. The object detection model can be determined based on the sample object to be detected. For example, if the sample object is a facial object, and the facial object is a human face, then the object detection model is a face detection model; if the facial object is a cat face, then the object detection model is a cat face detection model. The number of target key points can be one or more, for example, the number of target key points can be 1, 2, 3, etc., which is not limited in the embodiments of this application.

[0063] In a specific implementation, the computer device may first call the object detection model to extract features from the sample image to obtain sample features of the sample object, and then perform object prediction based on the sample features of the sample object to obtain position detection information of the target key point of the sample object and position detection information of the reference object of the target key point. Figure 3 As shown, the object detection model includes an input layer, a feature extraction layer, a detection box detector, a key point detector, a position loss function and an output layer; wherein, the input layer can be used to input a sample image, and the feature extraction layer can be used to extract features from the input sample image; the detection box detector can be used to predict the detection box of the extracted features; the key point detector can be used to predict the key points of the extracted features; the position loss function can be used to calculate the loss value of the object detection model; the output layer can be used to output the position detection information of the key points of the sample object, as well as the position detection information of the object detection box.

[0064] In one embodiment, since object detection on a sample image typically involves detecting the sample object enclosed by an object detection box, a position loss function can be constructed based on the positional relationship between the key points of the sample object and the object detection box to optimize the performance of the object detection model. In this case, the reference object can include an object detection box, which can be used to indicate the location of the sample object. The object detection box can be a rectangular bounding box or an irregular detection box, such as an irregular polygonal detection box. The object detection model is invoked to perform object detection on the sample image to obtain position detection information of the target key points of the sample object and position detection information of the reference object for the target key points. A specific implementation method is as follows: a computer device inputs the sample image into a feature extraction layer via an input layer, then uses the feature extraction layer to extract features from the sample image to obtain sample features of the sample object. The computer device then uses a detection box detector to predict detection boxes on the sample features to obtain object detection box position detection information, and uses a key point detector to predict key points on the sample features to obtain position detection information of the target key points of the sample object.

[0065] In another embodiment, practice shows that since a sample object generally includes one or more key points, and the key points in the sample object have a positional relationship with each other, a position loss function can be established based on the positional relationship between the key points to optimize the performance of the object detection model. In this case, when the sample object includes multiple key points, the reference object can include one or more key points in the multiple key points of the sample object except the target key point. The computer device calls Figure 3The object detection model shown in the figure uses a keypoint detector to predict keypoints for a sample feature, thereby obtaining position detection information for multiple keypoints of the sample object. These multiple keypoints include target keypoints, each of which can be used to indicate a feature of the sample object. For example, if the sample object is a human face, the two keypoints are the left eye keypoint and the right eye keypoint. The left eye keypoint can be used to indicate the left eye of the face, and the right eye keypoint can be used to indicate the right eye of the face.

[0066] In summary, the reference object may include at least one of the following: an object detection frame, and one or more key points other than the target key point among multiple key points of the sample object.

[0067] S203. Obtain a position loss function of the object detection model. The position loss function is constructed based on a target position relationship that needs to be satisfied between the target key point and the reference object.

[0068] In one embodiment, the reference object may include: an object detection frame; the position loss function is constructed based on the target position relationship that must be satisfied between the target key point and the object detection frame. At this time, the target position relationship is used to indicate that the target key point must be within the object detection frame.

[0069] In another embodiment, the reference object may include: one or more key points other than the target key point among the multiple key points of the sample object. The position loss function is constructed based on the target position relationship that the target key point and each key point other than the target key point among the multiple key points of the sample object must satisfy. In this case, the target position relationship can be used to indicate that the target key point must be located at the position corresponding to each key point other than the target key point among the multiple key points of the sample object. For example, the multiple key points of the sample object are the left eye key point, the right eye key point, and the nose key point; wherein, the target key point is assumed to be the nose key point, and the reference key point is assumed to be the left eye key point and the left corner of the mouth key point; the position loss function is constructed based on the target position relationship that the nose key point, the left eye key point, and the left corner of the mouth key point must satisfy. The target position relationship is used to indicate that the nose key point must be located at the position corresponding to the left eye key point, and the position corresponding to the left corner of the mouth key point.

[0070] In another embodiment, in order to further improve the accuracy of optimization of the object detection model, the reference object may include: an object detection frame and one or more key points of the sample object other than the target key point, and the position loss function may include a first loss function and a second loss function, wherein the first loss function may be constructed by a first position relationship between the target key point and the object detection frame, and the first position relationship is used to indicate that the target key point must be within the object detection frame; the second loss function may be constructed by a second position relationship between the target key point and one or more key points of the sample object other than the target key point, and at this time, the second position relationship is used to indicate that the target key point must be located at the position corresponding to each key point of the sample object other than the target key point.

[0071] S204: Using a position loss function, a target loss value is calculated based on the position detection information of the target key point and the position detection information of the reference object.

[0072] In a specific implementation, when the reference object is an object detection frame, the position loss function includes a first loss function; the computer device can use the first loss function to calculate the first loss value based on the position detection information of the target key point and the position detection information of the object detection frame, that is, in this case, the target loss value includes the first loss value.

[0073] When the reference object is one or more key points other than the target key point among the multiple key points of the sample object, the position loss function includes a second loss function. The computer device may use the second loss function to calculate the second loss value based on the position detection information of the target key point and the position detection information of each key point other than the target key point among the multiple key points of the sample object. That is, in this case, the target loss value includes the second loss value.

[0074] When the reference object includes an object detection box and one or more key points other than the target key point among multiple key points of the sample object, the position loss function may include a first loss function and a second loss function, and the computer device uses the first loss function to calculate the first loss value and uses the second loss function to calculate the second loss value; in this case, the target loss value of the computer device may include the first loss value and the second loss value.

[0075] S205. Optimize the model parameters of the object detection model according to the target loss value to obtain an optimized object detection model.

[0076] In one specific implementation, the computer device may optimize the model parameters of the object detection model in a direction that reduces the target loss value. In another specific implementation, the target loss value includes a first loss value and a second loss value; the computer device may calculate the model loss value of the object detection model based on the first loss value and the second loss value; and optimize the model parameters of the object detection model in a direction that reduces the model loss value.

[0077] In one embodiment, after obtaining the optimized object detection model, when the computer device detects an object detection request for an image to be processed, the computer device calls the optimized object detection model to perform object detection on the image to be processed, obtains position detection information of the key points of the target object and the position detection information of the object detection frame, and then performs business processing based on the position detection information of the key points of the target object and the position detection information of the object detection frame. The business processing can include target object detection, target object attribute recognition, target object pose estimation, etc.

[0078] In an embodiment of the present application, a computer device uses a position loss function to calculate a target loss value based on the position detection information of the target key points and the position detection information of the reference object. The computer device then optimizes the model parameters in the object detection model based on the target loss value to obtain an optimized object detection model. By calculating the target loss value based on the position detection information of the target key points and the position detection information of the reference object, a relatively accurate target loss value can be obtained, which can then be used to optimize the model, effectively improving the detection accuracy and robustness of the object detection model and ensuring the accuracy of subsequent object detection using the optimized object detection model.

[0079] Based on the model optimization solution provided above, the embodiment of the present application provides a model optimization method, which can be executed by the computer device mentioned above, which can be the terminal or server mentioned above. Figure 4 , Figure 4 The model optimization method is described by taking the reference object as the object detection frame and the position loss function including the first loss function, which may include the following steps S401-S406:

[0080] S401: Acquire a sample image, where the sample image includes a sample object.

[0081] S402: Calling an object detection model to perform object detection on a sample image, obtaining position detection information of target key points of the sample object and position detection information of a reference object of the target key points, where the reference object includes an object detection frame.

[0082] In a specific implementation, since the sample object includes one or more key points, the computer device calls the object detection model to perform object detection on the sample image, and can obtain the position detection information of multiple key points of the sample object and the position detection information of the object detection box; wherein, the multiple key points include the target key points.

[0083] S403: Obtain a position loss function of the object detection model, where the position loss function includes a first loss function.

[0084] Among them, the first loss function is constructed based on the target position relationship that must be satisfied between the target key point and the object detection frame. The target position relationship includes: a first position relationship must be satisfied between any key point and the object detection frame, and the first position relationship is used to indicate that any key point must be located within the object detection frame. When any key point is located within the object detection frame, it can be considered that any key point will not generate a loss value for the object detection model; when any key point is not located within the object detection frame, it can be considered that any key point will generate a loss value for the object detection model. In this case, the first loss function can be constructed based on the first position relationship.

[0085] It can be understood that, since any key point needs to be located within the object detection frame, it can be determined that the target key point also needs to be located within the object detection frame.

[0086] S404: Using a first loss function, calculate the loss value corresponding to the target key point based on the position detection information of the target key point and the position detection information of the object detection frame.

[0087] From the above, it can be seen that since the object detection box can be a rectangular bounding box or an irregular detection box, the first loss function is used to calculate the loss value corresponding to the target key point based on the position detection information of the target key point and the position detection information of the object detection frame. There is a certain difference. Next, when the object detection box is a rectangular bounding box and when the object detection box is an irregular detection frame, the loss value corresponding to the target key point is calculated respectively.

[0088] (1) When the object detection box is a rectangular bounding box, the rectangular bounding box includes multiple boundaries. Figure 5aThe rectangular bounding box shown includes four boundaries, namely the left boundary, the right boundary, the upper boundary, and the lower boundary. At this time, the position detection information of the target key point may include: the position coordinates of the target key point in the sample image, and the position coordinates include a horizontal coordinate value and a vertical coordinate value. The position detection information of the reference object may include: the position indication values ​​of each boundary of the object detection box in the sample image; the above-mentioned first position relationship is represented by the orientation relationship satisfied between any key point and each boundary. Among them, the orientation relationship that needs to be satisfied between any key point and each boundary is: any key point is below the upper boundary of the rectangular bounding box, above the lower boundary, to the right of the left boundary, and to the left of the right boundary.

[0089] When any key point and each boundary satisfy the following conditions: any key point is below the upper boundary of the rectangular bounding box, above the lower boundary, to the right of the left boundary, and to the left of the right boundary, any key point will not generate a loss value for the object detection model; when any key point and each boundary do not satisfy the following conditions: any key point is below the upper boundary, above the lower boundary, to the right of the left boundary, and to the left of the right boundary, that is, any key point is above the upper boundary, below the lower boundary, to the left of the left boundary, and to the right of the right boundary, any key point will generate a loss value for the object detection model; in this case, the loss value generated by the object detection model is generally negative. Figure 5a As shown, the sample object is a human face, and the sample object includes multiple key points, namely the left eye key point, the right eye key point and the nose key point. Among them, the left eye key point is to the left of the left boundary of the rectangular bounding box. At this time, the left eye key point will generate a loss value for the object detection model; while the right eye key point and the nose key point are both below the upper boundary of the rectangular bounding box, above the lower boundary, to the right of the left boundary and to the left of the right boundary. Neither the right eye key point nor the nose key point will generate a loss value for the object detection model.

[0090] In this case, the computer device adopts the first loss function to calculate the loss value corresponding to the target key point based on the position detection information of the target key point and the position detection information of the object detection frame. The specific implementation method is: the computer device traverses each boundary of the object detection frame; and obtains the position indication value of the current boundary currently traversed from the position detection information of the reference object, and then obtains the coordinate value related to the current boundary from the position coordinates of the target key point; and adopts the first loss function to calculate the loss value corresponding to the current boundary based on the position indication value of the current boundary and the obtained coordinate value.

[0091] Specifically, the computer device uses a first loss function to calculate an initial loss value corresponding to the current boundary based on the position indication value of the current boundary and the obtained coordinate value, and then determines whether the initial loss value of the current boundary is greater than an invalid value. If the initial loss value is greater than the invalid value, it is determined that the target key point satisfies the orientation relationship with the current boundary, and the loss value corresponding to the current boundary is determined to be equal to the invalid value; if the initial loss value is less than or equal to the invalid value, it is determined that the target key point does not satisfy the orientation relationship with the current boundary, and the initial loss value corresponding to the current boundary is negated to obtain the loss value corresponding to the current boundary. The invalid value can be "0". For example, the current boundary is the upper boundary. If the initial loss value corresponding to the current boundary is less than the invalid value "0", it is determined that the target key point does not meet the upper boundary; and the initial loss value corresponding to the current boundary is negated to obtain the loss value corresponding to the current boundary, where a number usually has an operator sign, which can be positive or negative; and the so-called negation operation refers to the operation of changing the operator sign of the number. For example, if the initial loss value is "-0.5", it indicates that the operator sign of the initial loss value is negative, then performing the negation operation on the initial loss value means: changing the negative operator sign of the initial loss value from negative to positive, thereby obtaining the loss value of the current boundary as 0.5.

[0092] Furthermore, after all boundaries are traversed, the loss values ​​corresponding to each boundary can be summed to obtain the loss value corresponding to the target key point.

[0093] For example, in Figure 5a In the figure, the upper left vertex of the object detection frame is used as the origin of the coordinate axis, the target key point is the left eye key point, and the position coordinates of the left eye key point are (xi, yi); wherein the horizontal coordinate value of the position coordinate is xi, and the vertical coordinate value is yi; the position detection information of the reference object includes the position indication value u of the upper boundary, the position indication value d of the lower boundary, the position indication value l of the left boundary, and the position indication value r of the right boundary. The computer device traverses the four boundaries of the object detection frame. If the current boundary traversed is the upper boundary, the computer device obtains the position indication value u of the upper boundary from the position detection information of the reference object; and obtains the coordinate value yi related to the upper boundary from the position coordinates of the left eye key point; then the first loss function is used to subtract the obtained coordinate value yi from the position indication value u of the upper boundary to obtain the initial loss value corresponding to the upper boundary, and judge whether the initial loss value corresponding to the upper boundary is greater than the invalid value 0; in the figure, Figure 5a In the example, the computer device determines that the initial loss value corresponding to the upper boundary is greater than the invalid value 0, indicating that the left eye key point satisfies the condition below the upper boundary, and the computer device determines that the loss value corresponding to the upper boundary is equal to the invalid value 0.

[0094] Then, the computer device continues to traverse the three boundaries except the upper boundary. If the current interface traversed is the left boundary, the computer device obtains the position indication value of the left boundary as l from the position detection information of the reference object, and obtains the coordinate value xi related to the left boundary from the position coordinates of the left eye key point. The computer device then uses the first loss function to subtract the position indication value d of the left boundary from the obtained coordinate value xi to obtain the initial loss value corresponding to the left boundary; and determines whether the initial loss value corresponding to the left boundary is greater than the invalid value 0. Figure 5a In the example, the computer device determines that the initial loss value corresponding to the left boundary is less than the invalid value 0, indicating that the left eye key point does not satisfy the requirement of being to the right of the left boundary. The computer device performs a negation operation on the initial loss value corresponding to the left boundary to obtain the loss value corresponding to the left boundary. Similarly, after all four boundaries are traversed, the computer device sums the loss values ​​corresponding to the four boundaries to obtain the corresponding loss value of the left eye key point.

[0095] The functional expression of the above first loss function is:

[0096] f(xi,yi,u,d,l,r)=-min(yi-u,0)-min(d-yi,0)-min(xi-l,0)-min(r-xi,0)

[0097] Among them, f(xi,yi,u,d,l,r) represents the loss value corresponding to the target key point; (xi,yi) represents the position coordinates of the target key point, (u,d,l,r) represents the object detection box, that is, the position indication value of the upper boundary of the object detection box is represented as u, the position indication value of the lower boundary is represented as d, the position indication value of the left boundary is l, and the position indication value of the right boundary is r; the invalid value is 0.

[0098] It should be noted that in a specific implementation, any vertex of the object detection box can be selected as the coordinate axis and the origin. Then, according to the first loss function described above, the loss value corresponding to the target key point is calculated based on the position coordinates of the target key point and the position coordinates of the object detection box. In this case, the first loss function will be different depending on the selected origin.

[0099] (2) When the object detection frame is an irregular detection frame, such as Figure 5bThe irregular detection frame shown. At this time, the position detection information of the target key point includes: the position coordinates of the target key point in the sample image; the position detection information of the reference object includes: the position coordinates of each vertex of the object detection frame in the sample image; the object detection frame is represented by multiple vertex vectors, and any vertex vector is determined based on the position coordinates of two adjacent vertices of the object detection frame. Specifically, the computer device can traverse multiple vertices in the object detection frame and use the current vertex currently traversed as the starting point; then the computer device determines the adjacent vertices of the current vertex and uses the adjacent vertices of the current vertex as the end point; then, based on the position coordinates of the current vertex and the position coordinates of the adjacent vertices, determine the vertex vector of the current vertex. When multiple vertices are traversed, the computer device generates multiple vertex vectors, and the multiple vertex vectors are used to represent the object detection frame.

[0100] For example, in Figure 5b In the figure, the object detection frame is an irregular quadrilateral, and the position coordinates of the four vertices of the irregular quadrilateral are: the position coordinates of the first vertex (bx1, by1), the position coordinates of the second vertex (bx2, by2), the position coordinates of the third vertex (bx3, by3), and the position coordinates of the fourth vertex (bx4, by4). The computer device traverses the four vertices of the irregular quadrilateral and takes the currently traversed first vertex as the starting point. The computer device then determines that the adjacent vertex of the first vertex is the second vertex and takes the second vertex as the end point. The computer device then obtains the vertex vector V_b1 corresponding to the first vertex based on the position coordinates of the first vertex (bx1, by1) and the position coordinates of the second vertex (bx2, by2). The computer device then continues to traverse the remaining three vertices and takes the currently traversed second vertex as the starting point. The computer device then determines The adjacent vertex of the second vertex is the third vertex, and the third vertex is used as the end point; then, the computer device obtains the vertex vector V_b2 = (bx3-bx2, by3-by2) corresponding to the second vertex based on the position coordinates of the second vertex and the position coordinates of the third vertex; then the computer device continues to traverse the remaining 2 vertices until all vertices are traversed, and finally the vertex vector V_b3 = (bx4-bx3, by4-by3) corresponding to the third vertex and the vertex vector V_b4 = (bx1-bx4, by1-by4) corresponding to the fourth vertex can be obtained.

[0101] After determining multiple vertex vectors, the first positional relationship can be represented by the orientation relationship that must be satisfied between any key point and each vertex vector. Since each currently traversed vertex is used as the starting point when determining the vertex vector, and the adjacent vertex of the currently traversed vertex is determined as the end point, the multiple vertex vectors finally obtained should be connected at the end. Based on this, the orientation relationship that must be satisfied between any key point and each vertex vector can be: any key point needs to be on the right side of each vertex vector or any key point needs to be on the left side of each vertex vector. For example, Figure 5b In the example, any key point must be on the right side of each vertex vector. In practice, the required orientation relationship between any key point and each vertex vector is determined by how the vertex vectors are constructed.

[0102] In this case, the computer device uses the first loss function to calculate the loss value corresponding to the target key point based on the position detection information of the target key point and the position detection information of the object detection frame. The specific implementation method is as follows: the computer device traverses multiple vertices of the object detection frame and determines the target vertex vector corresponding to the current vertex currently traversed. The target vertex vector refers to the vertex vector constructed using the current vertex as the starting point; and obtains the position coordinates of the current vertex from the position detection information of the reference object, and then calculates the reference vertex vector between the current vertex and the target key point based on the position coordinates of the current vertex and the position coordinates of the target key point; then uses the first loss function to calculate the loss value corresponding to the current vertex based on the reference vertex vector and the target vertex vector. Specifically, the computer device uses the first loss function to perform an outer product operation on the reference vertex vector and the target vertex vector to obtain the outer product result corresponding to the current vertex.

[0103] The computer device then determines whether the outer product result corresponding to the current vertex is greater than an invalid value. If the outer product result corresponding to the current vertex is greater than the invalid value, it is considered that the target key point is located at the target position of the target vertex vector, and the target key point will not generate a loss value for the object detection model, and the loss value corresponding to the current vertex is determined to be equal to the invalid value; if the outer product result corresponding to the current vertex is less than or equal to the invalid value, it is considered that the target key point is not located at the target position of the target vertex vector, and the target key point generates a loss value for the object detection model, and the outer product result corresponding to the current vertex is a negative value. The computer device performs a negation operation on the outer product result corresponding to the current vertex to obtain the loss value corresponding to the current vertex.

[0104] Then, after all vertices of the object detection box are traversed, the loss values ​​corresponding to each vertex are summed to obtain the loss value corresponding to the target key point.

[0105] For example, in Figure 5cIn this example, the position coordinates of the target key point are (xi, yi); the target key point needs to be located to the right of each vertex vector; the computer device traverses the four vertices of the object detection box and determines that the target vertex vector corresponding to the first vertex currently traversed is V_b1 = (bx2-bx1, by2-by1), where the target vertex vector is the vertex vector constructed with the first vertex as the starting point; the computer device then obtains the position coordinates of the first vertex from the position detection information of the reference object as (bx1, by1), and calculates the reference vertex vectors (xi-bx1, yi-by1) of the first vertex and the target key point based on the position coordinates of the first vertex (bx1, by1) and the position coordinates of the target key point (xi, yi). Next, the computer device uses a first loss function to perform an outer product operation on the reference vertex vector (xi-bx1, yi-by1) and the target vertex vector V_b1 = (bx2-bx1, by2-by1), obtaining the outer product result corresponding to the first vertex.

[0106] Then, the computer device determines whether the outer product result corresponding to the first vertex is greater than the invalid value 0; Figure 5c , the computer device determines that the outer product result corresponding to the first vertex is greater than the invalid value 0, indicating that the target key point is to the right of the target vertex vector. The computer device determines that the loss value corresponding to the first vertex is equal to the invalid value 0. The computer device then continues to traverse the remaining 3 vertices and determines that the target vertex vector corresponding to the second vertex currently traversed is V_b2 = (bx3-bx2, by3-by2). The target vertex vector refers to the vertex vector constructed with the second vertex as the starting point. The computer device then obtains the position coordinates of the second vertex from the position detection information of the reference object as (bx2, by2), and calculates the reference vertex vector (xi-bx2, yi-by2) of the second vertex and the target key point based on the position coordinates (bx2, by2) of the second vertex and the position coordinates (xi, yi).

[0107] Next, the computer device uses the first loss function to perform an outer product operation on the reference vertex vector (xi-bx2, yi-by2) and the target vertex vector V_b2 = (bx3-bx2, by3-by2) to obtain the outer product result corresponding to the second vertex. The computer device then determines whether the outer product result corresponding to the second vertex is greater than the invalid value 0. Figure 5cIn the example, when the computer device determines that the outer product result corresponding to the second vertex is greater than the invalid value 0, it means that the target key point is to the right of the target vertex vector, and the computer device determines that the loss value corresponding to the second vertex is equal to the invalid value. The computer device then continues to traverse the remaining two vertices, and finally obtains the loss values ​​corresponding to the four vertices. The loss values ​​corresponding to the four vertices are then summed to obtain the loss value corresponding to the target key point. The functional expression of the above-mentioned first loss function is:

[0108] f(xi,yi,V_b(j))=sum(-min((xi-bxj,yi-byj)×V_b(j),0)),j∈[1,m]

[0109] Among them, f(xi, yi, V_b(j)) represents the loss value corresponding to the target key point; j represents the vertex number of the irregular detection box, j∈[1,m] represents the number of vertices; (xi, yi) represents the coordinate position of the target key point; (bxj, byj) represents the position coordinate of the current vertex, bxj represents the horizontal coordinate value in the position coordinate of the current vertex, byj represents the vertical coordinate value in the position coordinate of the current vertex, and V_b(j) represents the vertex vector of the current vertex.

[0110] It should be noted that Figure 5b A schematic diagram of an irregular detection frame is given for illustrative purposes. In practical applications, the irregular detection frame may be an irregular detection frame formed by polygons, for example, an irregular detection frame formed by pentagons.

[0111] S405: Determine a first loss value according to the loss value corresponding to the target key point.

[0112] In a specific implementation, the computer device may directly determine the loss value corresponding to the target key point as the first loss value.

[0113] In another specific implementation, when the reference object also includes one or more key points other than the target key point among the multiple key points of the sample object, the computer device uses a first loss function to calculate a loss value corresponding to each key point in the reference object based on the position detection information of each key point in the reference object and the position detection information of the object detection frame. The computer device then sums the loss value corresponding to the target key point and the loss value corresponding to each key point in the reference object to obtain a first loss value. Due to differences in object detection frames, the calculation method of this first loss value varies.

[0114] (1) When the object detection box is a rectangular bounding box, the first loss value is calculated as follows:

[0115] Loss_bbox=sum f(xi,yi,u,d,l,r),i∈[0,M]

[0116] Among them, Loss_bbox represents the first loss value, sum represents summation; i represents the key point, and i takes an integer; M represents the number of key points; f(xi,yi,u,d,l,r) represents the loss value corresponding to each key point, when i takes any integer in [0,M].

[0117] (2) When the object detection frame is an irregular detection frame, the first loss value is calculated as follows:

[0118] Loss_bbox=sum{(i∈[0,M])}f(xi,yi,V_b(j))

[0119] Among them, Loss_bbox represents the first loss value, sum represents summation; i represents the key point, and i takes an integer; M represents the number of key points, and f(xi,yi,V_b(j)) represents the loss value corresponding to each key point.

[0120] It should be noted that the computer device adopts the first loss function to calculate the loss value corresponding to each key point in the reference object based on the position detection information of each key point in the reference object and the position detection information of the object detection frame. Please refer to the above-mentioned specific implementation method of using the first loss function to calculate the loss value corresponding to the target key point based on the position detection information of the target key point and the position detection information of the object detection frame, which will not be repeated here.

[0121] In one embodiment, if the sample object is a facial object, although there may be many key points, there will be annotations such as nose, left eye, right eye, left corner of mouth and right corner of mouth, and the facial object can be detected through these five key points. Among them, the five key points include the left eye key point, the right eye key point, the nose key point, the left corner of mouth key point and the right corner of mouth key point; these five key points include the target key point. Specifically, the computer device can filter out the left eye key point, the right eye key point, the nose key point, the left corner of mouth key point and the right corner of mouth key point from multiple key points, and then sum the loss value corresponding to the left eye key point, the loss value corresponding to the right eye key point, the loss value corresponding to the nose key point, the loss value corresponding to the left corner of mouth key point and the loss value corresponding to the right corner of mouth key point to obtain a first loss value. Among them, there are two ways to calculate the first loss value:

[0122] (1) When the object detection box is a rectangular bounding box, since the number of key points is 5, M = 4, and the calculation method of the first loss value is:

[0123] Loss_bbox=sum f(xi,yi,u,d,l,r),i∈[0,4]

[0124] Among them, Loss_bbox represents the first loss value, sum represents summation, i represents the key point, and i takes an integer; the position coordinates of these five key points can be recorded as (xi, yi) (i = 0, 1, 2, 3, 4), and correspond to the position coordinates of the left eye key point, the position coordinates of the right eye key point, the position coordinates of the nose key point, the position coordinates of the left corner of the mouth key point, and the position coordinates of the right corner of the mouth key point respectively; f(xi, yi, u, d, l, r) represents the loss value corresponding to each key point.

[0125] (2) When the object detection frame is an irregular detection frame, since the number of key points is 5, M = 4, and the calculation method of the first loss value is:

[0126] Loss_bbox=sum{(i∈[0,4])}f(xi,yi,V_b(j))

[0127] Among them, Loss_bbox represents the first loss value, sum represents summation, i represents the key point, and i takes an integer; the position coordinates of these five key points can be recorded as (xi, yi) (i = 0, 1, 2, 3, 4); f(xi, yi, V_b(j)) represents the loss value corresponding to each key point.

[0128] S406. Optimize model parameters of the object detection model according to the first loss value to obtain an optimized object detection model.

[0129] In a specific implementation, the computer device optimizes the model parameters of the object detection model in a direction in which the first loss value decreases to obtain an optimized object detection model.

[0130] In one embodiment, when optimizing an object detection model (e.g., a face detection model, a face key point detection model, etc.), a reference loss function may be introduced, and the model parameters of the object detection model may be optimized by combining the first loss value and the reference loss value corresponding to the reference loss function. In a specific implementation, the computer device may obtain the annotation information of the sample image, and then use the reference loss function to calculate the reference loss value based on the annotation information of the sample image; and obtain the model loss value of the object detection model based on the first loss value and the reference loss value, and then optimize the model parameters of the object detection model in the direction of reducing the model loss value of the object detection model to obtain the optimized object detection model.

[0131] The annotation information includes at least one of the following: position annotation information of each key point of the sample object, and position annotation information of the object detection box; the reference loss function may include at least one of the following: a key point loss function and an object detection box loss function, and the key point loss function and the object detection box loss function may be constructed using the L2 loss function of regression respectively. The reference loss value includes at least one of the following: a key point detection loss value and an object box detection loss value.

[0132] Specifically, when the annotation information is the position annotation information of each key point of the sample object, the reference loss function is the key point detection loss function, and the computer device uses the key point detection loss function to perform loss calculation on the position annotation information and position detection information of each key point of the sample object to obtain the key point loss value; when the annotation information is the position annotation information of the object detection box, the reference loss function is the object detection box loss function, and the computer device uses the object detection box loss function to perform loss calculation on the position annotation information of the object detection box and the position detection information of the object detection box to obtain the object detection box loss value; when the annotation information includes: the position annotation information of each key point of the sample object, and the position annotation information of the object detection box, the object detection box loss function and the key point detection loss function can be used respectively for calculation to obtain the key point loss value and the detection box loss value.

[0133] The specific implementation method of obtaining the model loss value of the object detection model based on the first loss value and the reference loss value is as follows: the computer device sums the first loss value and the reference loss value to obtain the model loss value of the object detection model; or, the computer device weights the first loss value and the reference loss value to obtain the model loss value of the object detection model.

[0134] In an embodiment of the present application, a computer device uses a first loss function to calculate the loss value corresponding to the target key point based on the position detection information of the target key point and the position detection information of the object detection frame, and then determines a first loss value based on the loss value corresponding to the target key point. The model parameters in the object detection model are then optimized based on the first loss value to obtain an optimized object detection model. By calculating the first loss value based on the position detection information of the target key point and the position detection information of the object detection frame, a relatively accurate first loss value can be obtained, so that the model can be optimized based on the first loss value, effectively improving the detection accuracy and robustness of the object detection model, and ensuring the accuracy of subsequent object detection using the optimized object detection model.

[0135] Based on the model optimization solution provided above, the embodiment of the present application also provides a model optimization method, which can be executed by the computer device mentioned above, and the computer device can be the terminal or server mentioned above. Figure 6 , Figure 6 The model optimization method is described with the reference object as the first key point and the second key point, the position loss function includes the second loss function, and the target loss value includes the second loss value of the second loss function, which may include the following steps S601-S607:

[0136] S601: Acquire a sample image, where the sample image includes a sample object.

[0137] S602. Call the object detection model to perform object detection on the sample image to obtain position detection information of the target key point of the sample object and position detection information of the reference object of the target key point, where the reference object includes a first key point and a second key point. The position detection information of the reference object includes: the position coordinates of the first key point in the sample image, and the position coordinates of the second key point in the sample image.

[0138] The position coordinates include a horizontal coordinate value and a vertical coordinate value.

[0139] In a specific implementation, a computer device calls an object detection model to perform object detection on a sample image, and obtains position detection information of multiple key points of the sample object; wherein the multiple key points include a target key point. Then the computer device selects a first key point and a second key point from the multiple key points. It should be noted that the first key point and the second key point refer to the key points other than the target key point in the multiple keys, the number of the first key points can be one or more, and the number of the second key points can be one or more. For example, the sample object is a human face, and the multiple key points of the human face include a nose key point, a left eye key point, a right eye key point, a left corner of the mouth key point, and a right corner of the mouth key point; the target key point is the nose key point, the two first key points are the left eye key point and the right eye key point, and the two second key points are the left corner of the mouth key point and the right corner of the mouth key point.

[0140] S603: Obtain a position loss function of the object detection model, where the position loss function includes a second loss function.

[0141] The target position relationship may include at least one of the following: a second position relationship must be satisfied between the target key point and the first key point, and a third position relationship must be satisfied between the target key point and the second key point; the second position relationship is used to indicate that the target key point must be located at the first position of the first key point. The third position relationship is used to indicate that the target key point must be located at the second position of the second key point. When the target key point is located at the first position of the first key point or the second position of the second key point, the target key point will not generate a loss value for the object detection model; when the target key point is not located at the first position of the first key point or the target key point is not located at the second position of the second key point, the target key point will generate a loss value for the object detection model.

[0142] In this case, the second loss function can be constructed based on at least one of the second positional relationship and the third positional relationship. Accordingly, the second loss value of the second loss function is any one of the following: a first loss sub-value calculated based on the target key point and the first key point, or a second loss sub-value calculated based on the target key point and the second key point, or the sum of the first loss sub-value and the second loss sub-value. Specifically, the second loss function is constructed based on the second positional relationship, and the second loss value is: the first loss sub-value calculated based on the target key point and the first key point; or, the second loss function is constructed based on the third positional relationship, and the second loss value is the second loss sub-value calculated based on the target key point and the second key point; or, the second loss function is constructed based on the second positional relationship and the third positional relationship, and the second loss value is the sum of the first loss sub-value and the second loss sub-value.

[0143] S604: Using the second loss function, calculate the first loss sub-value according to the position coordinates of the target key point and the position coordinates of the first key point.

[0144] As can be seen from the foregoing, the number of the first key points can be one or more.

[0145] (1) When the number of the first key point is one, the computer device uses the second loss function to calculate the first loss sub-value according to the position coordinates of the target key point and the position coordinates of the first key point as follows:

[0146] The computer device may subtract the horizontal coordinate value of the target key point from the horizontal coordinate value of the first key point to obtain an initial horizontal coordinate loss value of the target key point; then determine whether the initial horizontal coordinate loss value of the target key point is greater than an invalid value. If the initial horizontal coordinate loss value of the target key point is greater than the invalid value, it is determined that the target key point is in the first position of the first key point, and the horizontal coordinate loss value of the target key point is equal to the invalid value. If the initial horizontal coordinate loss value of the target key point is less than or equal to the invalid value, it is determined that the target key point is not in the first position of the first key point, and the initial horizontal coordinate loss value is negated to obtain the horizontal coordinate loss value of the target key point.

[0147] Next, the computer device subtracts the vertical coordinate value of the target key point from the vertical coordinate value of the first key point to obtain the initial vertical coordinate loss value of the target key point; then determines whether the initial vertical coordinate loss value of the target key point is greater than the invalid value. If the initial vertical coordinate loss value is greater than the invalid value, it is determined that the target key point is in the first position of the first key point, and the invalid value is determined as the vertical coordinate loss value of the target key point; if the initial vertical coordinate loss value is less than or equal to the invalid value, it is determined that the target key point is not in the first position of the first key point, and the initial vertical coordinate value is negated to obtain the vertical coordinate loss value of the target key point; then the computer device sums the horizontal coordinate loss value and the vertical coordinate loss value to obtain the first loss sub-value.

[0148] For example, the sample object is a human face, and the target key point is the left eye key point, and the position coordinates of the left eye key point in the sample image are (x1, y1). The first key point is the nose key point, and the position coordinates of the nose key point in the sample image are (x2, y2). The first orientation is that the left eye key point must be located to the upper left of the nose key point. The computer device subtracts the horizontal coordinate x1 of the left eye key point from the horizontal coordinate x2 of the nose key point to obtain the initial horizontal coordinate loss value of the left eye key point, and determines whether the initial horizontal coordinate loss value is greater than an invalid value. If the initial horizontal coordinate loss value is greater than an invalid value, it is considered that the left eye key point is located to the upper left of the nose key point, and the horizontal coordinate loss value of the left eye key point is determined to be equal to the invalid value. If the initial horizontal coordinate value is less than or equal to an invalid value, it is considered that the left eye key point is not located to the upper left of the nose key point, and the initial horizontal coordinate value is inverted to obtain the horizontal coordinate loss value of the left eye key point.

[0149] The computer device then subtracts the vertical coordinate y1 of the left eye keypoint from the vertical coordinate y2 of the nose keypoint to obtain an initial vertical coordinate loss value for the left eye keypoint, and determines whether the initial vertical coordinate loss value is greater than an invalid value. If the initial vertical coordinate loss value is greater than the invalid value, the left eye keypoint is determined to be above and left of the nose keypoint, and the vertical coordinate loss value of the left eye keypoint is determined to be equal to the invalid value. If the initial vertical coordinate value is less than or equal to the invalid value, the left eye keypoint is determined not to be above and left of the nose keypoint, and the initial vertical coordinate value is negated to obtain the vertical coordinate loss value of the left eye keypoint. The computer device then sums the horizontal coordinate loss value and the vertical coordinate loss value of the left eye keypoint to obtain a first loss sub-value corresponding to the left eye keypoint.

[0150] (2) When the number of the first key points is two, the computer device uses the second loss function to calculate the first loss sub-value according to the position coordinates of the target key point and the position coordinates of the first key point. The specific implementation method may include the following steps s11-s12:

[0151] s11. Calculate a first reference vector according to the position coordinates of the two first key points, and calculate a first reference vector according to the position coordinates of the starting point of the first reference vector and the position coordinates of the target key point.

[0152] In a specific implementation, the computer device may first select any one of the two first key points as the starting point and the other first key point as the ending point to construct the direction of the vector, and then calculate the first reference vector based on the position coordinates of the two first key points. The first orientation refers to the orientation that must be satisfied between the target key point and the first reference vector.

[0153] For example, Figure 7a As shown, the sample object is a human face, the target key point is the nose key point, the position coordinates of the nose key point are (x2, y2), the two first key points are the left eye key point and the right eye key point; the position coordinates of the left eye key point are (x0, y0), and the position coordinates of the right eye key point are (x1, y1). The computer device selects the left eye key point as the starting point and the right eye key point as the end point to construct the direction of the vector, and then calculates the first reference vector V_eye = (x1-x0, y1-y0) based on the position coordinates of the left eye key point and the right eye key point. That is, Figure 7aIn the example, the direction of the first reference vector is from left to right (i.e., the left eye keypoint points to the right eye keypoint). In this case, the first orientation means that the nose keypoint is to the right of the first reference vector formed by the left and right eye keypoints. The computer device then determines the left eye keypoint as the starting point of the first reference vector based on the direction of the first reference vector V_eye and calculates the first reference vector V_n1 = (x2 - x0, y2 - y0) based on the position coordinates of the left eye keypoint and the nose keypoint.

[0154] S12. Calculate a first loss sub-value using a second loss function based on a first outer product result of the first reference vector and the first reference vector. The first outer product result indicates whether the target keypoint is located at a first orientation of the first reference vector; and if the target keypoint is not located at the first orientation, a first loss sub-value is generated.

[0155] In a specific implementation, the computer device uses a second loss function to perform an outer product operation on the first baseline vector and the first reference vector to obtain a first outer product result. It should be noted that when performing the outer product operation on the first baseline vector and the first reference vector, it is necessary to determine the positions of the first baseline vector and the second reference vector in the outer product operation based on the first orientation; then determine whether the first outer product result is greater than an invalid value. If the first outer product result is greater than the invalid value, it is determined that the target key point is located at the first orientation of the first baseline vector, and the first loss sub-value is determined to be equal to the invalid value; if the first outer product result is less than or equal to the invalid value, it is determined that the target key point is not located at the first orientation of the first baseline vector, and the first loss sub-value is determined based on the first outer product result.

[0156] For example, in the above step s11, after the computer device determines the first reference vector V_eye = (x1-x0, y1-y0) and the first reference vector V_n1 = (x2-x0, y2-y0), the second loss function can be used to perform an outer product operation on the first reference vector V_eye = (x1-x0, y1-y0) and the first reference vector V_n1 = (x2-x0, y2-y0) to obtain a first outer product result, which is V_eye×V_n1. Practice has shown that when the outer product result of a×b is a positive number, it means that a is to the right of b; when the outer product result of a×b is a negative number, it means that a is to the left of b. In an embodiment of the present application, when the first outer product result is a positive number, the nose key point is located to the right of the first reference vector formed by the left eye key point and the right eye key point; when the first outer product result is a negative number, the nose key point is located to the left of the first reference vector formed by the left eye key point and the right eye key point. In Figure 7a, the computer device determines that the first outer product result is greater than the invalid value 0, the computer device determines that the nose key point is located to the right of the first reference vector formed by the left eye key point and the right eye key point, and determines that the first loss sub-value is equal to the invalid value.

[0157] In one embodiment, the computer device may directly use the first loss sub-value as the second loss value and execute step S607.

[0158] In another embodiment, after determining the first loss sub-value, the computer device may further execute step S605 in order to further improve the performance of the model.

[0159] S605: Using the second loss function, calculate the second loss sub-value according to the position coordinates of the target key point and the position coordinates of the second key point.

[0160] As can be seen from the above, the number of the second key points is one or more.

[0161] (1) When the number of the second key point is one, the computer device uses the second loss function to calculate the second loss sub-value according to the position coordinates of the target key point and the position coordinates of the second key point as follows:

[0162] The computer device may subtract the horizontal coordinate value of the target key point from the horizontal coordinate value of the second key point to obtain an initial horizontal coordinate loss value of the target key point; then determine whether the initial horizontal coordinate loss value of the target key point is greater than an invalid value. If the initial horizontal coordinate loss value of the target key point is greater than the invalid value, it is determined that the target key point is in the second position of the second key point, and the horizontal coordinate loss value of the target key point is equal to the invalid value. If the initial horizontal coordinate loss value of the target key point is less than or equal to the invalid value, it is determined that the target key point is not in the second position of the second key point, and the initial horizontal coordinate loss value is negated to obtain the horizontal coordinate loss value of the target key point.

[0163] Next, the computer device subtracts the vertical coordinate value of the target key point from the vertical coordinate value of the second key point to obtain the initial vertical coordinate loss value of the target key point; then determines whether the initial vertical coordinate loss value of the target key point is greater than the invalid value. If the initial vertical coordinate loss value is greater than the invalid value, it is determined that the target key point is in the second position of the second key point, and the invalid value is determined as the vertical coordinate loss value of the target key point; if the initial vertical coordinate loss value is less than or equal to the invalid value, it is determined that the target key point is not in the second position of the second key point, and the initial vertical coordinate value is inverted to obtain the vertical coordinate loss value of the target key point; then the computer device sums the horizontal coordinate loss value and the vertical coordinate loss value to obtain the second loss sub-value corresponding to the target key point.

[0164] It should be noted that, according to actual needs, the computer device can also directly use the horizontal coordinate loss value or the vertical coordinate loss value as the second loss sub-value.

[0165] For example, the sample object is a human face. Assume that the target keypoint is the left eye keypoint, and the position coordinates of the left eye keypoint in the sample image are (x1, y1). The second keypoint is the left corner of the mouth keypoint, and the position coordinates of the left corner of the mouth keypoint in the sample image are (x3, y3). The second orientation is that the left eye keypoint must be above the left corner of the mouth keypoint. The computer device subtracts the vertical coordinate y1 of the left eye keypoint from the vertical coordinate y3 of the left corner of the mouth keypoint to obtain an initial vertical coordinate loss value of the left eye keypoint, and determines whether the initial vertical coordinate loss value is greater than an invalid value. If the initial vertical coordinate loss value is greater than the invalid value, it is determined that the left eye keypoint is above the left corner of the mouth keypoint, and the vertical coordinate loss value of the left eye keypoint is determined to be equal to the invalid value. If the initial vertical coordinate value is less than or equal to the invalid value, it is determined that the left eye keypoint is not above the left corner of the mouth keypoint, and the initial vertical coordinate value is inverted to obtain the vertical coordinate loss value of the left eye keypoint. Then, the computer device directly uses the vertical coordinate loss value as the second loss sub-value corresponding to the left eye keypoint.

[0166] (2) When the number of the second key points is two, the computer device uses the second loss function to calculate the second loss sub-value according to the position coordinates of the target key point and the position coordinates of the second key point. The specific implementation method may include the following steps s21-s22:

[0167] s21. Calculate a second reference vector according to the position coordinates of the two second key points, and calculate a second reference vector according to the position coordinates of the starting point of the second reference vector and the position coordinates of the target key point.

[0168] In a specific implementation, the computer device may first select any one of the two second key points as a starting point, use the other second key point as an end point to construct a vector direction, and then calculate the second reference vector based on the position coordinates of the two second key points. The second orientation refers to the orientation that must be satisfied between the target key point and the second reference vector.

[0169] Exemplarily, when the sample object is a facial object, the target key point is the nose key point, the two first key points include the left eye key point and the right eye key point, and the straight line between the left eye key point and the right eye key point is the eye corner line; the two second key points include the left corner of the mouth key point and the right corner of the mouth key point, and the straight line between the left corner of the mouth key point and the right corner of the mouth key point is the mouth corner line; practice has shown that the nose key point should be located in the middle of the eye corner line and the mouth corner line; this practice is applicable to faces with large postures, rotated faces, etc.; the rotated face refers to a face with the head tilted to the left or right. At this time, the first orientation and the second orientation are as follows: (1) When the direction of the first reference vector formed by the left eye key point and the right eye key point is from the left eye key point to the right eye key point, and the direction of the second reference vector formed by the left corner of the mouth key point and the right corner of the mouth key point is from the left corner of the mouth key point to the right corner of the mouth key point, the first orientation is to the right of the first reference vector formed by the left eye key point and the right eye key point; the second orientation is to the left of the second reference vector formed by the left corner of the mouth key point and the right corner of the mouth key point. (2) When the direction of the first reference vector formed by the left eye key point and the right eye key point is from the right eye key point to the left eye key point, and the direction of the second reference vector formed by the left corner of the mouth key point and the right corner of the mouth key point is from the right corner of the mouth key point to the left corner of the mouth key point, the first orientation is to the left of the first reference vector formed by the left eye key point and the right eye key point; the second orientation is to the right of the second reference vector formed by the left corner of the mouth key point and the right corner of the mouth key point.

[0170] For example, in Figure 7b , when the two second key points selected by the computer device from multiple key points are the left corner of the mouth key point and the right corner of the mouth key point, the position coordinates of the left corner of the mouth key point are (x3, y3), and the position coordinates of the right corner of the mouth are (x4, y4). The computer device selects the left corner of the mouth key point as the starting point and the right corner of the mouth key point as the ending point to construct the direction of the vector, and then calculates the second reference vector V_mouth = (x4-x3, y4-y3) based on the position coordinates of the left corner of the mouth key point and the position coordinates of the right corner of the mouth key point. That is, the direction of the second reference vector is from left to right (that is, the left corner of the mouth key point points to the direction of the right corner of the mouth key point), and the second orientation is that the nose key point is to the left of the second reference vector formed by the left corner of the mouth key point and the right corner of the mouth key point.

[0171] Then, the computer device determines the left corner of the mouth key point as the starting point of the second reference vector based on the second reference vector V_mouth, and calculates the second reference vector V_n2 = (x2-x3, y2-y3) based on the position coordinates of the left corner of the mouth key point and the position coordinates of the nose key point.

[0172] S22. Calculate a second loss sub-value using a second loss function based on a second outer product result of the second reference vector and the second reference vector. The second outer product result indicates whether the target keypoint is located at a second orientation of the second reference vector. If the target keypoint is not located at the second orientation, a second loss sub-value is generated.

[0173] In a specific implementation, the computer device uses a second loss function to perform an outer product operation on the second reference vector and the second reference vector to obtain a second outer product result; then determines whether the second outer product result is greater than an invalid value. If the second outer product result is greater than the invalid value, it is determined that the target key point is located at the second orientation of the second reference vector, and the second loss sub-value is determined to be equal to the invalid value; if the second outer product result is less than or equal to the invalid value, it is determined that the target key point is not located at the second orientation of the second reference vector, and the second loss sub-value is determined based on the second outer product result. For example, in the above step s21, after the computer device determines the second reference vector V_mouth=(x4-x3, y4-y3) and the second reference vector V_n2=(x2-x3, y2-y3), the second loss function can be used to perform an outer product operation on the second reference vector V_mouth=(x4-x3, y4-y3) and the second reference vector V_n2=(x2-x3, y2-y3) to obtain a second outer product result, which is V_mouth×V_n2. Practice has shown that when the second outer product result is a positive number, the nose key point is located on the left side of the second reference vector formed by the left and right corners of the mouth key points; when the second outer product result is a negative number, the nose key point is located on the right side of the second reference vector formed by the left and right corners of the mouth key points. Figure 7b In the embodiment, the computer device determines that the second outer product result is greater than the invalid value 0, the computer device determines that the nose key point is on the left side of the second reference vector formed by the left corner of the mouth key point and the right corner of the mouth key point, and determines that the second vector loss value is equal to the invalid value.

[0174] In one embodiment, the computer device may directly use the second loss sub-value as the second loss value and execute step S607.

[0175] S606 : Perform a sum operation on the first loss sub-value and the second loss sub-value to obtain a second loss value.

[0176] Among them, when the number of first key points is two and the number of second key points is two, the second loss function Loss_landmark is:

[0177] Loss_landmark=-min(V_n1×V_eye,0)-min(V_mouth×V_n2,0)

[0178] Among them, -min(V_n1×V_eye,0) represents the first loss sub-value; 0 is an invalid value;

[0179] -min(V_mouth×V_n2,0) represents the second loss sub-value.

[0180] Since the “×” above represents the outer product operation, the second loss function Loss_landmark can also be:

[0181] Loss_landmark=-min{(x2-x0)(y1-y0)-(x1-x0)(y2-y0),0}

[0182] -min{(x4-x3)(y2-y3)-(x2-x3)(y4-y3),0}

[0183] S607: Optimize the model parameters of the object detection model according to the second loss value to obtain an optimized object detection model.

[0184] In a specific implementation, the computer device optimizes the model parameters of the object detection model in a direction of reducing the second loss value to obtain an optimized object detection model.

[0185] In one embodiment, a computer device may obtain the annotation information of a sample image, and then use a reference loss function to calculate a reference loss value based on the annotation information of the sample image; and obtain a model loss value of the object detection model based on the second loss value and the reference loss value, and then optimize the model parameters of the object detection model in the direction of reducing the model loss value of the object detection model to obtain an optimized object detection model.

[0186] In an embodiment of the present application, a computer device may use a second loss function to calculate a first loss sub-value and a second loss sub-value based on the position detection information of the target key point of the sample object and the position detection information of the first key point and the second key point of the target key point; then, the first loss sub-value and the second loss sub-value are summed to obtain a second loss value, and the model parameters in the object detection model are optimized based on the second loss value to obtain an optimized object detection model. By calculating the second loss value based on the position detection information of the target key point and the position detection information of the first key point and the second key point, a relatively accurate second loss value can be obtained, so that the model can be optimized based on the second loss value, effectively improving the detection accuracy and robustness of the object detection model, and ensuring the accuracy of subsequent object detection using the optimized object detection model.

[0187] Based on the model optimization solution provided above, the embodiment of the present application provides a model optimization method, which can be executed by the computer device mentioned above, which can be the terminal device or server mentioned above. Figure 8 , the position loss function includes a first loss function and a second loss function, the target loss value includes a first loss value of the first loss function, and a second loss value of the second loss function to illustrate the model optimization method, which may include the following steps S801-S806:

[0188] S801: Acquire a sample image, where the sample image includes a sample object.

[0189] S802: Calling an object detection model to perform object detection on a sample image, and obtaining position detection information of target key points of the sample object and position detection information of reference objects of the target key points.

[0190] S803: Obtain a position loss function of the object detection model, where the position loss function includes a first loss function and a second loss function.

[0191] Among them, the first loss function is constructed based on the first position relationship that needs to be satisfied between the target key point and the object detection frame; the second loss function is constructed based on the second position relationship that needs to be satisfied between the target key point and the first key point.

[0192] S804. Using the position loss function, calculate the target loss value based on the position detection information of the target key point and the position detection information of the reference object. The target loss value includes the first loss value of the first loss function and the second loss value of the second loss function.

[0193] The specific implementation of step S804 may refer to the specific implementation of steps S403-S405 and the specific implementation of steps S603-S604, which will not be repeated here.

[0194] S805. Calculate a model loss value of the object detection model according to the first loss value and the second loss value.

[0195] In a specific implementation, the computer device sums the first loss value and the second loss value to obtain a model loss value of the object detection model.

[0196] In one embodiment, to balance the first loss value and the second loss value, the first loss value and the second loss value may be weighted. Specifically, the computer device obtains a first weight value of the first loss function and a second weight value of the second loss function, and weights the first loss value using the first weight value, and weights the second loss value using the second weight value. The weighted first loss value and the weighted second loss value are then summed to obtain a model loss value for the object detection model. The first weight value and the second weight value may be positive numbers.

[0197] In another embodiment, the computer device may obtain the annotation information of the sample image and use a reference loss function; then, based on the annotation information of the sample image, calculate a reference loss value; then, sum the weighted first loss value, the weighted second loss value, and the reference loss value to obtain a model loss value of the object detection model. The model loss value of the object detection model can be represented by L, and the calculation formula of the model loss value is:

[0198] L=loss+alpha_1Loss_bbox+alpha_2Loss_landmark

[0199] Among them, loss represents the reference loss value, alpha_1 represents the first weight value, alpha_2 represents the second weight value, loss_bbox represents the first loss value, and loss_landmark represents the second loss value.

[0200] S806. Optimize the model parameters of the object detection model in a direction of reducing the model loss value to obtain an optimized object detection model.

[0201] In one embodiment, a computer device sends the optimized object detection model to any node in the blockchain network, and any node verifies the optimized object detection model. When the verification is passed, any node generates a block using the optimized object detection model and sends the generated block to other nodes in the blockchain network for consensus. When the consensus is successful, the block is saved in the blockchain network, thereby preventing the optimized object detection model from being tampered with.

[0202] In an embodiment of the present application, the computer device determines the first loss value and the second loss value respectively through the first loss function and the second loss function, and determines the model loss value of the object detection model in combination with the first loss value and the second loss value, thereby ensuring the accuracy of the obtained model loss value, and optimizing the object detection model based on the relatively accurate model loss value, which can effectively improve the detection accuracy and robustness of the object detection model, thereby ensuring the accuracy of object detection by the optimized object detection model.

[0203] Based on the description of the above model optimization method embodiment, the present application embodiment also discloses a model optimization device, which can be a computer program (including program code) running on the above-mentioned computer device. The model optimization device can execute Figure 2 、 Figure 4 、 Figure 6 and Figure 8 See the method shown in Figure 9 , the model optimization device can run the following units:

[0204] An acquiring unit 901 is configured to acquire a sample image, where the sample image includes a sample object;

[0205] The processing unit 902 is configured to call an object detection model to perform object detection on the sample image to obtain position detection information of target key points of the sample object and position detection information of reference objects of the target key points;

[0206] The acquisition unit 901 is further configured to acquire a position loss function of the object detection model, where the position loss function is constructed based on a target position relationship that needs to be satisfied between the target key point and the reference object;

[0207] The processing unit 902 is further configured to calculate a target loss value based on the position detection information of the target key point and the position detection information of the reference object using the position loss function;

[0208] The processing unit 902 is further configured to optimize the model parameters of the object detection model according to the target loss value to obtain an optimized object detection model.

[0209] In another embodiment, the sample object includes one or more key points; the reference object includes an object detection frame of the sample object;

[0210] The position loss function includes a first loss function, and the target loss value includes: the first loss value of the first loss function; wherein, the target position relationship includes: the first position relationship that must be satisfied between any key point of the sample object and the object detection frame, and the first loss function is constructed based on the first position relationship, and the first position relationship is used to indicate that any key point must be located within the object detection frame.

[0211] In another embodiment, when the processing unit 902 is used to calculate the target loss value based on the position detection information of the target key point and the position detection information of the reference object using the position loss function, it can be specifically used to:

[0212] Using the first loss function, calculating a loss value corresponding to the target key point based on the position detection information of the target key point and the position detection information of the reference object;

[0213] Determine a first loss value according to the loss value corresponding to the target key point.

[0214] In another embodiment, the position detection information of the target key point includes: the position coordinates of the target key point in the sample image, and the position coordinates include a horizontal coordinate value and a vertical coordinate value;

[0215] The object detection frame is a rectangular bounding box, and the position detection information of the reference object includes: position indication values ​​of each boundary of the object detection frame in the sample image; wherein the first positional relationship is represented by the orientation relationship that must be satisfied between any key point and each boundary;

[0216] The processing unit 902, when used to calculate the loss value corresponding to the target key point based on the position detection information of the target key point and the position detection information of the reference object using the first loss function, may be specifically configured to:

[0217] Traversing each boundary of the object detection frame;

[0218] Acquire a position indication value of a current boundary currently traversed from the position detection information of the reference object, and acquire a coordinate value related to the current boundary from the position coordinates of the target key point;

[0219] Using the first loss function, according to the position indication value of the current boundary and the acquired coordinate values, calculate a loss value corresponding to the current boundary;

[0220] After all the boundaries are traversed, the loss values ​​corresponding to the boundaries are summed to obtain the loss value corresponding to the target key point.

[0221] In another embodiment, the position detection information of the target key point includes: the position coordinates of the target key point in the sample image;

[0222] The object detection frame is an irregular detection frame, and the position detection information of the reference object includes: the position coordinates of each vertex of the object detection frame in the sample image; the object detection frame is represented by a plurality of vertex vectors, and each vertex vector is determined based on the position coordinates of two adjacent vertices of the object detection frame; wherein the first positional relationship is represented by an orientation relationship that must be satisfied between any key point and each vertex vector;

[0223] The processing unit 902, when used to calculate the loss value corresponding to the target key point based on the position detection information of the target key point and the position detection information of the reference object using the first loss function, may be specifically configured to:

[0224] Traversing multiple vertices of the object detection frame and determining a target vertex vector corresponding to a current vertex currently traversed, wherein the target vertex vector refers to a vertex vector constructed using the current vertex as a starting point;

[0225] Acquire the position coordinates of the current vertex from the position information of the reference object, and calculate a reference vertex vector between the current vertex and the target key point according to the position coordinates of the current vertex and the position coordinates of the target key point;

[0226] Using the first loss function, according to the reference vertex vector and the target vertex vector, calculate the loss value corresponding to the current vertex;

[0227] After all vertices of the object detection frame are traversed, the loss values ​​corresponding to the vertices are summed to obtain the loss value corresponding to the target key point.

[0228] In another embodiment, the sample object includes multiple key points, and the reference object further includes one or more key points among the multiple key points except the target key point; when the processing unit 902 is used to determine the first loss value based on the loss value corresponding to the target key point, it can be specifically used to:

[0229] Using the first loss function, calculating a loss value corresponding to each key point in the reference object based on position detection information of each key point in the reference object and position detection information of the object detection frame;

[0230] The loss value corresponding to the target key point and the loss value corresponding to each key point in the reference object are summed to obtain a first loss value.

[0231] In another embodiment, the sample object includes a plurality of key points; the reference object includes: a first key point and a second key point selected from the plurality of key points;

[0232] The position loss function includes a second loss function, and the target loss value includes: a second loss value of the second loss function; wherein the target position relationship includes at least one of the following: a second position relationship that needs to be satisfied between the target key point and the first key point, and a third position relationship that needs to be satisfied between the target key point and the second key point;

[0233] The second positional relationship is used to indicate that the target key point needs to be located at a first position of the first key point, and the third positional relationship is used to indicate that the target key point needs to be located at a second position of the second key point;

[0234] The second loss function is constructed based on at least one positional relationship among the second positional relationship and the third positional relationship, and the second loss value is any one of the following: a first loss sub-value calculated based on the target key point and the first key point, or a second loss sub-value calculated based on the target key point and the second key point, or the sum of the first loss sub-value and the second loss sub-value.

[0235] In another embodiment, the position detection information of the target key point includes: the position coordinates of the target key point in the sample image; the reference object includes the first key point and the second key point, and the position detection information of the reference object includes: the position coordinates of the first key point in the sample image, and the position coordinates of the second key point in the sample image;

[0236] The processing unit 902, when used to calculate the target loss value based on the position detection information of the target key point and the position detection information of the reference object using the position loss function, can be specifically used to:

[0237] Using the second loss function, calculating a first loss sub-value according to the position coordinates of the target key point and the position coordinates of the first key point;

[0238] Using the second loss function, calculating a second loss sub-value according to the position coordinates of the target key point and the position coordinates of the second key point;

[0239] A sum operation is performed on the first loss sub-value and the second loss sub-value to obtain a second loss value.

[0240] In another embodiment, the number of the first key points is two, and the processing unit 902, when used to calculate the first loss sub-value according to the position coordinates of the target key point and the position coordinates of the first key point using the second loss function, can be specifically used to:

[0241] Calculating a first reference vector based on the position coordinates of the two first key points, and calculating a first reference vector based on the position coordinates of the starting point of the first reference vector and the position coordinates of the target key point; wherein the first orientation refers to the orientation that needs to be satisfied between the target key point and the first reference vector;

[0242] The second loss function is used to calculate a first loss sub-value based on the first outer product result of the first baseline vector and the first reference vector; wherein the first outer product result is used to indicate whether the target key point is located at the first orientation of the first baseline vector; and when the target key point is not located at the first orientation, the first loss sub-value is generated.

[0243] In another embodiment, the processing unit 902, when used to calculate the first loss sub-value based on the first outer product result of the first benchmark vector and the first reference vector using the second loss function, can be specifically used to:

[0244] Using the second loss function, performing an outer product operation on the first benchmark vector and the first reference vector to obtain a first outer product result;

[0245] If the first outer product result is greater than an invalid value, determining that the target key point is located at the first orientation of the first reference vector, and determining that the first loss sub-value is equal to the invalid value;

[0246] If the first outer product result is less than or equal to the invalid value, it is determined that the target key point is not located at the first orientation of the first reference vector, and the first loss sub-value is determined according to the first outer product result.

[0247] In another embodiment, the sample object is a facial object; the target key point is the nose key point, the two first key points include the left eye key point and the right eye key point, the first orientation is the right side of the first reference vector formed by the left eye key point and the right eye key point; the two second key points include the left corner of the mouth key point and the right corner of the mouth key point; the second orientation is the left side of the second reference vector formed by the left corner of the mouth key point and the right corner of the mouth key point.

[0248] In another embodiment, the position loss function includes a first loss function and a second loss function, and the target loss value includes: a first loss value of the first loss function and a second loss value of the second loss function; the processing unit 902, when used to optimize the model parameters of the object detection model according to the target loss value to obtain the optimized object detection model, can be specifically used to:

[0249] Calculating a model loss value of the object detection model according to the first loss value and the second loss value;

[0250] The model parameters of the object detection model are optimized in a direction of reducing the model loss value to obtain an optimized object detection model.

[0251] According to one embodiment of the present application, Figure 2 、 Figure 4 、 Figure 6 and Figure 8 Each step involved in the method shown can be performed by Figure 9 The model optimization device shown is executed by each unit. For example, Figure 2 Step S201 shown is performed by Figure 9 The acquisition unit 901 shown is executed, and step S202 is performed by Figure 9 The processing unit 902 shown in FIG. 1 is executed, and step S203 is performed by Figure 9 The acquisition unit 901 shown is executed, and steps S204 to S205 are performed by Figure 9 The processing unit 902 shown is executed. For example, Figure 4 Step S401 shown is performed by Figure 9 The acquisition unit 901 shown is executed, and step S402 is performed by Figure 9 The processing unit 902 shown is executed, and step 403 is performed by Figure 9 The acquisition unit 901 shown is executed, and steps S404 to S406 are performed by Figure 9 For example, step S601 is performed by the processing unit 902 shown in Figure 9 The acquisition unit 901 shown is executed, and step S602 is performed by Figure 9 The processing unit 902 shown is executed, and step 603 is performed by Figure 9 The acquisition unit 901 shown in FIG is executed, and steps S604 to S607 are performed by Figure 9 The processing unit 902 shown in FIG. 1 is executed. For example, step S801 is performed by Figure 9 The acquisition unit 901 shown is executed, and step S802 is performed by Figure 9 The processing unit 902 shown is executed, and step 803 is performed by Figure 9 The acquisition unit 901 shown is executed, and steps S804 to S806 are performed by Figure 9 The processing unit 902 is shown to execute.

[0252] According to another embodiment of the present application, Figure 9 The various units in the model optimization device shown can be separately or all merged into one or several other units to constitute, or a certain (some) unit can also be split into multiple smaller units in function to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the function of a unit can also be realized by multiple units, or the function of multiple units can be realized by one unit. In other embodiments of the present application, other units can also be included based on the model optimization device. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0253] According to another embodiment of the present application, the system can be implemented by including a central processing unit (CPU), a random access memory (RAM), a read-only memory (ROM) and other processing elements and storage elements. For example, a general computing device such as a computer can execute the following operations: Figure 2 、 Figure 4 、 Figure 6 or Figure 8 A computer program (including program code) for each step of the corresponding method shown in FIG. Figure 9 The model optimization device shown in the figure and the model optimization method of the embodiment of the present application are implemented. The computer program can be recorded on a computer-readable recording medium, for example, and loaded into the above-mentioned computer device through the computer-readable recording medium and run therein.

[0254] In an embodiment of the present application, after acquiring a sample image and invoking an object detection model to perform object detection on the sample image to obtain position detection information of target key points and position detection information of a reference object, a computer device may use a position loss function to calculate a target loss value based on the position detection information of the target key points and the position detection information of the reference object; and optimize the model parameters in the object detection model based on the target loss value to obtain an optimized object detection model. By calculating the target loss value based on the position detection information of the target key points and the position detection information of the reference object, a relatively accurate target loss value can be obtained, and the model can be optimized based on the target loss value, effectively improving the detection accuracy and robustness of the object detection model.

[0255] Based on the description of the above model optimization method embodiment, the present application embodiment also discloses a computer device, see Figure 10 The computer device may include at least a processor 1001, an input device 1002, an output device 1003, and a computer storage medium 1004. The processor 1001, input device 1002, output device 1003, and computer storage medium 1004 in the computer device may be connected via a bus or other means.

[0256] The computer storage medium 1004 is a memory device in a computer device, which is used to store programs and data. It is understandable that the computer storage medium 1004 here can include both the built-in storage medium of the computer device and, of course, the extended storage medium supported by the computer device. The computer storage medium 1004 provides a storage space, which stores the operating system of the computer device. In addition, one or more instructions suitable for being loaded and executed by the processor 1001 are also stored in the storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer storage medium here can be a high-speed RAM memory; optionally, it can also be at least one computer storage medium away from the aforementioned processor. The processor can be called a central processing unit (CPU), which is the core and control center of the computer device, suitable for implementing one or more instructions, specifically loading and executing one or more instructions to realize the corresponding method flow or function.

[0257] In one embodiment, the processor 1001 may load and execute one or more instructions stored in a computer storage medium to implement the corresponding steps of the method in the above-mentioned embodiment of the model optimization method. In a specific implementation, the processor 1001 loads one or more instructions in the computer storage medium and performs the following operations:

[0258] Acquire a sample image, where the sample image includes a sample object;

[0259] Calling an object detection model to perform object detection on the sample image to obtain position detection information of target key points of the sample object and position detection information of reference objects of the target key points;

[0260] Obtaining a position loss function of the object detection model, where the position loss function is constructed based on a target position relationship that needs to be satisfied between the target key point and the reference object;

[0261] Calculating a target loss value based on the position detection information of the target key point and the position detection information of the reference object using the position loss function;

[0262] Optimize the model parameters of the object detection model according to the target loss value to obtain an optimized object detection model.

[0263] In another embodiment, the sample object includes one or more key points; the reference object includes an object detection frame of the sample object;

[0264] The position loss function includes a first loss function, and the target loss value includes: the first loss value of the first loss function; wherein, the target position relationship includes: the first position relationship that must be satisfied between any key point of the sample object and the object detection frame, and the first loss function is constructed based on the first position relationship, and the first position relationship is used to indicate that any key point must be located within the object detection frame.

[0265] In another embodiment, when the position loss function is used to calculate the target loss value based on the position detection information of the target key point and the position detection information of the reference object, the one or more instructions are loaded and specifically executed by the processor 1001:

[0266] Using the first loss function, calculating a loss value corresponding to the target key point based on the position detection information of the target key point and the position detection information of the reference object;

[0267] Determine a first loss value according to the loss value corresponding to the target key point.

[0268] In another embodiment, the position detection information of the target key point includes: the position coordinates of the target key point in the sample image, and the position coordinates include a horizontal coordinate value and a vertical coordinate value;

[0269] The object detection frame is a rectangular bounding box, and the position detection information of the reference object includes: position indication values ​​of each boundary of the object detection frame in the sample image; wherein the first positional relationship is represented by the orientation relationship that must be satisfied between any key point and each boundary;

[0270] When the first loss function is used to calculate the loss value corresponding to the target key point based on the position detection information of the target key point and the position detection information of the reference object, the one or more instructions are loaded and specifically executed by the processor 1001:

[0271] Traversing each boundary of the object detection frame;

[0272] Acquire a position indication value of a current boundary currently traversed from the position detection information of the reference object, and acquire a coordinate value related to the current boundary from the position coordinates of the target key point;

[0273] Using the first loss function, according to the position indication value of the current boundary and the acquired coordinate values, calculate a loss value corresponding to the current boundary;

[0274] After all the boundaries are traversed, the loss values ​​corresponding to the boundaries are summed to obtain the loss value corresponding to the target key point.

[0275] In another embodiment, the position detection information of the target key point includes: the position coordinates of the target key point in the sample image; the object detection frame is an irregular detection frame, and the position detection information of the reference object includes: the position coordinates of each vertex of the object detection frame in the sample image; the object detection frame is represented by multiple vertex vectors, and any vertex vector is determined based on the position coordinates of two adjacent vertices of the object detection frame; wherein the first position relationship is represented by the orientation relationship that must be satisfied between any key point and each vertex vector; accordingly, when using the first loss function to calculate the loss value corresponding to the target key point based on the position detection information of the target key point and the position detection information of the reference object, the one or more instructions are loaded and specifically executed by the processor 1001:

[0276] Traversing multiple vertices of the object detection frame and determining a target vertex vector corresponding to a current vertex currently traversed, wherein the target vertex vector refers to a vertex vector constructed using the current vertex as a starting point;

[0277] Acquire the position coordinates of the current vertex from the position information of the reference object, and calculate a reference vertex vector between the current vertex and the target key point according to the position coordinates of the current vertex and the position coordinates of the target key point;

[0278] Using the first loss function, according to the reference vertex vector and the target vertex vector, calculate the loss value corresponding to the current vertex;

[0279] After all vertices of the object detection frame are traversed, the loss values ​​corresponding to the vertices are summed to obtain the loss value corresponding to the target key point.

[0280] In another embodiment, the sample object includes multiple key points, and the reference object further includes one or more key points among the multiple key points except the target key point; accordingly, when determining the first loss value based on the loss value corresponding to the target key point, the one or more instructions are loaded and specifically executed by the processor 1001:

[0281] Using the first loss function, calculating a loss value corresponding to each key point in the reference object based on position detection information of each key point in the reference object and position detection information of the object detection frame;

[0282] The loss value corresponding to the target key point and the loss value corresponding to each key point in the reference object are summed to obtain a first loss value.

[0283] In another embodiment, the sample object includes a plurality of key points; the reference object includes: a first key point and a second key point selected from the plurality of key points; the position loss function includes a second loss function, and the target loss value includes: a second loss value of the second loss function; wherein the target position relationship includes at least one of the following: a second position relationship that must be satisfied between the target key point and the first key point, and a third position relationship that must be satisfied between the target key point and the second key point;

[0284] The second positional relationship is used to indicate that the target key point needs to be located at a first position of the first key point, and the third positional relationship is used to indicate that the target key point needs to be located at a second position of the second key point;

[0285] The second loss function is constructed based on at least one positional relationship among the second positional relationship and the third positional relationship, and the second loss value is any one of the following: a first loss sub-value calculated based on the target key point and the first key point, or a second loss sub-value calculated based on the target key point and the second key point, or the sum of the first loss sub-value and the second loss sub-value.

[0286] In another embodiment, the position detection information of the target key point includes: the position coordinates of the target key point in the sample image; the reference object includes the first key point and the second key point, and the position detection information of the reference object includes: the position coordinates of the first key point in the sample image, and the position coordinates of the second key point in the sample image; accordingly, when using the position loss function to calculate the target loss value based on the position detection information of the target key point and the position detection information of the reference object, the one or more instructions are loaded and specifically executed by the processor 1001:

[0287] Using the second loss function, calculating a first loss sub-value according to the position coordinates of the target key point and the position coordinates of the first key point;

[0288] Using the second loss function, calculating a second loss sub-value according to the position coordinates of the target key point and the position coordinates of the second key point;

[0289] A sum operation is performed on the first loss sub-value and the second loss sub-value to obtain a second loss value.

[0290] In another embodiment, the number of the first key points is two. Accordingly, when the second loss function is used to calculate the first loss sub-value according to the position coordinates of the target key point and the position coordinates of the first key point, the one or more instructions are loaded and specifically executed by the processor 1001:

[0291] Calculating a first reference vector based on the position coordinates of the two first key points, and calculating a first reference vector based on the position coordinates of the starting point of the first reference vector and the position coordinates of the target key point; wherein the first orientation refers to the orientation that needs to be satisfied between the target key point and the first reference vector;

[0292] The second loss function is used to calculate a first loss sub-value based on the first outer product result of the first baseline vector and the first reference vector; wherein the first outer product result is used to indicate whether the target key point is located at the first orientation of the first baseline vector; and when the target key point is not located at the first orientation, the first loss sub-value is generated.

[0293] In another embodiment, when the second loss function is used to calculate the first loss sub-value based on the first outer product result of the first baseline vector and the first reference vector, the one or more instructions are loaded and specifically executed by the processor 1001:

[0294] Using the second loss function, performing an outer product operation on the first benchmark vector and the first reference vector to obtain a first outer product result;

[0295] If the first outer product result is greater than an invalid value, determining that the target key point is located at the first orientation of the first reference vector, and determining that the first loss sub-value is equal to the invalid value;

[0296] If the first outer product result is less than or equal to the invalid value, it is determined that the target key point is not located at the first orientation of the first reference vector, and the first loss sub-value is determined according to the first outer product result.

[0297] In another embodiment, the sample object is a facial object; the target key point is the nose key point, the two first key points include the left eye key point and the right eye key point, the first orientation is the right side of the first reference vector formed by the left eye key point and the right eye key point; the two second key points include the left corner of the mouth key point and the right corner of the mouth key point; the second orientation is the left side of the second reference vector formed by the left corner of the mouth key point and the right corner of the mouth key point.

[0298] In another embodiment, the position loss function includes a first loss function and a second loss function, and the target loss value includes: a first loss value of the first loss function and a second loss value of the second loss function; accordingly, when the model parameters of the object detection model are optimized according to the target loss value and the optimized object detection model is obtained, the one or more instructions are loaded and specifically executed by the processor 1001:

[0299] Calculating a model loss value of the object detection model according to the first loss value and the second loss value;

[0300] The model parameters of the object detection model are optimized in a direction of reducing the model loss value to obtain an optimized object detection model.

[0301] In an embodiment of the present application, after acquiring a sample image and invoking an object detection model to perform object detection on the sample image to obtain position detection information of target key points and position detection information of a reference object, a computer device may use a position loss function to calculate a target loss value based on the position detection information of the target key points and the position detection information of the reference object; and optimize the model parameters in the object detection model based on the target loss value to obtain an optimized object detection model. By calculating the target loss value based on the position detection information of the target key points and the position detection information of the reference object, a relatively accurate target loss value can be obtained, and the model can be optimized based on the target loss value, effectively improving the detection accuracy and robustness of the object detection model.

[0302] It should be noted that the embodiment of the present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned model optimization method embodiment. Figure 2 、 Figure 4 、 Figure 6 or Figure 8 The steps performed in .

[0303] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0304] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that implementing all or part of the processes of the above embodiment and making equivalent changes in accordance with the claims of this application still fall within the scope of the invention.

Claims

1. A model optimization method, characterized in that: include: Acquire a sample image, where the sample image includes a sample object; Calling an object detection model to perform object detection on the sample image to obtain position detection information of target key points of the sample object and position detection information of reference objects of the target key points; Obtaining a position loss function of the object detection model, where the position loss function is constructed based on a target position relationship that needs to be satisfied between the target key point and the reference object; Calculating a target loss value based on the position detection information of the target key point and the position detection information of the reference object using the position loss function; Optimizing the model parameters of the object detection model according to the target loss value to obtain an optimized object detection model; The sample object includes a plurality of key points; the reference object includes: a first key point and a second key point selected from the plurality of key points; The position loss function includes a second loss function, and the target loss value includes: a second loss value of the second loss function; wherein the target position relationship includes at least one of the following: a second position relationship that must be satisfied between the target key point and the first key point, and a third position relationship that must be satisfied between the target key point and the second key point; the second position relationship is used to indicate that the target key point must be located at a first orientation of the first key point, and the third position relationship is used to indicate that the target key point must be located at a second orientation of the second key point; The second loss function is constructed based on at least one positional relationship among the second positional relationship and the third positional relationship, and the second loss value is any one of the following: a first loss sub-value calculated based on the target key point and the first key point, or a second loss sub-value calculated based on the target key point and the second key point, or the sum of the first loss sub-value and the second loss sub-value.

2. The method according to claim 1, wherein The sample object includes one or more key points; the reference object includes an object detection frame of the sample object; The position loss function includes a first loss function, and the target loss value includes: the first loss value of the first loss function; wherein, the target position relationship includes: the first position relationship that must be satisfied between any key point of the sample object and the object detection frame, and the first loss function is constructed based on the first position relationship, and the first position relationship is used to indicate that any key point must be located within the object detection frame.

3. The method according to claim 2, wherein The method of using the position loss function to calculate a target loss value according to the position detection information of the target key point and the position detection information of the reference object includes: Using the first loss function, calculating a loss value corresponding to the target key point based on the position detection information of the target key point and the position detection information of the reference object; Determine a first loss value according to the loss value corresponding to the target key point.

4. The method according to claim 3, wherein The position detection information of the target key point includes: the position coordinates of the target key point in the sample image, and the position coordinates include a horizontal coordinate value and a vertical coordinate value; The object detection frame is a rectangular bounding box, and the position detection information of the reference object includes: position indication values ​​of each boundary of the object detection frame in the sample image; wherein the first positional relationship is represented by the orientation relationship that must be satisfied between any key point and each boundary; The using the first loss function to calculate the loss value corresponding to the target key point according to the position detection information of the target key point and the position detection information of the reference object includes: Traversing each boundary of the object detection frame; Acquire a position indication value of a current boundary currently traversed from the position detection information of the reference object, and acquire a coordinate value related to the current boundary from the position coordinates of the target key point; Using the first loss function, according to the position indication value of the current boundary and the acquired coordinate values, calculate a loss value corresponding to the current boundary; After all the boundaries are traversed, the loss values ​​corresponding to the boundaries are summed to obtain the loss value corresponding to the target key point.

5. The method according to claim 3, wherein The position detection information of the target key point includes: the position coordinates of the target key point in the sample image; The object detection frame is an irregular detection frame, and the position detection information of the reference object includes: the position coordinates of each vertex of the object detection frame in the sample image; the object detection frame is represented by a plurality of vertex vectors, and each vertex vector is determined based on the position coordinates of two adjacent vertices of the object detection frame; wherein the first positional relationship is represented by an orientation relationship that must be satisfied between any key point and each vertex vector; The using the first loss function to calculate the loss value corresponding to the target key point according to the position detection information of the target key point and the position detection information of the reference object includes: Traversing multiple vertices of the object detection frame and determining a target vertex vector corresponding to a current vertex currently traversed, wherein the target vertex vector refers to a vertex vector constructed using the current vertex as a starting point; Acquire the position coordinates of the current vertex from the position information of the reference object, and calculate a reference vertex vector between the current vertex and the target key point according to the position coordinates of the current vertex and the position coordinates of the target key point; Using the first loss function, according to the reference vertex vector and the target vertex vector, calculate the loss value corresponding to the current vertex; After all vertices of the object detection frame are traversed, the loss values ​​corresponding to the vertices are summed to obtain the loss value corresponding to the target key point.

6. The method according to any one of claims 3 to 5, characterized in that The sample object includes a plurality of key points, and the reference object further includes one or more key points among the plurality of key points except the target key point; The determining of the first loss value according to the loss value corresponding to the target key point includes: Using the first loss function, calculating a loss value corresponding to each key point in the reference object based on position detection information of each key point in the reference object and position detection information of the object detection frame; The loss value corresponding to the target key point and the loss value corresponding to each key point in the reference object are summed to obtain a first loss value.

7. The method according to claim 1, wherein The position detection information of the target key point includes: the position coordinates of the target key point in the sample image; the reference object includes the first key point and the second key point, and the position detection information of the reference object includes: the position coordinates of the first key point in the sample image, and the position coordinates of the second key point in the sample image; The method of using the position loss function to calculate a target loss value according to the position detection information of the target key point and the position detection information of the reference object includes: Using the second loss function, calculating a first loss sub-value according to the position coordinates of the target key point and the position coordinates of the first key point; Using the second loss function, calculating a second loss sub-value according to the position coordinates of the target key point and the position coordinates of the second key point; A sum operation is performed on the first loss sub-value and the second loss sub-value to obtain a second loss value.

8. The method according to claim 7, wherein The number of the first key points is two, The using the second loss function to calculate the first loss sub-value according to the position coordinates of the target key point and the position coordinates of the first key point includes: Calculating a first reference vector based on the position coordinates of the two first key points, and calculating a first reference vector based on the position coordinates of the starting point of the first reference vector and the position coordinates of the target key point; wherein the first orientation refers to the orientation that needs to be satisfied between the target key point and the first reference vector; The second loss function is used to calculate a first loss sub-value based on the first outer product result of the first baseline vector and the first reference vector; wherein the first outer product result is used to indicate whether the target key point is located at the first orientation of the first baseline vector; and when the target key point is not located at the first orientation, the first loss sub-value is generated.

9. The method according to claim 8, wherein The step of using the second loss function to calculate a first loss sub-value according to a first outer product result of the first benchmark vector and the first reference vector includes: Using the second loss function, performing an outer product operation on the first benchmark vector and the first reference vector to obtain a first outer product result; If the first outer product result is greater than an invalid value, determining that the target key point is located at the first orientation of the first reference vector, and determining that the first loss sub-value is equal to the invalid value; If the first outer product result is less than or equal to the invalid value, it is determined that the target key point is not located at the first orientation of the first reference vector, and the first loss sub-value is determined according to the first outer product result.

10. The method according to any one of claims 7 to 9, characterized in that The sample object is a facial object; the target key point is a nose key point, the two first key points include a left eye key point and a right eye key point, and the first orientation is to the right of a first reference vector formed by the left eye key point and the right eye key point; The two second key points include the left corner of the mouth key point and the right corner of the mouth key point; The second orientation is to the left of a second reference vector formed by the left mouth corner key point and the right mouth corner key point.

11. The method according to claim 1, wherein The position loss function includes a first loss function and a second loss function, the target loss value includes: a first loss value of the first loss function, and a second loss value of the second loss function; optimizing the model parameters of the object detection model according to the target loss value to obtain an optimized object detection model includes: Calculating a model loss value of the object detection model according to the first loss value and the second loss value; The model parameters of the object detection model are optimized in the direction of reducing the model loss value to obtain an optimized object detection model.

12. A model optimization device, characterized in that: include: an acquiring unit, configured to acquire a sample image, wherein the sample image includes a sample object; a processing unit, configured to call an object detection model to perform object detection on the sample image, and obtain position detection information of target key points of the sample object and position detection information of reference objects of the target key points; The acquisition unit is further configured to acquire a position loss function of the object detection model, where the position loss function is constructed based on a target position relationship that needs to be satisfied between the target key point and the reference object; The processing unit is further configured to calculate a target loss value based on the position detection information of the target key point and the position detection information of the reference object using the position loss function; The processing unit is further configured to optimize the model parameters of the object detection model according to the target loss value to obtain an optimized object detection model; The sample object includes a plurality of key points; the reference object includes: a first key point and a second key point selected from the plurality of key points; The position loss function includes a second loss function, and the target loss value includes: a second loss value of the second loss function; wherein the target position relationship includes at least one of the following: a second position relationship that must be satisfied between the target key point and the first key point, and a third position relationship that must be satisfied between the target key point and the second key point; the second position relationship is used to indicate that the target key point must be located at a first orientation of the first key point, and the third position relationship is used to indicate that the target key point must be located at a second orientation of the second key point; The second loss function is constructed based on at least one positional relationship among the second positional relationship and the third positional relationship, and the second loss value is any one of the following: a first loss sub-value calculated based on the target key point and the first key point, or a second loss sub-value calculated based on the target key point and the second key point, or the sum of the first loss sub-value and the second loss sub-value.

13. A computer device, characterized in that: include: a processor adapted to implement one or more computer programs; as well as A computer storage medium storing one or more computer programs, wherein the one or more computer programs are suitable for being loaded by the processor and executing the model optimization method according to any one of claims 1 to 11.

14. A computer storage medium, characterized in that The computer storage medium stores a computer program, and when the computer program is executed by a processor, the model optimization method according to any one of claims 1 to 11 is executed.

Images