Positive and negative sample identification method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a positive and negative sample recognition method and device, electronic equipment and a storage medium, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a sample image containing at least two sample targets; acquiring a sample coordinate of each sample target in the sample image; performing anchoring loss calculation according to the sample coordinates and a preset anchoring frame to obtain an anchoring loss matrix; performing anchoring frame number calculation on the sample target according to the anchoring loss matrix to obtain the anchoring frame number of each sample target; and according to the number of the anchoring frames, the anchoring loss matrix and a preset anchoring frame, carrying out anchoring frame distribution on the sample target to obtain a positive anchoring frame and a negative anchoring frame of each sample target. According to the embodiment of the invention, the recognition accuracy of positive and negative samples in the picture can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a positive and negative sample identification method and device, an electronic device and a storage medium. Background Art

[0002] Positive samples refer to instances that belong to the target category or meet specific conditions in the data set, and negative samples refer to instances that do not belong to the target category or meet specific conditions in the data set. For example, when selecting a target object in an image, a positive sample is an anchor box that includes the target object to be detected, and a negative sample is an anchor box that does not include the target object to be detected. However, when the image includes multiple objects and the objects overlap, the anchor boxes will affect each other, making it difficult to define the boundaries of the anchor boxes so that the anchor boxes can anchor the target object as much as possible globally without affecting the anchoring of other anchor boxes to the target object. Therefore, how to improve the recognition accuracy of positive and negative samples in images has become an urgent problem to be solved. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a positive and negative sample identification method and device, an electronic device and a storage medium, aiming to improve the recognition accuracy of positive and negative samples in an image.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a positive and negative sample identification method, the method comprising:

[0005] Acquire a sample image containing at least two sample objects;

[0006] Obtaining the sample coordinates of each of the sample objects in the sample image;

[0007] Anchor loss calculation is performed according to the sample coordinates and the preset anchor frame to obtain an anchor loss matrix;

[0008] Calculating the number of anchor boxes for the sample targets according to the anchor loss matrix to obtain the number of anchor boxes for each sample target;

[0009] Anchor boxes are assigned to the sample targets according to the number of anchor boxes, the anchor loss matrix, and the preset anchor boxes to obtain positive anchor boxes and negative anchor boxes for each of the sample targets.

[0010] In some embodiments, the assigning anchor boxes to the sample targets according to the number of anchor boxes, the anchor loss matrix, and the preset anchor boxes to obtain a positive anchor box and a negative anchor box for each of the sample targets includes:

[0011] Performing aggregation calculation according to the anchor loss matrix and the preset regularization parameter to obtain a kernel matrix;

[0012] Performing aggregation calculation according to the preset column matrix, the core matrix and the number of anchor boxes to obtain a target row matrix;

[0013] Performing aggregation calculation according to the target row matrix and the kernel matrix to obtain a target column matrix;

[0014] Performing aggregation calculation according to the target column matrix and the target row matrix to obtain a transmission matrix;

[0015] Anchor frames are allocated to the sample targets according to the number of anchor frames, the transfer matrix, and the preset anchor frames to obtain the positive anchor frame and the negative anchor frame of each sample target.

[0016] In some embodiments, the transfer matrix includes adaptation data of each of the sample targets to each of the preset anchor frames, and the assigning of anchor frames to the sample targets according to the number of anchor frames, the transfer matrix, and the preset anchor frames to obtain the positive anchor frame and the negative anchor frame of each of the sample targets includes:

[0017] According to all the preset anchor boxes and the fitness data corresponding to each preset anchor box, the corresponding sample targets are sorted in descending order to obtain a target descending sequence;

[0018] Allocating anchor boxes to the corresponding sample targets according to the target descending sequence and the number of anchor boxes corresponding to each sample target, to obtain the positive anchor box of each sample target;

[0019] The positive anchor box of each of the sample targets is filtered out according to the preset anchor box to obtain the negative anchor box of each of the sample targets.

[0020] In some embodiments, calculating the number of anchor boxes for the sample targets according to the anchor loss matrix to obtain the number of anchor boxes for each of the sample targets includes:

[0021] The anchor loss matrix is ​​screened according to a preset anchor box screening threshold to obtain a screening loss matrix;

[0022] Summing the screening loss matrix to obtain a total screening loss value;

[0023] The total value of the screening loss is rounded down to obtain the number of anchor boxes for each sample target.

[0024] In some embodiments, the anchor loss calculation is performed according to the sample coordinates and the preset anchor frame to obtain an anchor loss matrix, including:

[0025] Perform overlapping coordinate screening according to the sample coordinates and the preset anchor frame to obtain overlapping coordinates;

[0026] Performing area calculation according to the overlapping coordinates to obtain the overlapping coordinate area;

[0027] Performing area calculation according to the anchor frame to obtain the anchor frame area;

[0028] The ratio between the overlapping coordinate area and the anchor box area is calculated to obtain the anchor loss matrix.

[0029] In some embodiments, after assigning anchor boxes to the sample targets according to the number of anchor boxes, the anchor loss matrix, and the preset anchor boxes to obtain a positive anchor box and a negative anchor box for each of the sample targets, the method further includes:

[0030] Obtaining a sample category of the sample target, and training a preset multi-target detection model according to the sample image, the sample category of the sample target, the positive anchor box, and the negative anchor box to obtain a target multi-target detection model;

[0031] A selected image is acquired, and multi-target detection is performed on the selected image according to the target multi-target detection model.

[0032] In some embodiments, the training of a preset multi-target detection model according to the sample image, the sample category of the sample target, the positive anchor box, and the negative anchor box to obtain a target multi-target detection model includes:

[0033] Inputting the sample image into the preset multi-target detection model to perform multi-target detection, and obtaining the predicted category and predicted anchor box of the sample target;

[0034] Calculate the loss value according to the predicted anchor box, the positive anchor box, the negative anchor box, the predicted category and the sample category to obtain a training loss value;

[0035] Parameters of the preset multi-target detection model are optimized according to the training loss value to obtain the target multi-target detection model.

[0036] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes a positive and negative sample identification device, the device comprising:

[0037] An image acquisition module, used to acquire a sample image containing at least two sample targets;

[0038] A coordinate acquisition module, used for acquiring the sample coordinates of each sample target in the sample image;

[0039] A loss calculation module, used to calculate the anchor loss according to the sample coordinates and the preset anchor frame to obtain an anchor loss matrix;

[0040] A quantity calculation module, used to calculate the number of anchor boxes of the sample targets according to the anchor loss matrix, to obtain the number of anchor boxes of each sample target;

[0041] The anchor box allocation module is used to allocate anchor boxes to the sample targets according to the number of anchor boxes, the anchor loss matrix and the preset anchor boxes to obtain a positive anchor box and a negative anchor box for each of the sample targets.

[0042] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0043] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0044] The present application proposes a positive and negative sample recognition method and device, an electronic device and a storage medium, which obtains a sample image containing at least two sample targets, then obtains the sample coordinates of each sample target in the sample image, and then calculates the anchor loss according to the sample coordinates and the preset anchor frame to obtain an anchor loss matrix, thereby determining the loss value between the pixel point of the sample target and the anchor frame, and providing a data basis for the subsequent global anchor frame allocation based on the loss value; further, the number of anchor frames of the sample target is calculated according to the anchor loss matrix to obtain the anchor loss matrix of each sample target. The number of anchor boxes is determined to determine the number of anchor boxes for each sample target, so that the anchor box of each sample target can anchor the sample object as much as possible while minimizing the anchoring influence on other sample targets; further, the sample targets are assigned anchor boxes according to the number of anchor boxes, the anchor loss matrix and the preset anchor boxes, and the positive anchor box and the negative anchor box of each sample target are obtained, so as to improve the recognition accuracy of positive and negative samples in the picture, so that the anchor box can anchor the sample object without affecting other anchor boxes anchoring other sample objects, and achieve the best anchoring effect globally. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of the positive and negative sample identification method provided in an embodiment of the present application;

[0046] Figure 2 yes Figure 1 Flow chart of step S103 in FIG.

[0047] Figure 3 yes Figure 1 Flow chart of step S104 in FIG.

[0048] Figure 4 yes Figure 1 Flow chart of step S105 in FIG.

[0049] Figure 5 yes Figure 4 Flow chart of step S405 in FIG.

[0050] Figure 6 is a flow chart of a positive and negative sample identification method provided by another embodiment of the present application;

[0051] Figure 7 yes Figure 6 Flowchart of step S602 in FIG.

[0052] Figure 8 is a structural diagram of a positive and negative sample identification device provided in an embodiment of the present application;

[0053] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0057] First, some nouns involved in this application are analyzed:

[0058] Anchor Box: Anchor box is a technique used in the field of computer vision and machine learning, especially playing a key role in object detection algorithms. As a branch of artificial intelligence, anchor box belongs to the intersection of computer science and image processing. The main function of anchor box is to predefine possible object bounding boxes when training object detection models to help the model learn how to identify and locate various objects in the image. These predefined boxes have different shapes and sizes to accommodate different types of objects. Anchor box technology is often used in deep learning frameworks to improve detection accuracy by presetting boxes of multiple shapes and sizes. For example, in traffic monitoring systems, anchor boxes can help models accurately identify and locate different types of vehicles; in retail analysis, it can be used to detect goods on shelves. By simulating and optimizing the process of object positioning in images, anchor box technology has promoted the development of image recognition technology and made computer vision applications more accurate and reliable.

[0059] Multi-object detection: Multi-object detection is a technology in the field of computer vision and artificial intelligence. It is used to simultaneously identify and locate multiple objects from images or videos. It is a branch of artificial intelligence and an interdisciplinary subject between computer science and image processing. Multi-object detection uses deep learning models to identify different types of objects and determine their locations and sizes by analyzing pixels and features in images. This technology is widely used in technical fields such as self-driving cars, video surveillance, robot navigation, and intelligent interaction. The implementation of multi-object detection usually involves convolutional neural networks or other types of artificial neural networks, which improve the accuracy and efficiency of detection by training a large amount of labeled data. This technology promotes the development and application of computer vision technology by simulating the human eye's ability to analyze complex scenes, and provides a basis for visual understanding for various intelligent systems.

[0060] Positive samples refer to instances that belong to the target category or meet specific conditions in the data set, and negative samples refer to instances that do not belong to the target category or meet specific conditions in the data set. For example, when selecting a target object in an image, a positive sample is an anchor box that includes the target object to be detected, and a negative sample is an anchor box that does not include the target object to be detected. However, when the image includes multiple objects and the objects overlap, the anchor boxes will affect each other, making it difficult to define the boundaries of the anchor boxes so that the anchor boxes can anchor the target object as much as possible globally without affecting the anchoring of other anchor boxes to the target object. Therefore, how to improve the recognition accuracy of positive and negative samples in images has become an urgent problem to be solved.

[0061] Based on this, the embodiments of the present application provide a positive and negative sample identification method and device, an electronic device and a storage medium, aiming to improve the recognition accuracy of positive and negative samples in images.

[0062] The embodiments of the present application provide a positive and negative sample identification method and device, an electronic device, and a storage medium, which are specifically described through the following embodiments. First, the positive and negative sample identification method in the embodiments of the present application is described.

[0063] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0064] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0065] The positive and negative sample identification method provided in the embodiment of the present application relates to the field of image processing technology. The positive and negative sample identification method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or can be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the positive and negative sample identification method, etc., but is not limited to the above forms.

[0066] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0067] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0068] Figure 1 is an optional flowchart of the positive and negative sample identification method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.

[0069] Step S101, obtaining a sample image containing at least two sample targets;

[0070] Step S102, obtaining the sample coordinates of each sample target in the sample image;

[0071] Step S103, calculating the anchor loss according to the sample coordinates and the preset anchor frame to obtain an anchor loss matrix;

[0072] Step S104, calculating the number of anchor boxes for the sample targets according to the anchor loss matrix to obtain the number of anchor boxes for each sample target;

[0073] Step S105, assigning anchor boxes to sample targets according to the number of anchor boxes, the anchor loss matrix and the preset anchor boxes, and obtaining positive anchor boxes and negative anchor boxes for each sample target.

[0074] In the steps S101 to S105 shown in the embodiment of the present application, a sample image containing at least two sample targets is obtained, and then the sample coordinates of each sample target in the sample image are obtained, and then the anchor loss is calculated according to the sample coordinates and the preset anchor frame to obtain the anchor loss matrix, thereby determining the loss value between the pixel point of the sample target and the anchor frame, and providing a data basis for the subsequent global anchor frame allocation based on the loss value; further, the number of anchor frames of the sample target is calculated according to the anchor loss matrix to obtain the number of anchor frames of each sample target. , thereby determining the number of anchor boxes for each sample target, so that the anchor box of each sample target can anchor the sample object as much as possible while minimizing the anchoring influence on other sample targets; further, the sample targets are assigned anchor boxes according to the number of anchor boxes, the anchor loss matrix and the preset anchor boxes, and the positive anchor boxes and negative anchor boxes of each sample target are obtained, thereby improving the recognition accuracy of positive and negative samples in the image, achieving the goal that the anchor box can anchor the sample object without affecting other anchor boxes anchoring other sample objects, and achieving the best anchoring effect globally.

[0075] In step S101 of some embodiments, the sample target refers to the target in the sample image, for example, in the image of the traffic road, the sample target includes "car", "pedestrian" or "traffic light", etc. For example, in the animal image, the sample target may be various animals. Each sample image is an image, which may be obtained from a video frame or a camera, and each sample image contains at least two sample targets.

[0076] In step S102 of some embodiments, the sample coordinates are the pixel coordinates of the sample target in the sample image, which can be obtained by pre-setting the pixel coordinate values ​​or by obtaining them through a pre-trained deep learning network, such as using a YOLO (You Only Look Once) deep learning network for pixel annotation or by using a Faster R-CNN deep learning network for pixel annotation. The specific method of annotating pixels is not specifically limited in this application.

[0077] See also Figure 2 In some embodiments, step S103 may include but is not limited to steps S201 to S204:

[0078] Step S201, screening overlapping coordinates according to sample coordinates and a preset anchoring frame to obtain overlapping coordinates;

[0079] Step S202, performing area calculation according to the overlapping coordinates to obtain the overlapping coordinate area;

[0080] Step S203, calculating the area of ​​the anchor frame to obtain the area of ​​the anchor frame;

[0081] Step S204, calculating the ratio between the overlapping coordinate area and the anchor box area to obtain an anchor loss matrix.

[0082] In steps S201 to S204 shown in the embodiment of the present application, overlapping coordinates are screened according to the sample coordinates and the preset anchor frame to obtain the overlapping coordinates, and then the area is calculated according to the overlapping coordinates to obtain the overlapping coordinate area, and then the area is calculated according to the anchor frame to obtain the anchor frame area, and finally the ratio between the overlapping coordinate area and the anchor frame area is calculated to obtain the anchor loss matrix, so as to determine the loss value between the pixel point of the sample target and the anchor frame, and provide a data basis for the subsequent global anchor frame allocation based on the loss value.

[0083] In step S201 of some embodiments, the preset anchor frame includes multiple preset anchor frames, the minimum size of each anchor frame is a 1×1 anchor frame, the maximum size is consistent with the size of the sample image, and the size of the anchor frame can be arbitrarily set between 1×1 and the size of the sample image. The present application does not impose specific restrictions on the specific size of the anchor frame, and does not impose specific restrictions on the number of anchor frames. The overlapping area of ​​each anchor frame is screened out according to each sample target, that is, the coordinates of the sample target anchored by the anchor frame are the overlapping coordinates. For example, the sample coordinates are ((0,0), (0,1), (1,0), (1,1), (2,1)), one of the anchor frames in the preset anchor frame is a 2×3 anchor frame, and the four rectangular coordinates of the frame are ((0,0), (0,2), (1,0), (1,2)), respectively, then the overlapping coordinates are ((0,0), (0,1), (1,0), (1,1)).

[0084] In step S202 of some embodiments, the number of overlapping coordinates is counted as the overlapping coordinate area. For example, if the overlapping coordinates are ((0,0), (0,1), (1,0), (1,1)), the overlapping coordinate area is 4.

[0085] In step S203 of some embodiments, the number of all coordinate points in the anchor box is counted as the area of ​​the anchor box. For example, one of the anchor boxes in the preset anchor box is a 2×3 anchor box, and the four rectangular coordinates of the box are ((0,0), (0,2), (1,0), (1,2)), and the area of ​​the anchor box is 6.

[0086] In step S204 of some embodiments, the ratio between the overlapping coordinate area and the anchor box is calculated to obtain the anchor loss of the sample target to the anchor box. For one anchor box, the anchor loss between it and all sample targets is counted, and then all anchor boxes are traversed to obtain the anchor loss between all anchor boxes and all sample targets, and obtain the anchor loss matrix. For example, there are a first sample target and a second sample target, a first anchor box and a second anchor box, a first sample target and a second sample target, and the anchor loss between the first anchor box and the second anchor box is shown in Table 1:

[0087] First anchor box Second anchor box First sample target 0.6 0.7 Second sample target 0.2 0.1

[0088] Table 1

[0089] See also Figure 3 In some embodiments, step S104 may include but is not limited to steps S301 to S303:

[0090] Step S301, filtering the anchor loss matrix according to a preset anchor box filtering threshold to obtain a filtering loss matrix;

[0091] Step S302, summing up the screening loss matrix to obtain a total screening loss value;

[0092] Step S303, rounding down the total value of the screening loss to obtain the number of anchor boxes for each sample target.

[0093] In steps S301 to S303 shown in the embodiment of the present application, the anchor loss matrix is ​​screened according to a preset anchor box screening threshold to obtain a screening loss moment, and then the screening loss matrix is ​​summed to obtain a total screening loss value. Finally, the total screening loss value is rounded down to obtain the number of anchor boxes for each sample target, so as to globally enable the anchor box of each sample target to anchor the sample object as much as possible while minimizing the anchoring influence on other sample targets.

[0094] In steps S301 to S303 of some embodiments, the preset anchor box screening threshold is a preset threshold for screening the anchor box. For example, for each sample target, the anchor box screening threshold is set to 3, that is, for a sample target, the values ​​of the matrix with the top 3 overlapping areas with the anchor box are selected to obtain the screening loss matrix. For example, there are three sample targets and six anchor boxes, and the preset anchor box screening threshold is 3, as shown in Table 2:

[0095] Box 1 Box 2 Box 3 Box 4 Box 5 Box 6 Goal 1 0.7 0.2 0.5 0.1 0.3 0.4 Goal 2 0.1 0.8 0.3 0.6 0.2 0.5 Goal 3 0.4 0.3 0.6 0.2 0.9 0.7

[0096] Table 2

[0097] For Target 1, select Box 1, Box 3, and Box 6.

[0098] For target 2, select box 2, box 4, and box 6.

[0099] For target 3, select box 5, box 6 and box 3, and then merge the data to obtain the screening loss matrix, as shown in Table 3:

[0100] Goal 1 0.7 0.5 0.4 Goal 2 0.8 0.6 0.5 Goal 3 0.9 0.7 0.6

[0101] Table 3

[0102] Then the screening loss matrix is ​​summed to obtain the total screening loss value. For example, the total screening loss value of target 1 is 1.6, the total screening loss value of target 2 is 1.9, and the total screening loss value of target 3 is 2.2.

[0103] Then the total loss value is rounded down to obtain the number of anchor boxes for each sample target. For example, the number of anchor boxes for target 1 is 1, the number of anchor boxes for target 2 is 1, and the number of anchor boxes for target 3 is 2.

[0104] See also Figure 4 In some embodiments, step S105 may include but is not limited to steps S401 to S405:

[0105] Step S401, performing aggregation calculation according to the anchor loss matrix and the preset regularization parameter to obtain a kernel matrix;

[0106] Step S402, performing aggregation calculation according to the preset column matrix, core matrix and number of anchor boxes to obtain a target row matrix;

[0107] Step S403, performing aggregation calculation according to the target row matrix and the kernel matrix to obtain a target column matrix;

[0108] Step S404, performing aggregation calculation according to the target column matrix and the target row matrix to obtain a transmission matrix;

[0109] Step S405 , assigning anchor frames to sample targets according to the number of anchor frames, the transfer matrix, and the preset anchor frames, to obtain a positive anchor frame and a negative anchor frame for each sample target.

[0110] In steps S401 to S405 shown in the embodiment of the present application, aggregation calculations are performed according to the anchor loss matrix and the preset regularization parameter to obtain a kernel matrix, and then aggregation calculations are performed according to the preset column matrix, the kernel matrix and the number of anchor boxes to obtain a target row matrix, and then aggregation calculations are performed according to the target row matrix and the kernel matrix to obtain a target column matrix, and then aggregation calculations are performed according to the target column matrix and the target row matrix to obtain a transfer matrix, and finally anchor boxes are assigned to sample targets according to the number of anchor boxes, the transfer matrix and the preset anchor boxes to obtain positive anchor boxes and negative anchor boxes for each sample target, thereby improving the recognition accuracy of positive and negative samples in the image, achieving the goal that the anchor box can anchor the sample object without affecting other anchor boxes anchoring other sample objects, and achieving the best anchoring effect globally.

[0111] In step S401 of some embodiments, an aggregation calculation is performed according to the anchor loss matrix and the preset regularization parameter to obtain a kernel matrix, and the aggregation calculation is shown in formula (1):

[0112]

[0113] Among them, K is the kernel matrix, exp() is the exponential function, c is the anchor loss matrix, and ∈ is the preset regularization parameter.

[0114] In step S402 of some embodiments, an aggregation calculation is performed according to a preset column matrix, a core matrix, and a number of anchor boxes to obtain a target row matrix. The aggregation calculation is shown in formula (2):

[0115]

[0116] Among them, u k+1 is the value of the row matrix at the k+1th iteration, a is the matrix composed of the number of anchor boxes corresponding to all sample targets, K is the kernel matrix, and v k is the value of the column matrix at the kth iteration. After reaching the preset number of iterations G, u G is the target row matrix. Initially, both the row matrix and the column matrix are all 1 vectors.

[0117] In step S403 of some embodiments, an aggregation calculation is performed based on the target row matrix and the kernel matrix to obtain a target column matrix. The aggregation calculation is shown in formula (3):

[0118]

[0119] Among them, v k+1 is the value of the column matrix at the k+1th iteration, 1 means that an anchor box can only be assigned to one sample target, K T is the transpose of the kernel matrix, v k+1is the value of the row matrix at the k+1th iteration. After reaching the preset number of iterations G, v G is the target column matrix.

[0120] It should be noted that the calculation of the row matrix and the column matrix is ​​an iterative process. After reaching a predetermined number of times, the target row matrix and the target column matrix are obtained.

[0121] In step S404 of some embodiments, an aggregation calculation is performed according to the target column matrix and the target row matrix to obtain a transmission matrix, and the aggregation calculation is shown in formula (4):

[0122] T=diag(u)·K·diag(v) (4),

[0123] Where T is the transmission matrix, diag is the diagonal function, u is the target row matrix, and v is the target column matrix.

[0124] See also Figure 5 In some embodiments, the transmission matrix includes the adaptability data of each sample target to each preset anchor frame, and step S405 includes but is not limited to steps S501 to S503:

[0125] Step S501, sorting the corresponding sample targets in descending order according to all preset anchor boxes and the fitness data corresponding to each preset anchor box to obtain a target descending sequence;

[0126] Step S502, assigning anchor boxes to corresponding sample targets according to the target descending sequence and the number of anchor boxes corresponding to each sample target, to obtain a positive anchor box for each sample target;

[0127] Step S503: Filter out the positive anchor box of each sample target according to the preset anchor box to obtain the negative anchor box of each sample target.

[0128] In steps S501 to S503 shown in the embodiment of the present application, the corresponding preset anchor boxes are sorted in descending order according to the fitness data corresponding to all sample targets and each preset anchor box to obtain a descending sequence of anchor boxes, and then the anchor boxes are allocated to the corresponding sample targets according to the descending sequence of anchor boxes and the number of anchor boxes corresponding to each sample target to obtain a positive anchor box for each sample target. Finally, the positive anchor box of each sample target is filtered out according to the preset anchor box to obtain a negative anchor box for each sample target, thereby obtaining a positive anchor box and a negative anchor box for each sample target, thereby improving the recognition accuracy of positive and negative samples in the picture, achieving the goal that the anchor box can anchor the sample object without affecting other anchor boxes anchoring other sample objects, and achieving the best anchoring effect globally.

[0129] In step S501 of some embodiments, the corresponding sample targets are sorted in descending order according to all preset anchor boxes and the fitness data corresponding to each preset anchor box. For example, the transfer matrix is ​​shown in Table 4:

[0130] Box 1 Box 2 Box 3 Box 4 Box 5 Goal 1 0.8 0.1 0.3 0.4 0.2 Goal 2 0.2 0.9 0.4 0.1 0.5 Goal 3 0.1 0.2 0.8 0.3 0.6

[0131] Table 4

[0132] The target descending sequence is: (box 1, target 1, target 2, target 3), (box 2, target 2, target 3, target 1), (box 3, target 3, target 2, target 1), (box 4, target 1, target 3, target 2), (box 5, target 3, target 2, target 1).

[0133] In step S502 of some embodiments, anchor boxes are assigned to corresponding sample targets according to the target descending sequence and the number of anchor boxes corresponding to each sample target. For example, the number of anchor boxes required for target 1 is 2, the number of anchor boxes required for target 2 is 1, and the number of anchor boxes required for target 3 is 2. The target descending sequence is: (box 1, target 1, target 2, target 3), (box 2, target 2, target 3, target 1), (box 3, target 3, target 2, target 1), (box 4, target 1, target 3, target 2), (box 5, target 3, target 2, target 1).

[0134] For box 1, assign to target 1. For box 2, assign to target 2. For box 3, assign to target 3. For box 4, assign to target 1. For box 5, assign to target 3.

[0135] Thus, a positive anchor box for each sample target is obtained.

[0136] In step S503 of some embodiments, the positive anchor box of each sample target is filtered out according to the preset anchor box to obtain the negative anchor box of each sample target. For example, there are five anchor boxes in total, and the positive anchor boxes corresponding to the first sample target are the first anchor box and the fourth anchor box. Then, the five anchor boxes are filtered out according to the first anchor box and the fourth anchor box to obtain the second anchor box, the third anchor box and the fifth anchor box as the negative anchor boxes of the sample target.

[0137] It should be noted that, by allocating anchor boxes, the anchor boxes can globally contain the most information about sample targets. For example, if the sample object is a car with a long rope tied to the tail, the anchor boxes can be allocated to frame the car but not the rope, so that the anchor boxes can contain the most important information.

[0138] See also Figure 6 In some embodiments, the positive and negative sample identification method further includes, but is not limited to, steps S601 to S602 after step S105:

[0139] Step S601, obtaining a sample category of a sample target, and training a preset multi-target detection model according to the sample image, the sample category of the sample target, the positive anchor box, and the negative anchor box to obtain a target multi-target detection model;

[0140] Step S602, obtaining a selected image, and performing multi-target detection on the selected image according to a target multi-target detection model.

[0141] In steps S601 to S602 shown in the embodiment of the present application, the sample category of the sample target is obtained, and the preset multi-target detection model is trained according to the sample image, the sample category of the sample target, the positive anchor box and the negative anchor box to obtain a target multi-target detection model, and then the selected image is obtained, and multi-target detection is performed on the selected image according to the target multi-target detection model, so that the preset multi-target detection model is trained by using the accurate anchor box as the data set, so that the preset multi-target detection model can well perceive the relationship between the image and the target, and a multi-target detection model that can make accurate predictions is obtained, and accurate predictions are made for the selected image.

[0142] In step S601 of some embodiments, the sample category is the category of the sample target, such as a car, a pedestrian, or a traffic light.

[0143] See also Figure 7 In some embodiments, step S601 may include but is not limited to steps S701 to S703:

[0144] Step S701, inputting the sample image into a preset multi-target detection model for multi-target detection, and obtaining the predicted category and predicted anchor box of the sample target;

[0145] Step S702, calculating the loss value according to the predicted anchor box, the positive anchor box, the negative anchor box, the predicted category and the sample category to obtain the training loss value;

[0146] Step S703, optimizing the parameters of the preset multi-target detection model according to the training loss value to obtain a target multi-target detection model.

[0147] In steps S701 to S703 shown in the embodiment of the present application, multi-target detection is performed by inputting the sample image into a preset multi-target detection model to obtain the predicted category and predicted anchor box of the sample target, and then the loss value is calculated according to the predicted anchor box, the positive anchor box, the negative anchor box, the predicted category and the sample category to obtain the training loss value, and finally the parameters of the preset multi-target detection model are optimized according to the training loss value to obtain the target multi-target detection model, so that the target multi-target detection model can perform accurate multi-target detection based on the image.

[0148] In step S701 of some embodiments, the preset multi-target detection model is a deep learning network that can detect targets in an image, select pixel regions where targets are located, and output the categories of targets. The sample image is input into the preset multi-target detection model, and the preset multi-target detection model performs multi-target detection to obtain the predicted category and predicted anchor box of the sample target.

[0149] In step S702 of some embodiments, a loss value is calculated according to the predicted anchor box, the positive anchor box, the negative anchor box, the predicted category and the sample category to obtain a training loss value. First, a cross entropy calculation is performed according to the predicted anchor box, the positive anchor box and the negative anchor box to obtain an anchor box loss value, and then the regression loss of the predicted category and the sample category is calculated to obtain a category loss value. Then, an aggregate calculation is performed based on a preset anchor box balance factor, a regression balance factor, an anchor box loss value and a category loss value to obtain a training loss value. The aggregate calculation is shown in formula (5):

[0150] L=λ1·L1+λ2·L2 (5),

[0151] Among them, L is the training loss value, λ1 is the anchor box balance factor, L1 is the anchor box loss value, λ2 is the regression balance factor, and L2 is the category loss value.

[0152] The parameters of the preset multi-target detection model are optimized according to the training loss value to obtain the target multi-target detection model.

[0153] In step S703 of some embodiments, the parameters of the preset multi-target detection model are optimized according to the training loss value by using the stochastic gradient descent method to obtain the target multi-target detection model.

[0154] In step S602 of some embodiments, an image is selected as an image that needs to be subjected to multi-target detection, the selected image is input into a target multi-target detection model, and the target multi-target detection model performs multi-target detection.

[0155] See also Figure 8 The embodiment of the present application also provides a positive and negative sample identification device, which can implement the above positive and negative sample identification method, and the device includes:

[0156] An image acquisition module 801 is used to acquire a sample image containing at least two sample targets;

[0157] A coordinate acquisition module 802 is used to acquire the sample coordinates of each sample target in the sample image;

[0158] The loss calculation module 803 is used to calculate the anchor loss according to the sample coordinates and the preset anchor frame to obtain the anchor loss matrix;

[0159] A quantity calculation module 804 is used to calculate the number of anchor boxes for the sample targets according to the anchor loss matrix to obtain the number of anchor boxes for each sample target;

[0160] The anchor box assignment module 805 is used to assign anchor boxes to sample targets according to the number of anchor boxes, the anchor loss matrix and the preset anchor boxes, and obtain the positive anchor box and the negative anchor box of each sample target.

[0161] The specific implementation of the positive and negative sample identification device is basically the same as the specific implementation of the positive and negative sample identification method described above, and will not be repeated here.

[0162] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned positive and negative sample identification method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0163] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0164] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0165] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the positive and negative sample identification method of the embodiment of this application;

[0166] Input / output interface 903, used to implement information input and output;

[0167] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0168] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0169] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0170] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned positive and negative sample identification method is implemented.

[0171] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0172] The positive and negative sample identification method, positive and negative sample identification device, electronic device and storage medium provided in the embodiments of the present application obtain a sample image containing at least two sample targets, then obtain the sample coordinates of each sample target in the sample image, and then perform anchor loss calculation according to the sample coordinates and a preset anchor frame to obtain an anchor loss matrix, thereby determining the loss value between the pixel point of the sample target and the anchor frame, and providing a data basis for subsequent global anchor frame allocation based on the loss value; further, the number of anchor frames of the sample target is calculated according to the anchor loss matrix to obtain the anchor loss matrix of each sample target. The number of anchor boxes of the target is determined, so as to determine the number of anchor boxes of each sample target, so that the anchor box of each sample target can anchor the sample object as much as possible while minimizing the anchoring influence on other sample targets; further, the sample targets are assigned anchor boxes according to the number of anchor boxes, the anchor loss matrix and the preset anchor boxes, and the positive anchor box and the negative anchor box of each sample target are obtained, so as to improve the recognition accuracy of positive and negative samples in the picture, so that the anchor box can anchor the sample object without affecting other anchor boxes anchoring other sample objects, and achieve the best anchoring effect globally.

[0173] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0174] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0175] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0176] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0177] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0178] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0179] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0180] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0181] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0182] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0183] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A positive and negative sample identification method, characterized in that: The method comprises: Acquire a sample image containing at least two sample objects; Obtaining the sample coordinates of each of the sample objects in the sample image; Anchor loss calculation is performed according to the sample coordinates and the preset anchor frame to obtain an anchor loss matrix; Calculating the number of anchor boxes for the sample targets according to the anchor loss matrix to obtain the number of anchor boxes for each sample target; Anchor boxes are assigned to the sample targets according to the number of anchor boxes, the anchor loss matrix, and the preset anchor boxes to obtain positive anchor boxes and negative anchor boxes for each of the sample targets.

2. The method according to claim 1, characterized in that The assigning anchor frames to the sample targets according to the number of anchor frames, the anchor loss matrix and the preset anchor frames to obtain a positive anchor frame and a negative anchor frame of each sample target includes: Performing aggregation calculation according to the anchor loss matrix and the preset regularization parameter to obtain a kernel matrix; Performing aggregation calculation according to the preset column matrix, the core matrix and the number of anchor boxes to obtain a target row matrix; Performing aggregation calculation according to the target row matrix and the kernel matrix to obtain a target column matrix; Performing aggregation calculation according to the target column matrix and the target row matrix to obtain a transmission matrix; Anchor frames are allocated to the sample targets according to the number of anchor frames, the transfer matrix, and the preset anchor frames to obtain the positive anchor frame and the negative anchor frame of each sample target.

3. The method according to claim 2, characterized in that The transfer matrix includes the adaptability data of each sample target to each preset anchor frame, and the anchor frame allocation is performed on the sample target according to the number of anchor frames, the transfer matrix and the preset anchor frame to obtain the positive anchor frame and the negative anchor frame of each sample target, including: According to all the preset anchor boxes and the fitness data corresponding to each preset anchor box, the corresponding sample targets are sorted in descending order to obtain a target descending sequence; Allocating anchor boxes to the corresponding sample targets according to the target descending sequence and the number of anchor boxes corresponding to each sample target, to obtain the positive anchor box of each sample target; The positive anchor box of each of the sample targets is filtered out according to the preset anchor box to obtain the negative anchor box of each of the sample targets.

4. The method according to claim 1, characterized in that: The calculating the number of anchor boxes for the sample targets according to the anchor loss matrix to obtain the number of anchor boxes for each sample target includes: The anchor loss matrix is ​​screened according to a preset anchor box screening threshold to obtain a screening loss matrix; Summing the screening loss matrix to obtain a total screening loss value; The total value of the screening loss is rounded down to obtain the number of anchor boxes for each sample target.

5. The method according to claim 1, characterized in that The anchor loss calculation is performed according to the sample coordinates and the preset anchor frame to obtain an anchor loss matrix, including: Perform overlapping coordinate screening according to the sample coordinates and the preset anchor frame to obtain overlapping coordinates; Performing area calculation according to the overlapping coordinates to obtain the overlapping coordinate area; Performing area calculation according to the anchor frame to obtain the anchor frame area; The ratio between the overlapping coordinate area and the anchor box area is calculated to obtain the anchor loss matrix.

6. The method according to any one of claims 1 to 5, characterized in that: After assigning anchor frames to the sample targets according to the number of anchor frames, the anchor loss matrix and the preset anchor frames to obtain a positive anchor frame and a negative anchor frame of each sample target, the method further includes: Obtaining a sample category of the sample target, and training a preset multi-target detection model according to the sample image, the sample category of the sample target, the positive anchor box, and the negative anchor box to obtain a target multi-target detection model; A selected image is acquired, and multi-target detection is performed on the selected image according to the target multi-target detection model.

7. The method according to claim 6, characterized in that The method of training a preset multi-target detection model according to the sample image, the sample category of the sample target, the positive anchor box, and the negative anchor box to obtain a target multi-target detection model includes: Inputting the sample image into the preset multi-target detection model to perform multi-target detection, and obtaining the predicted category and predicted anchor box of the sample target; Calculate the loss value according to the predicted anchor box, the positive anchor box, the negative anchor box, the predicted category and the sample category to obtain a training loss value; Parameters of the preset multi-target detection model are optimized according to the training loss value to obtain the target multi-target detection model.

8. A positive and negative sample identification device, characterized in that: The device comprises: An image acquisition module, used to acquire a sample image containing at least two sample targets; A coordinate acquisition module, used for acquiring the sample coordinates of each sample target in the sample image; A loss calculation module, used to calculate the anchor loss according to the sample coordinates and the preset anchor frame to obtain an anchor loss matrix; A quantity calculation module, used to calculate the number of anchor boxes of the sample targets according to the anchor loss matrix, to obtain the number of anchor boxes of each sample target; The anchor box allocation module is used to allocate anchor boxes to the sample targets according to the number of anchor boxes, the anchor loss matrix and the preset anchor boxes to obtain a positive anchor box and a negative anchor box for each of the sample targets.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the positive and negative sample identification method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the positive and negative sample identification method according to any one of claims 1 to 7 is implemented.

Images