A data identification method, device, equipment and readable storage medium

Abstract

The present application discloses a data processing method, device, equipment and readable storage medium, the method comprising: obtaining a target image, identifying N prediction areas of a target object in the target image; obtaining the prediction object category and prediction object label features respectively corresponding to the target object in each prediction area; obtaining the maximum category prediction probability among the category prediction probabilities respectively corresponding to the N prediction object categories, and determining the prediction area corresponding to the prediction object category with the maximum category prediction probability as the target prediction area; determining the coverage area commonly covered by the remaining prediction area and the target prediction area; determining the label feature similarity between the prediction object label features corresponding to the remaining prediction area and the prediction object label features corresponding to the target prediction area; determining the optimal prediction area of ​​the target object in the target image according to the similarity between the coverage area and the label features. The present application can improve the recognition accuracy of objects in images.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data identification method, device, equipment and readable storage medium. Background Art

[0002] Object detection is a very important computer vision task, which aims to identify the location and category of an object in an image. At present, the process of object detection is as follows: the convolutional neural network first outputs all possible detection boxes of objects in the image; then, based on the post-processing method of non-maximum suppression (NMS), the optimal detection box of each object is determined in these detection boxes.

[0003] Among them, for the post-processing method of non-maximum suppression, its main processing flow is: first, classify each detection frame according to its predicted object category, and sort the detection frames belonging to the same category according to the confidence level; then, calculate the overlapping area between the first-ranked detection frame and each other detection frame, and suppress (for example, delete) the detection frame whose overlapping area intersection over union (IOU) is greater than a certain threshold; finally, the detection frame with the highest confidence level of each category can be obtained.

[0004] However, the current post-processing method based on non-maximum suppression of overlapping areas is likely to have the problem of false suppression when two (or more) objects belonging to the same category in the image are very close. Because when the objects are close in the image, the detection frames corresponding to these objects have a large overlapping area, and their overlapping area intersection-over-union ratio is likely to be greater than the threshold, so false suppression will occur. For example, if user a and user b are very close in the image, their respective corresponding detection frames will also be very close, and their overlapping area intersection-over-union ratio is greater than the threshold, then the detection frame corresponding to user a or the detection frame corresponding to user b is likely to be deleted, that is, the detection frame that should not be deleted is likely to be deleted. In order to improve the problem of false suppression when the image objects are close, the threshold of the overlapping area intersection-over-union ratio can be increased, but this method of increasing the threshold will make some detection frames that should be suppressed unable to be successfully suppressed, so there is still a problem of false suppression, so that the accuracy of the final detection result is not high. Summary of the invention

[0005] The embodiments of the present application provide a data recognition method, apparatus, device, and readable storage medium, which can improve the recognition accuracy of objects in images.

[0006] On the one hand, an embodiment of the present application provides a data identification method, including:

[0007] Obtain a target image, and identify N predicted regions of the target object in the target image; N is a positive integer;

[0008] Obtain the predicted object category and predicted object label features corresponding to the target object in each prediction area;

[0009] Among the category prediction probabilities corresponding to the N prediction object categories, the maximum category prediction probability is obtained, and the prediction area corresponding to the prediction object category with the maximum category prediction probability is determined as the target prediction area;

[0010] Determine the coverage area jointly covered by the remaining prediction area and the target prediction area; the remaining prediction area is the prediction area other than the target prediction area among the N prediction areas;

[0011] Determine the label feature similarity between the predicted object label feature corresponding to the remaining predicted area and the predicted object label feature corresponding to the target predicted area;

[0012] According to the similarity between the coverage area and the label feature, the optimal prediction area of ​​the target object in the target image is determined among the N prediction areas.

[0013] An embodiment of the present application provides a data identification device, including:

[0014] A region recognition module, used to obtain a target image;

[0015] The region recognition module is further used to identify N predicted regions of the target object in the target image; N is a positive integer;

[0016] A data acquisition module, used to obtain the predicted object category and predicted object label features corresponding to the target object in each prediction area;

[0017] A target region acquisition module is used to obtain the maximum category prediction probability among the category prediction probabilities corresponding to the N prediction object categories, and determine the prediction region corresponding to the prediction object category with the maximum category prediction probability as the target prediction region;

[0018] A coverage area determination module is used to determine a coverage area jointly covered by the remaining prediction area and the target prediction area; the remaining prediction area is a prediction area other than the target prediction area among the N prediction areas;

[0019] A similarity determination module is used to determine the label feature similarity between the predicted object label feature corresponding to the remaining prediction area and the predicted object label feature corresponding to the target prediction area;

[0020] The optimal region determination module is used to determine the optimal prediction region of the target object in the target image among the N prediction regions according to the similarity between the coverage region and the label feature.

[0021] In one embodiment, the region identification module includes:

[0022] An image input unit, used to input a target image into an image recognition model, and output a candidate prediction region corresponding to the target image, an object existence probability corresponding to the candidate prediction region, and an initial category prediction probability corresponding to the candidate prediction region through the image recognition model; the object existence probability refers to the probability of an object existing in the candidate prediction region;

[0023] The region identification unit is used to obtain N prediction regions of the target object in the target image from among the candidate prediction regions according to the object existence probability and the initial category prediction probability corresponding to the candidate prediction region.

[0024] In one embodiment, the region identification unit includes:

[0025] A region selection subunit, used to determine a candidate prediction region where the object existence probability is greater than a probability threshold as a prediction region to be identified;

[0026] The category determination subunit is used to obtain the maximum initial category prediction probability from the initial category prediction probabilities corresponding to the prediction area to be identified, and determine the object category corresponding to the maximum initial category prediction probability as the predicted object category of the object contained in the prediction area to be identified;

[0027] The region classification subunit is used to classify the prediction region to be identified according to the prediction object category of the object contained in the prediction region to be identified, so as to obtain M region sets; the prediction object category of the object contained in the prediction region to be identified included in one region set belongs to the same object category;

[0028] The region recognition subunit is used to obtain a target region set from the M region sets, determine the object contained in the prediction region to be identified included in the target region set as the target object, and determine the prediction region to be identified included in the target region set as N prediction regions of the target object in the target image.

[0029] In one embodiment, the data identification device further includes:

[0030] A feature extraction module, used to extract regional features of candidate prediction regions through an image recognition model;

[0031] The label feature extraction module is used to extract regional features to obtain the label features of the predicted object corresponding to the candidate prediction region; the label features of the predicted object corresponding to the candidate prediction region include the label features of the predicted object corresponding to the N prediction regions respectively.

[0032] In one embodiment, the optimal region determination module includes:

[0033] A region operation unit, used for taking the region in the target prediction region that is not covered by the remaining prediction region as the first region to be operated;

[0034] The area operation unit is further used to use the area in the remaining prediction area that is not covered by the target prediction area as the second area to be operated;

[0035] The area calculation unit is further used to add the area corresponding to the first area to be calculated and the area corresponding to the second area to be calculated to obtain the area of ​​the first area;

[0036] The area calculation unit is further used to obtain the area of ​​the second area corresponding to the coverage area, and determine the area ratio between the area of ​​the second area and the area of ​​the first area;

[0037] The optimal region determination unit is used to determine the optimal prediction region of the target object in the target image from among the N prediction regions according to the area ratio and the label feature similarity.

[0038] In one embodiment, the optimal region determining unit is further specifically configured to determine the target prediction region among the N prediction regions as the optimal prediction region of the target object in the target image when the area ratio is greater than or equal to the ratio threshold and the label feature similarity is greater than or equal to the similarity threshold;

[0039] The optimal region determination unit is also specifically used to determine the remaining prediction regions and the target prediction region in the N prediction regions as the optimal prediction regions of the target object in the target image when the area ratio is less than the ratio threshold or the label feature similarity is less than the similarity threshold.

[0040] On the one hand, an embodiment of the present application provides another data identification method, including:

[0041] Obtain a sample training image, input the sample training image into a sample image recognition model, and output M sample prediction regions of the sample object in the sample training image, the sample prediction object category corresponding to the sample object in each sample prediction region, and the sample object label features corresponding to the sample object in each sample prediction region through the first convolutional network layer in the sample image recognition model; M is a positive integer;

[0042] Inputting the M sample object label features into the second convolutional network layer in the sample image recognition model, and outputting the predicted classification result between every two sample object label features among the M sample object label features through the second convolutional network layer;

[0043] Get the real area corresponding to each sample prediction area, and get the real object category corresponding to each real area;

[0044] Determine a first loss value according to the M sample prediction regions, the real regions corresponding to each sample prediction region, the real object categories corresponding to each real region, and the sample prediction object categories corresponding to the sample objects in each sample prediction region;

[0045] Determine a second loss value according to the predicted classification result between every two sample object label features among the M sample object label features and the actual classification result between every two sample object label features;

[0046] The sample image recognition model is adjusted according to the first loss value and the second loss value to obtain an image recognition model; the image recognition model is used to identify N prediction areas of the target object in the target image, the prediction object categories corresponding to the target objects in the N prediction areas, and the prediction object label features corresponding to the target objects in the N prediction areas; the prediction object categories corresponding to the target objects in the N prediction areas and the prediction object label features corresponding to the target objects in the N prediction areas are used to determine the optimal prediction area where the target object is located in the N prediction areas; N is a positive integer.

[0047] On the one hand, an embodiment of the present application provides another data identification device, including:

[0048] A sample input module is used to obtain a sample training image, input the sample training image into a sample image recognition model, and output M sample prediction regions of the sample object in the sample training image, the sample prediction object category corresponding to the sample object in each sample prediction region, and the sample object label features corresponding to the sample object in each sample prediction region through the first convolutional network layer in the sample image recognition model; M is a positive integer;

[0049] A real label acquisition module is used to obtain the real area corresponding to each sample prediction area;

[0050] The real label acquisition module is also used to obtain the real object category corresponding to each real area;

[0051] A loss value generating module, used to determine a first loss value according to the M sample prediction regions, the real regions corresponding to each sample prediction region, the real object categories corresponding to each real region, and the sample prediction object categories corresponding to the sample objects in each sample prediction region;

[0052] The loss value generating module is further used to determine a second loss value according to the predicted classification result between every two sample object label features in the M sample object label features and the actual classification result between every two sample object label features;

[0053] A model adjustment module is used to adjust the sample image recognition model according to the first loss value and the second loss value to obtain an image recognition model; the image recognition model is used to identify N prediction areas of the target object in the target image, the prediction object categories corresponding to the target objects in the N prediction areas, and the prediction object label features corresponding to the target objects in the N prediction areas; the prediction object categories corresponding to the target objects in the N prediction areas and the prediction object label features corresponding to the target objects in the N prediction areas are used to determine the optimal prediction area where the target object is located in the N prediction areas; N is a positive integer.

[0054] In one embodiment, the M sample prediction regions include sample prediction region k i ; i is a positive integer;

[0055] The real label acquisition module includes:

[0056] A configuration region acquisition unit, used to acquire a candidate configuration real region corresponding to a sample training image;

[0057] The configuration area acquisition unit is also used to determine the sample prediction area k i The sample coverage area that is commonly covered by the candidate configuration's real area;

[0058] A real area determination unit is used to obtain a sample prediction area k in the candidate configuration real area according to the sample coverage area. i The corresponding real area.

[0059] In one embodiment, the real area determination unit includes:

[0060] The ratio determination subunit is used to configure the candidate real area to the sample prediction area k i The uncovered area is taken as the first sample area, and the sample prediction area k i The area not covered by the real area of ​​the candidate configuration is used as the second sample area;

[0061] The ratio determination subunit is further used to determine the candidate configuration real area and the sample prediction area k according to the first sample area, the second sample area and the sample coverage area. i The sample area ratio s i ;

[0062] The real area determination subunit is used if the sample area ratio s iIf the area ratio is greater than the area ratio threshold, the candidate is configured as the real area and determined as the sample prediction area k i The corresponding real area.

[0063] In one embodiment, the M sample prediction regions also include a sample prediction region k j ; j is a positive integer; sample prediction area k i The sample prediction object category corresponding to the sample object in is based on the sample prediction area k output by the first convolutional network layer. i The sample object existence probability and sample category prediction probability of the sample object in the sample prediction area k are jointly determined; i The probability of existence of a sample object refers to the sample prediction area k i The probability that a sample object exists in the sample prediction area k j The sample prediction object category corresponding to the sample object in is based on the sample prediction area k output by the first convolutional network layer. j The sample object existence probability and sample category prediction probability of the sample object in the sample prediction area k are jointly determined; j The probability of existence of a sample object refers to the sample prediction area k j The probability that a sample object exists in ;

[0064] The loss value generation module includes:

[0065] Sample division unit, used to obtain sample prediction area k j The corresponding sample area ratio s j , the sample area ratio s i Ratio of sample area to j The sample prediction area whose area ratio is greater than or equal to the area ratio threshold is determined as the positive sample area, and the sample prediction area whose area ratio is less than the area ratio threshold is determined as the negative sample area;

[0066] A loss value generating unit, used to generate an initial loss value of the positive sample according to the positive sample area, the real area corresponding to the positive sample area, the real object category corresponding to the real area corresponding to the positive sample area, and the sample prediction object category corresponding to the sample object in the positive sample area;

[0067] The loss value generating unit is further used to generate a positive sample confidence loss value according to the existence probability of the sample object corresponding to the positive sample area;

[0068] The loss value generating unit is further used to generate a negative sample confidence loss value according to the existence probability of the sample object corresponding to the negative sample area;

[0069] The loss value generating unit is also used to add the positive sample initial loss value, the positive sample confidence loss value and the negative sample confidence loss value to obtain a first loss value.

[0070] In one embodiment, the loss value generating unit is further specifically used to determine the regional loss value according to the real region corresponding to the positive sample region and the positive sample region;

[0071] The loss value generating unit is further specifically used to determine the category loss value according to the sample prediction object category of the sample object in the positive sample area and the real object category corresponding to the real area corresponding to the positive sample area;

[0072] The loss value generating unit is further specifically used to add the regional loss value and the category loss value to obtain the initial loss value of the positive sample.

[0073] In one embodiment, the M sample prediction regions include a first sample prediction region and a second sample prediction region; the M sample object label features include a first sample object label feature corresponding to the first sample prediction region and a second sample object label feature corresponding to the second sample prediction region; the first sample prediction region and the second sample prediction region belong to positive sample regions;

[0074] The loss value generating unit is further specifically used to input the first sample object label feature and the second sample object label feature into the second convolutional network layer, perform feature classification on the first sample object label feature and the second sample object label feature through the second convolutional network layer, and obtain a predicted classification result between the first sample object label feature and the second sample object label feature;

[0075] The loss value generating unit is further specifically used to determine the true classification result between the first sample object label feature and the second sample object label feature according to the real object category corresponding to the real area corresponding to the first sample prediction area and the real object category corresponding to the real area corresponding to the second sample prediction area;

[0076] The loss value generating unit is further specifically used to determine the second loss value based on the predicted classification result between the first sample object label feature and the second sample object label feature and the actual classification result between the first sample object label feature and the second sample object label feature.

[0077] An embodiment of the present application provides a computer device, including: a processor and a memory;

[0078] The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the method in the embodiment of the present application.

[0079] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the method in the embodiment of the present application is executed.

[0080] In one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in one aspect of the embodiments of the present application.

[0081] In an embodiment of the present application, after identifying the N prediction areas of the target object in the target image, the prediction object category and the prediction object label features corresponding to the target object in each prediction area can be obtained; based on the category prediction probability corresponding to the prediction object category, the target prediction area can be obtained in the N prediction areas; and when screening the optimal prediction area of ​​the target object in the N prediction areas, the optimal prediction area of ​​the target object in the target image can be jointly determined based on the overlapping area (commonly covered coverage area) between the remaining prediction areas and the target prediction area and the similarity between the prediction object label features. It should be understood that the target object can be understood as an object in the target image, and the prediction area can be understood as the detection frame of the object. When determining the final detection frame of the target object, the present application not only considers the overlapping area between the detection frames, but also considers the similarity between the prediction object label features between the detection frames, and the optimal detection frame determined by the two is more accurate. In summary, the present application can improve the recognition accuracy of objects in images. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0083] Figure 1 This is a network architecture diagram provided by an embodiment of the present application;

[0084] Figure 2a This is a schematic diagram of a scenario for identifying location information of a target object provided by an embodiment of the present application;

[0085] Figure 2b is a schematic diagram of determining the intersection over union (IOU) of overlapping areas provided in an embodiment of the present application;

[0086] Figure 3 is a method flow chart of a data identification method provided by an embodiment of the present application;

[0087] Figure 4 is a schematic diagram of determining an optimal prediction area provided in an embodiment of the present application;

[0088] Figure 5 It is a flowchart of another data identification method provided in an embodiment of the present application;

[0089] Figure 6 It is a schematic diagram of a model training provided in an embodiment of the present application;

[0090] Figure 7 It is a structural schematic diagram of a data identification device provided in an embodiment of the present application;

[0091] Figure 8 is a structural schematic diagram of another data identification device provided in an embodiment of the present application;

[0092] Fig. 9 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0093] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0094] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0095] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0096] The solution provided in the embodiment of the present application belongs to the computer vision technology (Computer Vision, CV) and machine learning (Machine Learning, ML) under the field of artificial intelligence.

[0097] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphics processing so that the computer processes the images into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0098] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0099] See also Figure 1 , Figure 1 This is a network architecture diagram provided by an embodiment of the present application. Figure 1 As shown, the network architecture may include a service server 1000 and a user terminal cluster, wherein the user terminal cluster may include multiple user terminals, such as Figure 1As shown, it may specifically include user terminal 100a, user terminal 100b, user terminal 100c, ..., user terminal 100n. Figure 1 As shown, user terminal 100a, user terminal 100b, user terminal 100c, ..., user terminal 100n can respectively establish a network connection with the business server 1000, so that each user terminal can exchange data with the business server 1000 through the network connection, so that the business server 1000 can receive business data from each user terminal.

[0100] like Figure 1 Each user terminal shown in the figure may be integrated with a target application. When the target application is running in each user terminal, it can be used together with the above Figure 1 Data is exchanged between the business servers 1000 shown. Among them, the target application may include an application with data information functions such as displaying text, images, audio and video. For example, the application may be an image recognition application, which can be used by users to upload pictures, and view the locations of various objects in the picture, and perform subsequent processing according to the locations of these objects. If the image recognition application is a watermark recognition application, it can be used by users to upload pictures and view whether there are sensitive watermarks or illegal watermarks in the pictures, and mark the location of the sensitive watermark or illegal mark when it exists (for example, the location of the sensitive watermark or illegal mark can be framed); the application can also be an image classification application, which can be used by users to upload pictures and view the object category (such as elephants, tigers, larks, etc.) of the objects (such as people, animals, etc.) contained in the pictures; the application can also be a video detection application, which can be used by users to upload videos and detect whether there are illegal marks or sensitive watermarks in the video frames of the videos.

[0101] The business server 1000 in the present application can collect business data from the background of these applications, such as images uploaded by users and videos uploaded by users. Based on the collected business data, the business server 1000 can determine the object category to which the objects contained in these business data belong, as well as the location information (that is, the detection box) of the object in the business data (image or video). Furthermore, the business server 1000 can send the object category and location information to the user terminal, and the user can determine the object category to which the object in the video frame of the image or video belongs through the user terminal, and view the location information of the object in the image or in the video frame of the video, and the user can perform subsequent processing based on the viewed object category and location information. For example, a user uploads video A, and the business server 1000 detects that there is a sensitive watermark at a certain position in the video frame A of the video A. The business server 1000 can return the detection result to the user terminal corresponding to the user terminal of the user, and the user can view the detection result on the display page of the user terminal. Subsequently, the user can delete the sensitive watermark in the video A through the location information (the position of the video frame A in the video A) and the object category (sensitive watermark) returned by the business server 1000 to ensure the legality of the video A.

[0102] like Figure 1 The business server 1000 shown may be a server corresponding to the target application. The business server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, blockchain networks, and big data and artificial intelligence platforms.

[0103] For ease of understanding, the embodiment of the present application can select a user terminal from multiple user terminals as the target user terminal, and the target user terminal can include: smart phones, tablet computers, desktop computers and other smart terminals with data information display and playback functions. Figure 1 The user terminal corresponding to the user terminal 100a shown is used as the target user terminal. The target user terminal may be integrated with the above-mentioned target application. At this time, the user terminal 100a corresponding to the target user terminal may perform data exchange with the service server 1000.

[0104] For example, when a user is using a target application (such as a video detection application) in a user terminal, the business server 1000 can detect and collect that the user has uploaded a video through the user terminal corresponding to the user terminal. The business server 1000 can determine whether there is a target object (such as a person, an animal, a sensitive watermark, etc.) in the video. If the target object exists in the video, the business server 1000 determines the object category to which the target object belongs, as well as the location information of the target object in the video, and returns the object category and location information to the user terminal. The user can then view the object category and location information on the display page of the user terminal corresponding to the user terminal, and perform subsequent processing based on the object category and location information.

[0105] Optionally, it is understood that the user terminal can detect the collected business data (such as images or videos), and determine the object category to which the object contained in the business data belongs, as well as the location information of the object in the business data (images or videos). The user can view the object category and location information determined by the user terminal on the display page of the user terminal.

[0106] It can be understood that the specific implementation method for the service server 1000 or the user terminal to determine the object category to which the target object belongs and the location information of the target object in the image (or video frame) can be found in the subsequent Figure 3 The description in the corresponding embodiment.

[0107] It is understandable that the method provided in the embodiment of the present application can be executed by a computer device, including but not limited to a user terminal or a service server. The user terminal and the service server can be directly or indirectly connected via wired or wireless communication, and the present application does not limit this.

[0108] For easier understanding, please also refer to Figure 2a , Figure 2a is a schematic diagram of a scenario for identifying the location information of a target object provided by an embodiment of the present application. Figure 2a The business server shown may be the business server 1000 described above, and Figure 2a The user terminal A shown in the figure can be Figure 1 Any user terminal selected from the user terminal cluster of the corresponding embodiment, for example, the user terminal may be the user terminal 100a described above.

[0109] like Figure 2aAs shown, user a can upload the original image in the target application of user terminal A, and user terminal A can perform data preprocessing on the original image (such as data enhancement processing and image normalization processing on the original image) to obtain image 200, wherein the image 200 (i.e., the original image) includes objects 20a, 20b, 20c, and 20d. User terminal A can send the image 200 to the business server. Subsequently, after receiving the image 200, the business server can input the image 200 into the image recognition model deployed in the business server, and through the image recognition model, the predicted box (i.e., the predicted area) of each object in the image 200, the predicted object category of the object contained in each prediction box, and the predicted object label feature can be output. Optionally, the user terminal can send the original image to the business server, and the business server can perform data preprocessing on the original image, and input the image after data preprocessing into the image recognition model.

[0110] like Figure 2a As shown, the image recognition model can output the prediction box of each object in the image 200, as well as the object existence probability, category prediction probability, and object label feature corresponding to each prediction box. Among them, the object existence probability can be used to characterize the probability of the existence of an object (object) in the prediction box output by the image recognition model. When the object existence probability is greater than a certain threshold, it can be considered that there is an object in the prediction box, and the category of the object can be further determined by the category prediction probability; and the object label feature can be the feature of the object in the prediction box output by the image recognition model (for example, the object category in a prediction box is a person, the person's hair is red, and the clothes are blue, then "hair is red, clothes are blue" can be used as the object label feature of the prediction box).

[0111] Taking the object 20c in the image 200 as an example, the prediction boxes of the object 20c output by the image recognition model are prediction boxes r1 and prediction boxes r2; at the same time, the image recognition model can also output the object existence probabilities corresponding to the prediction boxes r1 and r2, and can also output the category prediction probabilities and object label features corresponding to the prediction boxes r1 and r2; wherein the object existence probability of the prediction box r1 is 0.9 (greater than the object existence probability threshold of 0.8), and the category prediction probability of the prediction box r1 is (0.6, 0.4), wherein the object category corresponding to the category prediction probability of 0.6 is a person, and the category prediction probability is (0.6, 0.4). The object category corresponding to 0.4 is animal. Since the category prediction probability 0.6 is greater than the category prediction probability 0.4, the predicted object category of the object 20c contained in the prediction box r1 can be determined as a person. The object existence probability of the prediction box r2 is 0.95, and the category prediction probability of the prediction box r2 is (0.8, 0.2), wherein the object category corresponding to the category prediction probability 0.8 is a person, and the object category corresponding to the category prediction probability 0.2 is an animal. Since the category prediction probability 0.8 is greater than the category prediction probability 0.2, the predicted object category of the object 20c contained in the prediction box r2 can be determined as a person. Similarly, it can be determined that in image 200, the predicted object category of object 20a contained in prediction box c1 is vehicle (its corresponding category prediction probability is 0.7), and the predicted object category of object 20a contained in prediction box c2 is vehicle (its corresponding category prediction probability is 0.8); it can be determined that in image 200, the predicted object category of prediction box m1 is animal (its corresponding category prediction probability is 0.75), and the predicted object category of prediction box m2 is animal (its corresponding category prediction probability is 0.85); it can be determined that in image 200, the predicted object category of prediction box h1 is person (its object category prediction probability is 0.7), and the predicted object category of prediction box h2 is person (its object category prediction probability is 0.75).

[0112] Furthermore, the business server may perform tag-based non-maximum suppression (Tag NMS) on the prediction boxes output by the image recognition model, thereby determining the optimal prediction box (that is, the optimal prediction area) corresponding to each object in the image 200. The specific method may be: the business server may determine the prediction object category of the object contained in each prediction box (hereinafter referred to as the prediction object category corresponding to each prediction box), and classify the prediction boxes. For example, Figure 2aIn the figure, the predicted object categories of prediction box c1 and prediction box c2 are both vehicles, so prediction box c1 and prediction box c2 can be classified into one category; the predicted object categories of prediction box r1, prediction box r2, prediction box h1, prediction box h2 are all people, so prediction box r1, prediction box r2, prediction box h1, prediction box h2 can be classified into one category; the predicted object categories of prediction box m1 and prediction box m2 are both animals, so prediction box m1 and prediction box m2 can be classified into one category. Subsequently, the business server can sort the prediction boxes according to the category prediction probabilities corresponding to the prediction boxes in each category, and perform Tag NMS processing based on the sorted results to obtain the optimal prediction box for each object.

[0113] The following will take the prediction box r1, prediction box r2, prediction box h1, and prediction box h2, all of which are predicted object categories of people, as an example to perform Tag NMS processing on the prediction box and obtain the specific method of the optimal prediction box for each object. First, the category prediction probabilities corresponding to the prediction box r1, prediction box r2, prediction box h1, and prediction box h2 can be obtained, and they can be sorted according to the category prediction probabilities. Because the prediction object category "person" corresponding to the prediction box r1 is determined by the category prediction probability of 0.6, the category prediction probability corresponding to the prediction box r1 can be 0.6; similarly, it can be determined that the category prediction probability corresponding to the prediction box r2 is 0.8, the category prediction probability corresponding to the prediction box h1 is 0.7, and the category prediction probability corresponding to the prediction box h2 is 0.75; then the prediction box r1, prediction box r2, prediction box h1, and prediction box h2 can be sorted in order from large to small according to their category prediction probabilities, and the sorted sequence is {prediction box r2, prediction box h2, prediction box h1, prediction box r1}.

[0114] Furthermore, the prediction box r2 located at the first sequence position in the sequence {prediction box r2, prediction box h2, prediction box h1, prediction box r1} (that is, the prediction box whose category prediction probability is the maximum category prediction probability) can be used as the target prediction box (that is, the target prediction area); then, the remaining prediction boxes (including prediction box h2, prediction box h1, prediction box r1) can be compared with the target prediction area to determine the overlapping area between them and the target prediction box r2 (that is, the area of ​​the overlapping area); then, the object label features corresponding to the prediction box r2, prediction box h2, prediction box h1, and prediction box r1 can be obtained, and the label feature similarity between the prediction box h2 and the prediction box r2, the label feature similarity between the prediction box h1 and the prediction box r2, and the label feature similarity between the prediction box r1 and the prediction box r2 can be calculated. Among them, the specific implementation method of calculating the label feature similarity can be found in the subsequent Figure 3 The description in the corresponding embodiment.

[0115] Then, the intersection over union (IOU) between the two boxes can be determined based on the overlapping area between any two boxes (between any remaining prediction box and the target prediction box). To understand the IOU between two boxes, please refer to Figure 2b , Figure 2b Schematic diagram of an overlap area intersection-over-union (IOU) ratio provided in an embodiment of the present application. Figure 2b As shown, the prediction box may include a prediction box 2000a and a prediction box 2000b, and there is an overlapping area 2000 between the prediction box 2000a and the prediction box 2000b. The area corresponding to the overlapping area 2000 can be understood as the overlapping area between the prediction box 2000a and the prediction box 2000b; the area A' in the prediction box 2000a can be understood as the part that is not covered by the prediction box 2000b except the area 2000; similarly, the area B' in the prediction box 20000b can be understood as the part that is not covered by the prediction box 2000a except the area 2000. The union result between area A' and area B' can be obtained (such as adding area A' and area B' to obtain A'∪B'), and the ratio between the overlapping area (area 2000, that is, A'∩B') and the union result (that is, ), determined as the IOU between prediction box 2000a and prediction box 2000b.

[0116] It should be understood that whether to retain the remaining prediction boxes can be determined based on the IOU between every two boxes (between any remaining prediction box and the target prediction box) and the label feature similarity (only when the IOU between a remaining prediction box and the target prediction box is greater than the ratio threshold and the label feature similarity is also greater than the similarity threshold, the remaining prediction box will be deleted; otherwise it will be retained), and then the optimal prediction box for each object in the image 200 is determined by the retained prediction box. For example, taking the prediction box r1 and the prediction box r2 as an example, it can be determined that the IOU between the prediction box r1 and the prediction box r2 is 0.9, and the IOU is greater than the ratio threshold (such as 0.67); the label feature similarity between the prediction box r1 and the prediction box r2 can be obtained as 0.98, and the label feature similarity is greater than the similarity threshold (0.5). Because the IOU is greater than the ratio threshold and the label feature similarity is also greater than the similarity threshold, the prediction box r1 can be deleted; similarly, taking the prediction box h2 and the prediction box r2 as an example, it can be determined that the IOU between the prediction box h2 and the prediction box r2 is 0, and the IOU is less than the ratio threshold (such as 0.67); the label feature similarity between the prediction box h2 and the prediction box r2 can be obtained as 0.23, and the label feature similarity is less than the similarity threshold (0.5); because the IOU is less than the ratio threshold and the label feature similarity is also less than the similarity threshold, the prediction box h2 can be retained. Similarly, using the above method, the prediction box h1 can also be retained.

[0117] It should be understood that based on the intersection over union (IOU) of overlapping areas and the similarity of label features, the prediction box r2 in the sequence {prediction box r2, prediction box h2, prediction box h1, prediction box r1} can be deleted, and the deleted sequence is {prediction box r2, prediction box h2, prediction box h1}. Further, a new sequence, namely {prediction box h2, prediction box h1}, can be formed based on the remaining prediction boxes (that is, the remaining prediction area) other than the target prediction box (that is, the target prediction area), and then the prediction box h2 in the new sequence {prediction box h2, prediction box h1} is used as the new target prediction box (that is, the new target prediction area), and the prediction box h1 is used as the new remaining prediction box; similarly, the same method as above can be used to determine whether to retain the prediction box h1 based on the IOU and label feature similarity between the remaining prediction box h1 and the target prediction box h2. For example, it can be determined that the IOU between the prediction box h1 and the prediction box h2 is 0.89, which is greater than the ratio threshold (such as 0.67); the label feature similarity between the prediction box h1 and the prediction box h2 can be obtained as 0.98, which is greater than the similarity threshold (0.5). Because the IOU is greater than the ratio threshold and the label feature similarity is also greater than the similarity threshold, the prediction box h1 can be deleted.

[0118] It should be understood that through the above Tag NMS process of {prediction box r2, prediction box h2, prediction box h1, prediction box r1}, the prediction boxes finally retained are prediction box r2 and prediction box h2, where prediction box r2 is the position of object 20c in the image 200; prediction box h2 is the position of object 20d in the image 200. Similarly, prediction boxes belonging to the same prediction object category in prediction box c1, prediction box c2, prediction box m1, and prediction box m2 can be classified into one category, sorted according to the category prediction probability, and then Tag NMS processing is performed on the sorted prediction boxes. Figure 2a As shown, the prediction boxes finally retained include a prediction box c2 and a prediction box m2. The prediction box c2 is the position of the object 20a in the image 200, and the prediction box m2 is the position of the object 20b in the image 200.

[0119] Furthermore, the service server may return the recognition result (the optimal predicted area of ​​each object in the image 200) after the Tag NMS processing to the user terminal A, and the user a may view the recognition result in the display interface of the user terminal A and perform subsequent processing.

[0120] It should be understood that when performing non-maximum suppression on the prediction frames, if the distance between the image objects is relatively close, the overlap area between the two prediction frames is relatively large, and the overlap area intersection-over-union ratio will also be relatively large. If only the overlap area intersection-over-union ratio is considered for non-maximum suppression, the prediction frames that should not be suppressed (such as deleted) will be deleted, resulting in the problem of erroneous suppression. For example, Figure 2a As shown, taking object 20c and object 20d as an example, if the two objects are very close in position in image 200, then in prediction box r1, prediction box r2, prediction box h1, and prediction box h2, there are many overlapping parts between any two prediction boxes, and the overlapping area will be large. Then, after sorting the sequence {prediction box r2, prediction box h2, prediction box h1, prediction box r1} according to the category prediction probability, when performing non-maximum suppression processing, if only the intersection-over-union ratio of overlapping areas is considered, then because the overlap between prediction box h1, prediction box h2 and prediction box r2 is The areas are both large, and their overlapping area intersection-to-union ratios are also greater than the ratio threshold, so the prediction boxes h1 and h2 will be suppressed (such as deleted). It can be seen that because the prediction boxes h1 and h2 are the positions of the object 20d in the image 200, they should not be deleted. That is to say, during the non-maximum suppression processing, only the overlapping area intersection-to-union ratio is considered, which will cause the prediction boxes h1 and h2 to be mistakenly deleted, resulting in the absence of a prediction box for the object 20d, causing the position of the object 20d in the image 200 to not be recognized in the final recognition result.

[0121] When performing non-maximum suppression processing on the prediction frames, the present application not only considers the overlap area intersection-over-union ratio between the prediction frames, but also considers the label feature similarity of the object label features between the prediction frames. The prediction frame will be suppressed only when the overlap area intersection-over-union ratio is greater than the ratio threshold and the label feature similarity is also greater than the similarity threshold. Figure 2a As shown, taking the prediction box h2 and the prediction box r2 as an example, if the objects 20c and 20d are very close in position in the image 200, then the prediction box h2 and the prediction box r2 have more overlap and a larger overlap area, but because the prediction box h2 and the prediction box r2 correspond to different objects, the object 20d in the prediction box h2 and the object 20c in the prediction box r2 have different object label features (for example, the object 20c has long red hair and red clothes, while the object 20d has short black hair and white clothes), then the label feature similarity between the prediction box h2 and the prediction box r2 will be very low, then even if the intersection-and-union ratio of the overlapping areas between the prediction box h2 and the prediction box r2 is greater than the ratio threshold, but because the label feature similarity is lower than the similarity threshold, when the prediction box r2 is used as the target prediction box, the prediction box h2 will not be deleted, but will be temporarily retained. It can be seen that when performing non-maximum suppression processing on the prediction box, the present application adopts a multi-dimensional angle based on the intersection-overlapping area ratio and the similarity of label features, which can improve the accuracy of the image recognition results.

[0122] For further information, see Figure 3 , Figure 3 is a method flow chart of a data identification method provided by an embodiment of the present application. The method can be performed by a user terminal (for example, the above Figure 1 Any user terminal in the user terminal cluster shown in the figure, such as user terminal 100b), can also be executed by a service server (for example, the above Figure 1 The method may be executed by the service server 1000 shown in the figure, or may be executed by the user terminal and the service server together. For ease of understanding, this embodiment takes the method executed by the above service server as an example for explanation. The method may at least include the following steps S101-S106:

[0123] Step S101, acquiring a target image, and identifying N prediction regions of a target object in the target image; N is a positive integer.

[0124] In this application, the target image may refer to an image after data enhancement and image normalization. It can be understood that after obtaining an original image, data enhancement and image normalization processing can be performed on it, and the original image after data enhancement and image normalization processing can be called the target image; the target image may include one or more objects (such as vehicles, people, birds, elephants, monkeys, buildings, watermarks, drinks, etc.), and the one or more objects can be called one or more objects, and any object in the one or more objects can be called a target object.

[0125] It can be understood that the position of the target object in the target image can be referred to as the region, and the present application can input the target image into the image recognition model, and the image recognition model can identify N prediction regions of the target object in the target image. The specific method can be: the target image can be input into the image recognition model, and the image recognition model can output the candidate prediction region corresponding to the target image, the object existence probability corresponding to the candidate prediction region, and the initial category prediction probability corresponding to the candidate prediction region; wherein the object existence probability can refer to the probability of the object existing in the candidate prediction region; according to the object existence probability and the initial category prediction probability corresponding to the candidate prediction region, the N prediction regions of the target object in the target image can be obtained in the candidate prediction region.

[0126] Among them, for the initial category prediction probability corresponding to the candidate prediction area according to the object existence probability, a specific method for obtaining N prediction areas of the target object in the target image in the candidate prediction area can be: the candidate prediction area with the object existence probability greater than the probability threshold can be determined as the prediction area to be identified; the maximum initial category prediction probability can be obtained from the initial category prediction probability corresponding to the prediction area to be identified, and the object category corresponding to the maximum initial category prediction probability can be determined as the predicted object category of the object contained in the prediction area to be identified; according to the predicted object category of the object contained in the prediction area to be identified, the prediction area to be identified can be regionally classified to obtain M area sets; wherein the predicted object categories of the objects contained in the prediction area to be identified included in one area set belong to the same object category; subsequently, a target area set can be obtained from the M area sets, the object contained in the prediction area to be identified included in the target area set can be determined as the target object, and the prediction area to be identified included in the target area set can be determined as the N prediction areas of the target object in the target image.

[0127] In addition, after outputting the candidate prediction regions, the image recognition model in the present application may also output the predicted object label features corresponding to each candidate prediction region, and the predicted object label features may refer to the object features of the objects contained in each candidate prediction region, and the object features may include the features of the objects presented in the target image (e.g., hair color features, hair length features, facial features (e.g., eye size features, pupil color features, mouth thickness features, mouth size features, nose height features), clothing color features, clothing length features, etc.). The specific method for the image recognition model to output the predicted object label features corresponding to the candidate prediction regions may be: the regional features of the candidate prediction regions may be extracted through the image recognition model; then, the regional features may be subjected to feature extraction, thereby obtaining the predicted object label features corresponding to the candidate prediction regions; wherein, the predicted object label features corresponding to the candidate prediction regions include the predicted object label features corresponding to the N prediction regions, respectively.

[0128] To facilitate understanding of the specific process of identifying the N predicted regions of the target object in the target image, the following will combine the above Figure 2a The corresponding embodiments are described as follows. Figure 2aAs shown, image 200 can be used as a target image. After the business server inputs the image 200 into the image recognition model, the image recognition model can output prediction box c1, prediction box c2, prediction box r1, prediction box r2, prediction box h1, prediction box h2, prediction box m1, prediction box m2. Prediction box c1, prediction box c2, prediction box r1, prediction box r2, prediction box h1, prediction box h2, prediction box m1, prediction box m2 can all be referred to as candidate prediction areas. In addition, the image recognition model can also output the object existence probability, category prediction probability and object label features corresponding to each prediction box. The prediction box whose object existence probability is greater than the probability threshold can be referred to as a prediction area to be identified. For example, if the object existence probabilities of the above prediction boxes c1, prediction box c2, prediction box r1, prediction box r2, prediction box h1, prediction box h2, prediction box m1, prediction box m2, prediction box c1, prediction box c2, prediction box r1, prediction box r2, prediction box h1, prediction box h2, prediction box m1, prediction box m2 are all greater than the probability threshold, then these prediction boxes can be referred to as prediction areas to be identified, and the corresponding prediction object category can be further determined according to the category prediction probability of each prediction area to be identified. Furthermore, in the prediction area to be identified, the prediction boxes belonging to the same prediction object category can be classified into one category, so that multiple area sets can be obtained. For example, the prediction boxes c1 and the prediction box c2 belonging to the object category of "vehicle" can be used as a region set, the prediction boxes m1 and the prediction box m2 belonging to the object category of "animal" can be used as a region set, and the prediction boxes r1, the prediction box r2, the prediction box h1, and the prediction box h2 belonging to the object category of "person" can be used as a region set. It should be understood that the objects contained in any area set can be determined as target objects, and this area set can be called the N prediction areas of the target object; for example, the objects 20c and 20d contained in the area set composed of prediction box r1, prediction box r2, prediction box h1, and prediction box h2 can be determined as target objects, then the prediction box r1, prediction box r2, prediction box h1, and prediction box h2 can be determined as the N prediction areas of the target object (including object 20c and object 20d).

[0129] Optionally, after obtaining the initial category prediction probability corresponding to each prediction box, when performing region division, the region division is performed according to the initial category prediction probability to obtain different region sets. Figure 2aThe initial category prediction probability of the prediction frame m1 shown in the figure is (0.65, 0.35), wherein the object category corresponding to the initial category prediction probability of 0.65 is "animal", and the object category corresponding to the initial category prediction probability of 0.35 is "person". When performing region division, the prediction frames that can be obtained with the same object category of "person" include prediction frame r1, prediction frame r2, prediction frame h1, prediction frame h2, prediction frame m1, and prediction frame m2. In other words, prediction frame r1, prediction frame r2, prediction frame h1, prediction frame h2, prediction frame m1, and prediction frame m2 can form a region set {prediction frame r1, prediction frame r2, prediction frame h1, prediction frame h2, prediction frame m1, prediction frame m2}, and the category prediction probabilities of each prediction frame in the region set for the object category of "person" are different from each other. When the regions are divided according to the category prediction probability of the object category "animal", a region set of {prediction box r1, prediction box r2, prediction box h1, prediction box h2, prediction box m1, prediction box m2} can also be obtained. The category prediction probabilities of each prediction box for the object category "animal" are different.

[0130] Step S102: Obtain the predicted object category and predicted object label features corresponding to the target object in each prediction area.

[0131] In the present application, because the image recognition model can output the predicted object category and predicted object label features corresponding to each candidate prediction area, and because the candidate prediction area includes N prediction areas, each prediction area in the N areas also corresponds to a predicted object category and a predicted object label features, the predicted object category and predicted object label features corresponding to each prediction area can be obtained.

[0132] Step S103 , obtaining the maximum category prediction probability among the category prediction probabilities corresponding to the N prediction object categories, and determining the prediction area corresponding to the prediction object category having the maximum category prediction probability as the target prediction area.

[0133] In the present application, because the predicted object category of the target object contained in each prediction area is determined based on the initial category prediction probability output by the image recognition model, the specific method is to determine the object category corresponding to the maximum initial category prediction probability in the initial category prediction probability output by the image recognition model as the predicted object category corresponding to the prediction area, and the category prediction probability corresponding to the predicted object category can be understood as the maximum initial category prediction probability. For example, the initial category prediction probability corresponding to the prediction box u1 is (0.9, 0.1), the object category corresponding to the initial category prediction probability 0.9 is "flower", and the object category corresponding to the initial category prediction probability 0.1 is "tree". Because the initial category prediction probability 0.9 is greater than the initial category prediction probability 0.1, it can be determined that the predicted object category corresponding to the prediction box u1 is "flower", and at the same time, the maximum initial category prediction probability 0.9 can be used as the category prediction probability of the predicted object category "flower".

[0134] It should be understood that the present application can obtain the category prediction probability corresponding to the prediction object category of each prediction area in N prediction areas, and sort the N prediction areas according to the order of the size of the category prediction probability. After sorting, the prediction area corresponding to the maximum category prediction probability can be obtained, and the prediction area corresponding to the maximum category prediction probability can be called the target prediction area. For example, the N prediction areas can be sorted in the order of category prediction probability from large to small. In the sequence obtained by sorting, the prediction area at the first sequence position is the prediction area corresponding to the maximum category prediction probability, and the prediction area at the first sequence position can be called the target prediction area.

[0135] Step S104, determining the coverage area jointly covered by the remaining prediction area and the target prediction area; the remaining prediction area is the prediction area other than the target prediction area among the N prediction areas.

[0136] In the present application, a coverage area commonly covered by each remaining prediction area and the target prediction area may be determined.

[0137] Step S105 , determining the label feature similarity between the predicted object label feature corresponding to the remaining prediction region and the predicted object label feature corresponding to the target prediction region.

[0138] In this application, the predicted object label features corresponding to each remaining prediction region can be obtained, and the predicted object label of the target prediction region can be obtained, and then the label feature similarity between each remaining prediction region and the target prediction region can be determined. a (a is a positive integer) as an example, the specific method for determining the similarity of label features can be: the remaining prediction area s can be obtained aThe corresponding predicted object label feature (which may be referred to as the first predicted object label feature) can be used to obtain the predicted object label feature corresponding to the target prediction area (which may be referred to as the second predicted object label feature); further, the first feature vector corresponding to the first predicted object label feature can be obtained, and the second feature vector corresponding to the second predicted object label feature can be obtained; the vector angle value between the first feature vector and the second feature vector can be determined, and the cosine similarity corresponding to the vector angle value can be determined, and the cosine similarity can be determined as the label feature similarity between the first feature vector and the second feature vector.

[0139] To facilitate understanding of the specific method of calculating the cosine similarity between the label features of two predicted objects, please refer to the following formula (1). The cosine similarity between the label features of two predicted objects can be calculated as shown in formula (1):

[0140]

[0141] Among them, as shown in formula 1, A and B can be two different prediction object label features, and A i It can be understood as the i-th predicted object label sub-feature included in A; B i It can be understood as the i-th predicted object label feature sub-feature included in B. For example, the object contained in the prediction box u1 is object 1, and its predicted object label features include {long red hair, blue top}, and the object contained in the prediction box u2 is object 2, and its predicted object label features include {short black hair, red top}. Then the "long red hair" and "blue top" can both be called predicted object label sub-features (corresponding to A in formula (1) i ), and the predicted object label sub-features "red long hair" and "blue top" together constitute the predicted object label feature {red long hair, blue top} corresponding to the prediction box u1; similarly, "black short hair" and "red top" can both be called predicted object label sub-features (corresponding to B in formula (2) i , and the predicted object label sub-features "black short hair" and "red top" together constitute the predicted object label features {red long hair, blue top} corresponding to the prediction box u2.

[0142] Optionally, a specific method for determining the similarity of label features may also be: the remaining prediction area s may be obtained aThe corresponding predicted object label feature (which may be referred to as the first predicted object label feature) can be used to obtain the predicted object label feature corresponding to the target prediction area (which may be referred to as the second predicted object label feature); further, the first feature vector corresponding to the first predicted object label feature can be obtained, and the second feature vector corresponding to the second predicted object label feature can be obtained; the vector distance (such as the Euclidean distance) between the first feature vector and the second feature vector can be determined, and the vector distance can be determined as the label feature similarity between the first feature vector and the second feature vector.

[0143] Step S106, determining the optimal prediction region of the target object in the target image from among the N prediction regions according to the similarity between the coverage region and the label feature.

[0144] In the present application, according to the similarity between the covered area and the label feature, the specific method for determining the optimal prediction area of ​​the target object in the target image among N prediction areas may be as follows: the area not covered by the remaining prediction area in the target prediction area may be used as the first area to be calculated; the area not covered by the target prediction area in the remaining prediction area may be used as the second area to be calculated; then, the area corresponding to the first area to be calculated and the area corresponding to the second area to be calculated may be added to obtain the area of ​​the first area; the area of ​​the second area corresponding to the covered area may be obtained, and then the area ratio between the area of ​​the second area and the area of ​​the first area may be determined; according to the area ratio and the similarity between the label features, the optimal prediction area of ​​the target object in the target image may be determined among the N prediction areas.

[0145] Among them, the specific method for determining the optimal prediction area of ​​the target object in the target image among N prediction areas based on the area ratio and the label feature similarity may be: when the area ratio is greater than or equal to the ratio threshold, and the label feature similarity is greater than or equal to the similarity threshold, the target prediction area among the N prediction areas may be determined as the optimal prediction area of ​​the target object in the target image; and when the area ratio is less than the ratio threshold, or the label feature similarity is less than the similarity threshold, the remaining prediction areas among the N prediction areas and the target prediction area may both be determined as the optimal prediction areas of the target object in the target image.

[0146] It should be understood that the above-mentioned area ratio can be understood as the overlapping area intersection-and-union ratio. After determining the target prediction area, the overlapping area intersection-and-union ratio between the remaining prediction area and the target prediction area and the label feature similarity can be used to jointly determine whether to delete the remaining prediction area. Only when the area ratio satisfies or is greater than or equal to the ratio threshold and the label feature similarity satisfies or is greater than or equal to the similarity threshold, the corresponding remaining prediction area will be deleted, otherwise it will be retained.

[0147] It should be understood that the specific process of performing non-maximum suppression on the prediction boxes according to the intersection-overlap ratio of the overlapping areas between the prediction boxes and the similarity of the label features to obtain the optimal prediction area can be divided into the following steps 1 to 3:

[0148] Step 1: Sort the N prediction regions in order of the category prediction probabilities corresponding to the predicted object categories, and select the prediction box corresponding to the maximum category prediction probability in the sequence as the target prediction region.

[0149] Step 2: Traverse the remaining prediction regions except the target prediction region in the sequence. If the overlap area intersection-over-union ratio between a remaining prediction region and the current target prediction region is greater than or equal to a certain threshold (ratio threshold), and the label feature similarity is greater than or equal to another threshold (such as similarity threshold) at the same time, then delete the remaining prediction region (the reason for deletion is that when the overlap area intersection-over-union ratio exceeds the threshold and the label feature similarity also exceeds the threshold, it can be considered that the objects contained in the two prediction regions belong to the same object category and belong to the same object, then only the region with a higher category prediction probability (i.e., the target prediction region) needs to be left in the two prediction regions).

[0150] Step 3: After one traversal is completed, select a prediction box corresponding to the maximum category prediction probability from the at least two remaining prediction regions as the new target prediction region. Repeat steps 2 to 3 until all prediction regions are processed.

[0151] It should be understood that the above prediction region is understood as a prediction box. To facilitate understanding, according to the intersection over union (IOU) of the overlapping areas between the prediction boxes and the similarity of the label features, the prediction box (prediction region) is subjected to non-maximum suppression processing, so as to obtain the specific process of the optimal prediction region. Please refer to Figure 4 , Figure 4 It is a schematic diagram of determining an optimal prediction area provided in an embodiment of the present application.

[0152] like Figure 4As shown, the image 4001 may include a prediction box t1, a prediction box t2, a prediction box t3 and a prediction box t4; wherein, the prediction object categories of the prediction box t1, the prediction box t2, the prediction box t3 and the prediction box t4 in the image 4001 may all be "puppy", and the category prediction probability of the prediction box t1 for the prediction object category of puppy is 0.9, the category prediction probability of the prediction box t2 for the prediction object category of puppy is 0.8, the category prediction probability of the prediction box t3 for the prediction object category of puppy is 0.7, and the category prediction probability of the prediction box t4 for the prediction object category of puppy is 0.6. Further, it should be understood that the prediction box t1, prediction box t2, prediction box t3 and prediction box t4 can be sorted in the order of the category prediction probability of the predicted object category for the puppy, and the sorted sequence can be {prediction box t1, prediction box t2, prediction box t3, prediction box t4}. It should be understood that because the category prediction probability of the prediction box t1 is the maximum category prediction probability, the prediction box t1 can be used as the target prediction box (target prediction area), and the prediction box t2, prediction box t3 and prediction box t4 can all be called the remaining prediction boxes (remaining prediction areas).

[0153] Furthermore, if Figure 4 As shown, the remaining prediction boxes (including prediction box t2, prediction box t3 and prediction box t4) can be traversed, and the intersection-over-union ratio (IOU1) of the overlapping area between prediction box t2 and prediction box t1 is calculated to be 0.6, and the label feature similarity (similarity 1) is 0.7; the overlapping area (IOU2) between prediction box t3 and prediction box t1 is determined to be 0.4, and the label feature similarity (similarity 2) is 0.5; the overlapping area (IOU3) between prediction box t4 and prediction box t1 is determined to be 0.2, and the label feature similarity (similarity 3) is 0.8; taking the ratio threshold of 0.5 and the similarity threshold of 0.5 as an example, because among these IOUs and label feature similarities, only IOU1 is greater than the ratio threshold and the similarity 1 is also greater than the similarity threshold, and the rest do not meet the conditions, then the prediction box t2 can be deleted, and the prediction box t3 and the prediction box t4 can be temporarily retained. Thus, an image 4002 is obtained, in which the prediction box t2 has been deleted.

[0154] Furthermore, if Figure 4As shown, the prediction box corresponding to the maximum category prediction probability can be selected from prediction box t3 and prediction box t4. The prediction box is prediction box t3, and the prediction box t3 can be used as the new target prediction box. The overlap area intersection-over-union ratio (IOU4) between prediction box t3 and prediction box t4 can be calculated to be 0.7, and the label feature similarity (similarity 4) is 0.7. Because IOU4 is greater than the ratio threshold of 0.5, and the label feature similarity is also greater than the similarity threshold of 0.7, the prediction box t4 can be deleted. At this time, all prediction boxes are processed, so that the retained prediction boxes include prediction box t1 and prediction box t3, and thus image 4003 can be obtained. In image 4003, after the prediction box is subjected to non-maximum suppression processing, the corresponding optimal prediction boxes for the object category of the predicted object category "puppy" include prediction box t1 and prediction box t3.

[0155] In an embodiment of the present application, after identifying the N prediction areas of the target object in the target image, the prediction object category and the prediction object label features corresponding to the target object in each prediction area can be obtained; based on the category prediction probability corresponding to the prediction object category, the target prediction area can be obtained in the N prediction areas; and when screening the optimal prediction area of ​​the target object in the N prediction areas, the optimal prediction area of ​​the target object in the target image can be jointly determined based on the overlapping area (commonly covered coverage area) between the remaining prediction areas and the target prediction area and the similarity between the prediction object label features. It should be understood that the target object can be understood as an object in the target image, and the prediction area can be understood as the detection frame of the object. When determining the final detection frame of the target object, the present application not only considers the overlapping area between the detection frames, but also considers the similarity between the prediction object label features between the detection frames, and the optimal detection frame determined by the two is more accurate. In summary, the present application can improve the recognition accuracy of objects in images.

[0156] It should be understood that the image recognition model can output N predicted areas of the target object in the target image, predicted object categories (the image recognition model can output the object existence probability corresponding to each prediction area, the category prediction probability corresponding to each prediction area, and based on the object existence probability corresponding to each prediction area, the category prediction probability corresponding to each prediction area, the corresponding predicted object category can be determined), and the predicted object label features corresponding to each prediction area. In order to improve the recognition accuracy of the image recognition model, the present application makes the N predicted areas, predicted object categories, and predicted object label features output by the image recognition model more accurate, and the image recognition model can be trained. The specific process of model training can be referred to in the subsequent Figure 5 The description in the corresponding embodiment.

[0157] See also Figure 5 , Figure 5 is a flow chart of another data identification method provided by an embodiment of the present application. The data identification method may be a model training method, which may be performed by a user terminal (for example, the above Figure 1 Any user terminal in the user terminal cluster shown in the figure, such as user terminal 100b), can also be executed by a service server (for example, the above Figure 1 The method may be executed by the service server 1000 shown in the figure, or may be executed by the user terminal and the service server together. For ease of understanding, this embodiment takes the method executed by the above service server as an example for explanation. The method may at least include the following steps S201-S203:

[0158] Step S201, obtain a sample training image, input the sample training image into a sample image recognition model, and output M sample prediction areas of the sample object in the sample training image, the sample prediction object category corresponding to the sample object in each sample prediction area, and the sample object label features corresponding to the sample object in each sample prediction area through the first convolutional network layer in the sample image recognition model; M is a positive integer.

[0159] In the present application, the sample image recognition model can be a parameter-initialized image recognition model that has not been trained. The sample image recognition model can be a convolutional neural network (CNN) model, a yolo (You Only Look Once, yolo) model, or other models that can perform image recognition. Examples are not given one by one here. It can be understood that the sample training image is input into the sample image recognition model, and the sample prediction area of ​​the sample object in the sample training image can be output through the first convolutional network layer in the sample image recognition model, the sample object existence probability corresponding to the sample prediction area (referring to the probability that the sample object exists in the sample prediction area), the sample category prediction probability, and the sample object label feature; and the sample object existence probability and the sample category prediction probability of the sample prediction area can be jointly determined.

[0160] It should be noted that training the sample image recognition model is to train the first convolutional network layer. Figure 3 In the corresponding embodiment, the process of outputting the predicted object label features corresponding to each candidate prediction region can be output by the trained first convolutional network layer.

[0161] Step S202: input the M sample object label features into the second convolutional network layer in the sample image recognition model, and output the predicted classification result between every two sample object label features among the M sample object label features through the second convolutional network layer.

[0162] In the present application, the sample image recognition model also includes a second convolutional network layer, which can classify any two sample object label features. Classification is to determine whether the two sample object label features belong to the same object.

[0163] Step S203, obtaining the real area corresponding to each sample prediction area, and obtaining the real object category corresponding to each real area.

[0164] In this application, each sample training image will correspond to one or more different candidate configuration real regions (the real region of each object in each sample training image can be manually annotated, so each sample training image may include one or more annotated real regions), and the real region corresponding to each sample prediction region can be obtained in the candidate configuration real region; and the object category corresponding to the real region can be used as the real object category corresponding to the sample prediction region. i (i can be a positive integer) as an example, a specific method for obtaining the true area of ​​each sample prediction area is described, and the specific method can be: the candidate configuration true area corresponding to the sample training image can be obtained; then, the sample prediction area k can be determined i The sample coverage area that is commonly covered by the candidate configuration real area; according to the sample coverage area, the sample prediction area k can be obtained in the candidate configuration real area i The corresponding real area.

[0165] Among them, according to the sample coverage area, the sample prediction area k is obtained in the candidate configuration real area i The specific method of the corresponding real area can be: the candidate configuration real area sample prediction area k i The uncovered area is taken as the first sample area, and the sample prediction area k i The area not covered by the candidate configuration real area is used as the second sample area; according to the first sample area, the second sample area and the sample coverage area, the candidate configuration real area and the sample prediction area k can be determined i The sample area ratio s i ; If the sample area ratio s i If the area ratio is greater than the area ratio threshold, the candidate configuration true region can be determined as the true region of the sample object in the sample training image.

[0166] Step S204, determining a first loss value according to the M sample prediction regions, the real regions corresponding to each sample prediction region, the real object categories corresponding to each real region, and the sample prediction object categories corresponding to the sample objects in each sample prediction region.

[0167] In this application, the sample prediction area can be understood as the sample prediction box, the candidate configuration real area can be understood as the real box, and the sample training image corresponds to one or more real boxes. After the sample image recognition model outputs all the sample prediction boxes of the sample training image (all sample prediction boxes, including the sample prediction boxes of the sample objects in the sample training image), the overlapping area intersection-over-union (IOU) calculation can be performed between any sample prediction box in all the sample prediction boxes and all the real boxes. If there is a real box in all the real boxes whose IOU with the sample prediction box is greater than or equal to a certain threshold (such as an area ratio threshold), the sample prediction box can be determined as a positive sample, and the real box with an IOU greater than or equal to a certain threshold is determined as the real box (i.e., the real area) corresponding to the sample prediction box.

[0168] Optionally, it can be understood that if there are two or more real boxes among all the real boxes, and the IOU between them and the sample prediction box is greater than or equal to a certain threshold, then the real box with the largest IOU can be obtained among these two or more real boxes, and the real box with the largest IOU is determined as the real box label corresponding to the sample prediction box, and at the same time, the sample prediction box is determined as a positive sample. For example, all real boxes include real box 1, real box 2 and real box 3. Taking any sample prediction area among all sample prediction areas as the sample prediction area (sample prediction box) of the sample object in the sample training image as an example, it can be determined that the IOU between the sample prediction box and the real box 1 is 0.87, the IOU between the sample prediction box and the real box 2 is 0.88, and the IOU between the sample prediction box and the real box 2 is 0.2; the area ratio threshold is 0.6, because 0.87 and 0.88 are both greater than the area ratio threshold 0.6, the sample prediction box can be determined as a positive sample; at the same time, because 0.88 is greater than 0.87, the real box 2 can be determined as the real box label (real area) corresponding to the sample prediction box.

[0169] Similarly, if the IOU between all real boxes and a sample prediction box is less than the area ratio threshold, the sample prediction box can be determined as a negative sample, and there is no real box label (real area) corresponding to the negative sample in all real boxes.

[0170] It should be understood that, through the above description of determining the corresponding real area for each sample prediction area in the candidate configuration real area, all sample prediction areas output by the first convolutional network layer can be divided into positive sample areas and negative sample areas. The positive sample areas may correspond to real areas, while the negative sample areas represent the areas with prediction errors output by the first convolutional network layer, which do not correspond to real areas. And because the sample prediction area of ​​the sample object in the sample training image is included in all sample prediction areas, the sample prediction area of ​​the sample object in the sample training image will also be divided into positive sample areas and negative sample areas.

[0171] In the present application, for the positive sample area, it is expected that the sample image recognition model can predict closer to its corresponding real area, so it is expected to have a higher prediction confidence and a more accurate sample prediction box. At the same time, the sample prediction object category corresponding to the sample prediction box is also expected to be consistent with the corresponding real object category; for the negative sample area, it is expected to have as low a prediction confidence as possible to characterize that the prediction result is not accurate. That is to say, a positive sample confidence loss value (through this confidence loss value, the positive sample area has a higher prediction confidence), a positive sample area loss value (through the area loss value, the positive sample area can have a more accurate prediction box) and a category loss value (through the category loss value, the prediction object category of the prediction box is more accurate) can be generated according to the positive sample area; and a negative sample confidence loss value can be generated according to the negative sample area, and then a first loss value is jointly generated according to the positive sample confidence loss value, the positive sample area loss value, the positive sample category loss value and the negative sample confidence loss value. The M sample prediction areas also include the sample prediction area k j (i.e., the M sample prediction regions include the above sample prediction region k i And the sample prediction area k j ) as an example, where the sample prediction area k i The sample prediction object category corresponding to the sample object in is based on the sample prediction area k output by the first convolutional network layer. i The sample object existence probability and sample category prediction probability of the sample object in the sample prediction area k are jointly determined; i The probability of existence of a sample object refers to the sample prediction area k i The probability that a sample object exists in the sample prediction area k j The sample prediction object category corresponding to the sample object in is based on the sample prediction area k output by the first convolutional network layer. j The sample object existence probability and sample category prediction probability of the sample object in the sample prediction area k are jointly determined; j The probability of existence of a sample object refers to the sample prediction area k j The probability that a sample object exists in ;

[0172] The specific method for generating the first loss value is: obtaining the sample prediction area k j The corresponding sample area ratio s j , the sample area ratio s i Ratio of sample area to j The sample prediction area whose area ratio is greater than or equal to the area ratio threshold is determined as the positive sample area, and the sample prediction area whose sample area ratio is less than the area ratio threshold is determined as the negative sample area; the positive sample initial loss value is generated according to the positive sample area, the real area corresponding to the positive sample area, the real object category corresponding to the real area corresponding to the positive sample area, and the sample prediction object category corresponding to the sample object in the positive sample area; the positive sample confidence loss value is generated according to the existence probability of the sample object corresponding to the positive sample area; the negative sample confidence loss value is generated according to the existence probability of the sample object corresponding to the negative sample area; the positive sample initial loss value, the positive sample confidence loss value and the negative sample confidence loss value are added to obtain the first loss value.

[0173] Among them, the specific method for generating the initial loss value of the positive sample can be: determining the regional loss value according to the real area corresponding to the positive sample area and the positive sample area; determining the category loss value according to the sample predicted object category of the sample object in the positive sample area and the real object category corresponding to the real area corresponding to the positive sample area; adding the regional loss value and the category loss value to obtain the initial loss value of the positive sample.

[0174] The specific method for generating the first loss value can be shown in the following formula (2):

[0175]

[0176] Among them, (loss(x,y,w,h)) can be used to characterize the regional loss value generated by each positive sample area and its corresponding real area, where x, y, w, h can be used to characterize the coordinate position information of the positive sample area in the training sample image; loss(cls) can be used to characterize the category loss value generated by the sample prediction object category of each positive sample area and its corresponding real object category; loss(conf) can be used to characterize the confidence loss value corresponding to each positive sample area or negative sample area; ∑ neg loss(conf) can be used to characterize the sum of the confidence loss values ​​corresponding to all negative sample areas; ∑ pos (loss(x,y,w,h)+loss(cls)+loss(conf)) can be used to characterize the positive sample loss values ​​corresponding to all positive sample areas (the regional loss values, category loss values ​​and confidence loss values ​​of all positive sample areas are added together to obtain the positive sample loss value).

[0177] Among them, when calculating the above loss (x, y, w, h) (i.e., the regional loss value corresponding to the positive sample area), the loss value corresponding to the coordinate position information x, the loss value corresponding to the coordinate position information y, the loss value corresponding to the coordinate position information w, and the loss value corresponding to the coordinate position information h can be calculated, and then the loss value corresponding to the coordinate position information x, the loss value corresponding to the coordinate position information y, the loss value corresponding to the coordinate position information w, and the loss value corresponding to the coordinate position information h are merged (for example, added), and the loss (x, y, w, h) corresponding to each positive sample area can be obtained. For ease of understanding, the following will take the calculation of the loss value corresponding to the coordinate position information x as an example to illustrate the calculation of the loss value corresponding to each coordinate position information. The calculation of the loss value corresponding to the coordinate position information x can be shown in the following formula (3):

[0178] L1 loss(x,x truth )=|xx truth | Formula (3)

[0179] Among them, x can be used to represent the coordinate position information x, x in the positive sample area truth It can be used to represent the true value corresponding to the coordinate position information x, L1 loss (x, x truth ) can be used to characterize the loss value corresponding to the coordinate position information x; it should be understood that the coordinate position information y, coordinate position information w, and coordinate position information h in the positive sample area can also use the loss value calculation method shown in formula (3) to determine the loss values ​​corresponding to the coordinate position information y, coordinate position information w, and coordinate position information h, respectively.

[0180] Step S205, determining a second loss value according to the predicted classification result between every two sample object label features among the M sample object label features and the actual classification result between every two sample object label features.

[0181] It should be understood that the first convolutional network layer in the present application can also output the sample object label features corresponding to each sample prediction area, because the first convolutional network layer will output multiple sample prediction areas of the same object (for example, the same sample object), and the present application expects that the sample object label features corresponding to different sample prediction areas of the same object will be more similar, while the sample object label features of different objects can be more different. Then the present application can generate a label feature loss value based on the sample object label features corresponding to the sample prediction area, and because the negative sample area is a prediction area with incorrect prediction, the negative sample area can be ignored when generating the label feature loss value, that is, the present application can generate a label feature loss value corresponding to the positive sample area based on the sample object label features corresponding to the positive sample area, and the label feature loss value can be called a second loss value, and the second loss value can be used to train the sample image recognition model (i.e., the first convolutional network layer) together with the above-mentioned first loss value.

[0182] Taking the case where the M sample prediction regions include the first sample prediction region and the second sample prediction region; the M sample object label features include the first sample object label feature corresponding to the first sample prediction region and the second sample object label feature corresponding to the second sample prediction region; and the first sample prediction region and the second sample prediction region belong to the positive sample region as an example, a specific method for generating the second loss value may be: the first sample object label feature and the second sample object label feature may be input into the second convolutional network layer, and the first sample object label feature and the second sample object label feature may be feature classified by the second convolutional network layer to obtain a predicted classification result between the first sample object label feature and the second sample object label feature; the real classification result between the first sample object label feature and the second sample object label feature may be determined according to the real object category corresponding to the real region corresponding to the first sample prediction region and the real object category corresponding to the real region corresponding to the second sample prediction region; the second loss value may be determined according to the predicted classification result between the first sample object label feature and the second sample object label feature and the real classification result between the first sample object label feature and the second sample object label feature.

[0183] It should be understood that the total loss value can be generated according to the first loss value and the second loss value, and the sample image recognition model can be trained according to the total loss value. To facilitate understanding of the specific implementation method of generating the total loss value in the present application, please refer to the following formula (4). Formula (4) can be a representation of the total loss value generated by the present application:

[0184]

[0185] Among them, as in formula (4) pos(loss(x,y,w,h)+loss(cls)+loss(conf)) can be used to represent the sum of the regional loss value, category loss value and confidence loss value of all positive sample regions; loss(tag) can be used to represent the label feature loss value (i.e., the second loss value)) between any two positive sample regions; ∑ random pos loss(tag) can be used to represent the loss value obtained by summing up the label feature loss values ​​between every two positive sample regions after calculating the label feature loss values ​​between all positive sample regions (that is, the label feature loss values ​​corresponding to all positive sample regions).

[0186] It should be understood that, as shown in formula (4), when calculating the loss value corresponding to the positive sample area, the present application not only calculates the regional loss value, the category loss value, and the confidence loss value, but also calculates the label feature loss value corresponding to the positive sample area. The regional loss value, the category loss value, the confidence loss value, and the label feature loss value can be used together to generate the loss value corresponding to the positive sample area. pos The specific implementation method of (loss(x,y,w,h)+losscls+loss(conf)) can refer to the description in the above formula (3); and the specific implementation method of determining loss(tag) can be shown as the following formula (5):

[0187] BCE loss(t,t truth )=y·log(t)+(1-t truth )·log(1-t) Formula (5)

[0188] Among them, it should be understood that, unlike the sample prediction area and the sample prediction object category, the sample object label feature does not have a corresponding true value, and the sample object label feature is actually a feature vector. The present application expects that the sample object label features corresponding to different prediction frames of the same object can be more similar; while the sample object label features corresponding to the prediction frames of different objects can be more different. The present application can provide a convolutional neural network, and deploy the convolutional neural network in a sample image recognition model (i.e., the second convolutional network layer mentioned above). The sample object label features of different sample prediction areas can be extracted through the first convolutional network layer, and any two sample object label features can be classified through the second convolutional network layer; as in formula (5), t can be used to characterize the second convolutional network layer to classify the two sample object label features corresponding to any two sample prediction areas to obtain a predicted classification result (e.g., the predicted classification result may include the same category or different categories. If the categories are the same, t can be 1, and if the categories are different, it can be 0); as in formula (5), t truthIt can be used to characterize the true classification results corresponding to the two sample object labels corresponding to the two sample prediction regions (eg, the true classification results are of the same category or different categories).

[0189] It should be understood that the positive sample loss value corresponding to the positive sample area and the negative sample loss value corresponding to the negative sample area (i.e., the negative sample confidence loss value) can be determined by the above formula (4). By adding the positive sample loss value to the negative sample loss value, the total loss value (loss) can be obtained. 总 ).

[0190] Step S204, adjusting the sample image recognition model according to the first loss value and the second loss value to obtain an image recognition model; the image recognition model is used to identify N prediction areas of the target object in the target image, the prediction object categories corresponding to the target objects in the N prediction areas, and the prediction object label features corresponding to the target objects in the N prediction areas; the prediction object categories corresponding to the target objects in the N prediction areas and the prediction object label features corresponding to the target objects in the N prediction areas are used to determine the optimal prediction area where the target object is located in the N prediction areas; N is a positive integer.

[0191] In this application, the parameters of the sample image recognition model can be adjusted according to the above-mentioned first loss value and the second loss value (i.e., the total loss value), until the sample image recognition model meets the model convergence condition (such as when the number of training iterations is reached or the total loss value is within the error tolerance range), and the sample image recognition model that meets the model convergence condition is determined as the image recognition model. The image recognition model can be applied to subsequent image recognition applications.

[0192] In the embodiment of the present application, the sample image recognition model is trained by using sample training images, so that the sample image recognition model can accurately identify the prediction area of ​​the target object in the target image, the prediction object category of the target object contained in the prediction area, and the prediction object label features of the target object contained in the prediction area; and then the accurate optimal prediction area of ​​the target object can be determined based on the accurate prediction area, prediction object category, and prediction object label features. The recognition accuracy of the object in the image can be improved.

[0193] To understand the model training process of this application, please refer to Figure 6 , Figure 6 It is a schematic diagram of a model training provided in an embodiment of the present application.

[0194] like Figure 6As shown, the sample training image is input into the sample image recognition model, through which the sample image recognition model can output the sample prediction area of ​​the sample training image, the sample prediction object category corresponding to the sample prediction area (the sample image recognition model can output the sample object existence probability and the sample category prediction probability corresponding to the sample prediction area, and the sample object existence probability and the sample category prediction probability corresponding to the sample prediction area can be used to determine the sample prediction object category corresponding to the sample prediction area), and the sample object label feature corresponding to the sample prediction area. It should be understood that the sample prediction area may include positive sample areas and negative sample areas (for the specific implementation method of determining the positive sample area and the negative sample area, please refer to the above Figure 5 The description in the corresponding embodiment will not be repeated here), and according to the sample prediction area and the sample prediction object category of the positive sample area, the regional loss value, category loss value and confidence loss value corresponding to the positive sample area can be determined (the confidence loss value can be determined based on the object existence probability of the positive sample area); according to the object existence probability of the negative sample area, the confidence loss value corresponding to the negative sample area can be determined; and the regional loss value, category loss value, positive sample confidence loss value corresponding to the positive sample area and the negative sample confidence loss value corresponding to the negative sample area can be referred to as Figure 6 The loss value shown is 1 (i.e., the first loss value mentioned above).

[0195] Furthermore, if Figure 6 As shown, the sample object label features corresponding to the positive sample area can be randomly combined in pairs as a set of feature groups; the feature groups are input into Figure 6 In the convolutional neural network shown, the convolutional neural network can classify the two sample object label features and determine whether the two sample object label features belong to the same category; it should be understood that the convolutional neural network can be deployed in the sample image recognition model as the above-mentioned second convolutional network layer. The sample object label features corresponding to the sample prediction area can be extracted through the first convolutional network layer in the sample image recognition model, and any two sample object label features are input into the second convolutional network layer. The second convolutional network layer can classify the two sample object label features. Furthermore, based on the predicted classification results output by the convolutional neural network, the label feature loss value corresponding to the positive sample area can be calculated. The label feature loss value can be referred to as follows: Figure 6 The loss value 2 shown (i.e., the second loss value in the previous text) is shown; the sample image recognition model can be adjusted based on the loss value 1 and the loss value 2 to obtain the final image recognition model. For the specific implementation method of generating the loss value 1 and the loss value 2, please refer to the above Figure 5The description of generating the regional loss value corresponding to the positive sample area, the category loss value, the positive sample confidence loss value, the negative sample confidence loss value corresponding to the negative sample area, and the label feature loss value corresponding to the positive sample area in the corresponding embodiment will not be repeated here.

[0196] For further information, see Figure 7 , Figure 7 1 is a schematic diagram of the structure of a data identification device provided in an embodiment of the present application. The above-mentioned data identification device may be a computer program (including program code) running in a computer device, for example, the data identification device is an application software; the data identification device may be used to execute the above-mentioned Figure 3 The corresponding steps in the method provided by the corresponding embodiment. The data identification device 1 may include: a region identification module 11, a data acquisition module 12, a target region acquisition module 13, a coverage region determination module 14, a similarity determination module 15 and an optimal region determination module 16.

[0197] A region recognition module 11 is used to obtain a target image;

[0198] The region identification module 11 is further used to identify N predicted regions of the target object in the target image; N is a positive integer;

[0199] A data acquisition module 12 is used to obtain the predicted object category and predicted object label features corresponding to the target object in each prediction area;

[0200] The target region acquisition module 13 is used to acquire the maximum category prediction probability among the category prediction probabilities corresponding to the N prediction object categories, and determine the prediction region corresponding to the prediction object category with the maximum category prediction probability as the target prediction region;

[0201] The coverage area determination module 14 is used to determine the coverage area jointly covered by the remaining prediction area and the target prediction area; the remaining prediction area is the prediction area other than the target prediction area among the N prediction areas;

[0202] A similarity determination module 15 is used to determine the label feature similarity between the predicted object label feature corresponding to the remaining prediction area and the predicted object label feature corresponding to the target prediction area;

[0203] The optimal region determination module 16 is used to determine the optimal prediction region of the target object in the target image from among the N prediction regions according to the similarity between the coverage region and the label feature.

[0204] The specific implementation of the region identification module 11, the data acquisition module 12, the target region acquisition module 13, the coverage region determination module 14, the similarity determination module 15 and the optimal region determination module 16 can be found in the above Figure 3 The corresponding steps are steps S101 to S106 in the embodiment.

[0205] See also Figure 7 The region identification module 11 may include: an image input unit 111 and a region identification unit 112 .

[0206] The image input unit 111 is used to input the target image into the image recognition model, and output the candidate prediction region corresponding to the target image, the object existence probability corresponding to the candidate prediction region, and the initial category prediction probability corresponding to the candidate prediction region through the image recognition model; the object existence probability refers to the probability that the object exists in the candidate prediction region;

[0207] The region identification unit 112 is used to obtain N prediction regions of the target object in the target image from among the candidate prediction regions according to the object existence probability and the initial category prediction probability corresponding to the candidate prediction region.

[0208] The specific implementation of the image input unit 111 and the region identification unit 112 can be found in the above Figure 3 The description of step S101 in the corresponding embodiment will not be repeated here.

[0209] See also Figure 7 The region identification unit 112 may include: a region selection subunit 1121, a category determination subunit 1122, a region classification subunit 1123 and a region identification subunit 1124.

[0210] The region selection subunit 1121 is used to determine the candidate prediction region where the object existence probability is greater than the probability threshold as the prediction region to be identified;

[0211] The category determination subunit 1122 is used to obtain the maximum initial category prediction probability from the initial category prediction probabilities corresponding to the prediction area to be identified, and determine the object category corresponding to the maximum initial category prediction probability as the predicted object category of the object contained in the prediction area to be identified;

[0212] The region classification subunit 1123 is used to classify the prediction region to be identified according to the prediction object category of the object contained in the prediction region to be identified, and obtain M region sets; the prediction object category of the object contained in the prediction region to be identified included in one region set belongs to the same object category;

[0213] The region identification subunit 1124 is used to obtain a target region set from the M region sets, determine the object contained in the prediction region to be identified included in the target region set as the target object, and determine the prediction region to be identified included in the target region set as N prediction regions of the target object in the target image.

[0214] The specific implementation of the region selection subunit 1121, the category determination subunit 1122, the region classification subunit 1123 and the region identification subunit 1124 can be found in the above Figure 3 The description of step S101 in the corresponding embodiment will not be repeated here.

[0215] In one embodiment, the data identification device 1 may further include: a feature extraction module 17 and a label feature extraction module 18 .

[0216] A feature extraction module 17, used to extract regional features of candidate prediction regions through an image recognition model;

[0217] The label feature extraction module 18 is used to extract the regional features to obtain the prediction object label features corresponding to the candidate prediction region; the prediction object label features corresponding to the candidate prediction region include the prediction object label features corresponding to the N prediction regions respectively.

[0218] The specific implementation of the feature extraction module 17 and the label feature extraction module 18 can be found in the above Figure 3 The description of step S101 in the corresponding embodiment will not be repeated here.

[0219] See also Figure 7 The optimal region determination module 16 may include: a region operation unit 161 and an optimal region determination unit 162 .

[0220] The area operation unit 161 is used to use the area in the target prediction area that is not covered by the remaining prediction area as the first area to be operated;

[0221] The area operation unit 161 is further used to use the area in the remaining prediction area that is not covered by the target prediction area as the second area to be operated;

[0222] The area calculation unit 161 is further used to add the area corresponding to the first area to be calculated and the area corresponding to the second area to be calculated to obtain the area of ​​the first area;

[0223] The area calculation unit 161 is further used to obtain the area of ​​the second area corresponding to the coverage area, and determine the area ratio between the area of ​​the second area and the area of ​​the first area;

[0224] The optimal region determining unit 162 is used to determine the optimal prediction region of the target object in the target image from among the N prediction regions according to the area ratio and the label feature similarity.

[0225] The specific implementation of the region operation unit 161 and the optimal region determination unit 162 can be found in the above Figure 3 The description of step S106 in the corresponding embodiment will not be repeated here.

[0226] In one embodiment, the optimal region determining unit 162 is further specifically configured to determine the target prediction region among the N prediction regions as the optimal prediction region of the target object in the target image when the area ratio is greater than or equal to the ratio threshold and the label feature similarity is greater than or equal to the similarity threshold;

[0227] The optimal region determination unit 162 is further specifically used to determine the remaining prediction regions in the N prediction regions and the target prediction region as the optimal prediction regions of the target object in the target image when the area ratio is less than the ratio threshold or the label feature similarity is less than the similarity threshold.

[0228] In an embodiment of the present application, after identifying N prediction areas of the target object in the target image, the prediction object category and prediction object label features corresponding to the target object in each prediction area can be obtained; based on the category prediction probability corresponding to the prediction object category, the target prediction area can be obtained in the N prediction areas; and the optimal prediction area of ​​the target object can be screened out in the N prediction areas, based on the overlapping area (commonly covered coverage area) between the remaining prediction areas and the target prediction area, and the similarity between the prediction object label features, to jointly determine the optimal prediction area of ​​the target object in the target image. It should be understood that the target object can be understood as an object in the target image, and the prediction area can be understood as the detection frame of the object. When determining the final detection frame of the target object, the present application not only considers the overlapping area between the detection frames, but also considers the similarity between the prediction object label features between the detection frames, and the optimal detection frame jointly determined by the two is more accurate. In summary, the present application can improve the recognition accuracy of objects in images.

[0229] See also Figure 8 , Figure 8 1 is a schematic diagram of the structure of another data identification device provided in an embodiment of the present application. The above-mentioned data identification device may be a computer program (including program code) running in a computer device, for example, the data identification device is an application software; the data identification device may be used to execute the above-mentioned Figure 5The corresponding steps in the method provided by the corresponding embodiment. The data identification device 2 may include: a sample input module 21, a real label acquisition module 22, a loss value generation module 23 and a model adjustment module 24.

[0230] The sample input module 21 is used to obtain a sample training image, input the sample training image into the sample image recognition model, and output M sample prediction regions of the sample object in the sample training image, the sample prediction object category corresponding to the sample object in each sample prediction region, and the sample object label features corresponding to the sample object in each sample prediction region through the first convolutional network layer in the sample image recognition model; M is a positive integer;

[0231] A real label acquisition module 22 is used to obtain the real area corresponding to each sample prediction area;

[0232] The real label acquisition module 22 is also used to obtain the real object category corresponding to each real area;

[0233] A loss value generating module 23, configured to determine a first loss value according to the M sample prediction regions, the real regions corresponding to each sample prediction region, the real object categories corresponding to each real region, and the sample prediction object categories corresponding to the sample objects in each sample prediction region;

[0234] The loss value generating module 23 is further used to determine a second loss value according to the predicted classification result between every two sample object label features in the M sample object label features and the actual classification result between every two sample object label features;

[0235] The model adjustment module 24 is used to adjust the sample image recognition model according to the first loss value and the second loss value to obtain an image recognition model; the image recognition model is used to identify N prediction areas of the target object in the target image, the prediction object categories corresponding to the target objects in the N prediction areas, and the prediction object label features corresponding to the target objects in the N prediction areas; the prediction object categories corresponding to the target objects in the N prediction areas, and the prediction object label features corresponding to the target objects in the N prediction areas are used to determine the optimal prediction area where the target object is located in the N prediction areas; N is a positive integer.

[0236] The specific implementation of the sample input module 21, the real label acquisition module 22, the loss value generation module 23 and the model adjustment module 24 can be found in the above Figure 5 The description of steps S201 to S204 in the corresponding embodiment will not be repeated here.

[0237] In one embodiment, the M sample prediction regions include sample prediction region ki ; i is a positive integer;

[0238] See also Figure 8 The real label acquisition module 22 may include: a configuration area acquisition unit 221 and a real area determination unit 222 .

[0239] The configuration region acquisition unit 221 is used to acquire the candidate configuration real region corresponding to the sample training image;

[0240] The region acquisition unit 221 is configured to determine the sample prediction region k i The sample coverage area that is commonly covered by the candidate configuration's real area;

[0241] The real area determination unit 222 is used to obtain the sample prediction area k in the candidate configuration real area according to the sample coverage area. i The corresponding real area.

[0242] The specific implementation of the configuration area acquisition unit 221 and the real area determination unit 222 can be found in the above Figure 5 The description of step S202 in the corresponding embodiment will not be repeated here.

[0243] See also Figure 8 The real area determination unit 222 may include: a ratio determination subunit 2221 and a real area determination subunit 2222 .

[0244] The ratio determination subunit 2221 is used to configure the candidate to be a sample prediction area k in the real area i The uncovered area is taken as the first sample area, and the sample prediction area k i The area not covered by the real area of ​​the candidate configuration is used as the second sample area;

[0245] The ratio determination subunit 2221 is further used to determine the candidate configuration true area and the sample prediction area k according to the first sample area, the second sample area and the sample coverage area. i The sample area ratio s i ;

[0246] The real area determination subunit 2222 is used to determine the sample area ratio s i If the area ratio is greater than the area ratio threshold, the candidate is configured as the real area and determined as the sample prediction area k i The corresponding real area.

[0247] The specific implementation of the ratio determination subunit 2221 and the real area determination subunit 2222 can be found in the above Figure 5The description of step S202 in the corresponding embodiment will not be repeated here.

[0248] In one embodiment, the M sample prediction regions also include a sample prediction region k j ; j is a positive integer; sample prediction area k i The sample prediction object category corresponding to the sample object in is based on the sample prediction area k output by the first convolutional network layer. i The sample object existence probability and sample category prediction probability of the sample object in the sample prediction area k are jointly determined; i The probability of existence of a sample object refers to the sample prediction area k i The probability that a sample object exists in the sample prediction area k j The sample prediction object category corresponding to the sample object in is based on the sample prediction area k output by the first convolutional network layer. j The sample object existence probability and sample category prediction probability of the sample object in the sample prediction area k are jointly determined; j The probability of existence of a sample object refers to the sample prediction area k j The probability that a sample object exists in ;

[0249] The loss value generating module 23 may include: a sample dividing unit 231 and a loss value generating unit 232 .

[0250] The sample division unit 231 is used to obtain the sample prediction area k j The corresponding sample area ratio s j , the sample area ratio s i Ratio of sample area to j The sample prediction area whose area ratio is greater than or equal to the area ratio threshold is determined as the positive sample area, and the sample prediction area whose area ratio is less than the area ratio threshold is determined as the negative sample area;

[0251] A loss value generating unit 232 is used to generate an initial loss value of the positive sample according to the positive sample region, the real region corresponding to the positive sample region, the real object category corresponding to the real region corresponding to the positive sample region, and the sample predicted object category corresponding to the sample object in the positive sample region;

[0252] The loss value generating unit 232 is further used to generate a positive sample confidence loss value according to the existence probability of the sample object corresponding to the positive sample area;

[0253] The loss value generating unit 232 is further used to generate a negative sample confidence loss value according to the existence probability of the sample object corresponding to the negative sample area;

[0254] The loss value generating unit 232 is further used to add the positive sample initial loss value, the positive sample confidence loss value and the negative sample confidence loss value to obtain a first loss value.

[0255] In one embodiment, the loss value generating unit 232 is further specifically configured to determine the region loss value according to the real region corresponding to the positive sample region and the positive sample region;

[0256] The loss value generating unit 232 is further specifically used to determine a category loss value according to the sample prediction object category of the sample object in the positive sample area and the real object category corresponding to the real area corresponding to the positive sample area;

[0257] The loss value generating unit 232 is further specifically used to add the region loss value and the category loss value to obtain an initial loss value of the positive sample.

[0258] In one embodiment, the M sample prediction regions include a first sample prediction region and a second sample prediction region; the M sample object label features include a first sample object label feature corresponding to the first sample prediction region and a second sample object label feature corresponding to the second sample prediction region; the first sample prediction region and the second sample prediction region belong to positive sample regions;

[0259] The loss value generating unit 232 is further specifically used to input the first sample object label feature and the second sample object label feature into the second convolutional network layer, perform feature classification on the first sample object label feature and the second sample object label feature through the second convolutional network layer, and obtain a prediction classification result between the first sample object label feature and the second sample object label feature;

[0260] The loss value generating unit 232 is further specifically used to determine the true classification result between the first sample object label feature and the second sample object label feature according to the true object category corresponding to the true region corresponding to the first sample prediction region and the true object category corresponding to the true region corresponding to the second sample prediction region;

[0261] The loss value generating unit 232 is further specifically used to determine the second loss value according to the predicted classification result between the first sample object label feature and the second sample object label feature and the actual classification result between the first sample object label feature and the second sample object label feature.

[0262] In the embodiment of the present application, the sample image recognition model is trained by using sample training images, so that the sample image recognition model can accurately identify the prediction area of ​​the target object in the target image, the prediction object category of the target object contained in the prediction area, and the prediction object label features of the target object contained in the prediction area; and then the accurate optimal prediction area of ​​the target object can be determined based on the accurate prediction area, prediction object category, and prediction object label features. The recognition accuracy of the object in the image can be improved.

[0263] For further information, see Fig. 9 , Fig. 9 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig. 9 As shown above Figure 7 The data identification device 1 in the corresponding embodiment, or Figure 8 The data identification device 2 in the corresponding embodiment can be applied to the above-mentioned computer device 1000. The above-mentioned computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 also includes: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may optionally also be at least one storage device located away from the aforementioned processor 1001. Fig. 9 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0264] exist Fig. 9 In the computer device 1000 shown, the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0265] Obtain a target image, and identify N predicted regions of the target object in the target image; N is a positive integer;

[0266] Obtain the predicted object category and predicted object label features corresponding to the target object in each prediction area;

[0267] Among the category prediction probabilities corresponding to the N prediction object categories, the maximum category prediction probability is obtained, and the prediction area corresponding to the prediction object category with the maximum category prediction probability is determined as the target prediction area;

[0268] Determine the coverage area jointly covered by the remaining prediction area and the target prediction area; the remaining prediction area is the prediction area other than the target prediction area among the N prediction areas;

[0269] Determine the label feature similarity between the predicted object label feature corresponding to the remaining predicted area and the predicted object label feature corresponding to the target predicted area;

[0270] According to the similarity between the coverage area and the label feature, the optimal prediction area of ​​the target object in the target image is determined among the N prediction areas.

[0271] Or implement:

[0272] Obtain a sample training image, input the sample training image into a sample image recognition model, and output M sample prediction regions of the sample object in the sample training image, the sample prediction object category corresponding to the sample object in each sample prediction region, and the sample object label features corresponding to the sample object in each sample prediction region through the first convolutional network layer in the sample image recognition model; M is a positive integer;

[0273] Inputting the M sample object label features into the second convolutional network layer in the sample image recognition model, and outputting the predicted classification result between every two sample object label features among the M sample object label features through the second convolutional network layer;

[0274] Get the real area corresponding to each sample prediction area, and get the real object category corresponding to each real area;

[0275] Determine a first loss value according to the M sample prediction regions, the real regions corresponding to each sample prediction region, the real object categories corresponding to each real region, and the sample prediction object categories corresponding to the sample objects in each sample prediction region;

[0276] Determine a second loss value according to the predicted classification result between every two sample object label features among the M sample object label features and the actual classification result between every two sample object label features;

[0277] The sample image recognition model is adjusted according to the first loss value and the second loss value to obtain an image recognition model; the image recognition model is used to identify N prediction areas of the target object in the target image, the prediction object categories corresponding to the target objects in the N prediction areas, and the prediction object label features corresponding to the target objects in the N prediction areas; the prediction object categories corresponding to the target objects in the N prediction areas and the prediction object label features corresponding to the target objects in the N prediction areas are used to determine the optimal prediction area where the target object is located in the N prediction areas; N is a positive integer.

[0278] It should be understood that the computer device 1000 described in the embodiment of the present application can execute the above Figure 3 or Figure 5 The description of the data identification method in the corresponding embodiment can also be performed as described above. Figure 7 In the corresponding embodiment, the data identification device 1, or Figure 8 The description of the data identification device 2 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here either.

[0279] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the computer device 1000 for data processing mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, the computer program can execute the above-mentioned data processing. Figure 3 or Figure 5 The description of the above data processing method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0280] The computer-readable storage medium may be the data identification device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0281] In one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in one aspect of the embodiments of the present application.

[0282] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other step units inherent to these processes, methods, devices, products, or equipment.

[0283] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0284] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0285] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data identification method, characterized in that: include: Acquire a target image, input the target image into an image recognition model, and output, through the image recognition model, a candidate prediction region corresponding to the target image, an object existence probability corresponding to the candidate prediction region, and an initial category prediction probability corresponding to the candidate prediction region; the object existence probability refers to the probability that an object exists in the candidate prediction region; Determine the candidate prediction region where the object existence probability is greater than the probability threshold as the prediction region to be identified; obtain the maximum initial category prediction probability from the initial category prediction probabilities corresponding to the prediction region to be identified, and determine the object category corresponding to the maximum initial category prediction probability as the predicted object category of the object contained in the prediction region to be identified; perform regional classification on the prediction region to be identified according to the predicted object category of the object contained in the prediction region to be identified, and obtain M region sets; the predicted object categories of the objects contained in the prediction region to be identified included in one region set belong to the same object category; obtain a target region set from the M region sets, determine the object contained in the prediction region to be identified included in the target region set as the target object, and determine the prediction region to be identified included in the target region set as N prediction regions of the target object in the target image; N is a positive integer; Obtain the predicted object category and predicted object label features corresponding to the target object in each predicted area; Among the category prediction probabilities corresponding to the N prediction object categories, the maximum category prediction probability is obtained, and the prediction area corresponding to the prediction object category having the maximum category prediction probability is determined as the target prediction area; Determine a coverage area jointly covered by the remaining prediction area and the target prediction area; the remaining prediction area is a prediction area other than the target prediction area among the N prediction areas; Determine the label feature similarity between the predicted object label feature corresponding to the remaining prediction area and the predicted object label feature corresponding to the target prediction area; According to the similarity between the coverage area and the label feature, an optimal prediction area of ​​the target object in the target image is determined among the N prediction areas.

2. The method according to claim 1, characterized in that: The method further comprises: Extracting regional features of the candidate prediction region by using the image recognition model; Feature extraction is performed on the region features to obtain prediction object label features corresponding to the candidate prediction region; the prediction object label features corresponding to the candidate prediction region include prediction object label features corresponding to the N prediction regions respectively.

3. The method according to claim 1, characterized in that The determining, according to the similarity between the coverage area and the label feature, the optimal prediction area of ​​the target object in the target image among the N prediction areas comprises: Taking the area in the target prediction area that is not covered by the remaining prediction area as the first area to be calculated; The area in the remaining prediction area that is not covered by the target prediction area is used as the second area to be calculated; Adding the area corresponding to the first area to be calculated and the area corresponding to the second area to be calculated to obtain the area of ​​the first area; Acquire an area of ​​a second region corresponding to the coverage area, and determine an area ratio between the area of ​​the second region and the area of ​​the first region; The optimal prediction region of the target object in the target image is determined among the N prediction regions according to the area ratio and the label feature similarity.

4. The method according to claim 3, characterized in that The determining, in the N prediction regions, the optimal prediction region of the target object in the target image according to the area ratio and the label feature similarity comprises: When the area ratio is greater than or equal to a ratio threshold, and the label feature similarity is greater than or equal to a similarity threshold, determining the target prediction region among the N prediction regions as the optimal prediction region of the target object in the target image; When the area ratio is less than the ratio threshold, or the label feature similarity is less than the similarity threshold, the remaining prediction areas in the N prediction areas and the target prediction area are both determined as the optimal prediction areas of the target object in the target image.

5. A data identification method, characterized in that: include: Acquire a sample training image, input the sample training image into a sample image recognition model, and output, through the first convolutional network layer in the sample image recognition model, M sample prediction regions of the sample object in the sample training image, the sample prediction object category corresponding to the sample object in each sample prediction region, and the sample object label features corresponding to the sample object in each sample prediction region; M is a positive integer; Inputting the M sample object label features into the second convolutional network layer in the sample image recognition model, and outputting the predicted classification result between every two sample object label features among the M sample object label features through the second convolutional network layer; Obtaining the real area corresponding to each sample prediction area, and obtaining the real object category corresponding to each real area; Determine a first loss value according to the M sample prediction regions, the real regions corresponding to each of the sample prediction regions, the real object categories corresponding to each of the real regions, and the sample prediction object categories corresponding to the sample objects in each of the sample prediction regions; Determine a second loss value according to a predicted classification result between every two sample object label features among the M sample object label features and a true classification result between every two sample object label features; The sample image recognition model is adjusted according to the first loss value and the second loss value to obtain an image recognition model; the image recognition model is used to identify N prediction areas of the target object in the target image, the prediction object categories corresponding to the target objects in the N prediction areas, and the prediction object label features corresponding to the target objects in the N prediction areas; the prediction object categories corresponding to the target objects in the N prediction areas and the prediction object label features corresponding to the target objects in the N prediction areas are used to determine the optimal prediction area where the target object is located in the N prediction areas; N is a positive integer.

6. The method according to claim 5, characterized in that The M sample prediction regions include sample prediction region k i ; i is a positive integer; The obtaining of the real area corresponding to each sample prediction area includes: Obtaining a candidate configuration real region corresponding to the sample training image; Determine the sample prediction area k i The sample coverage area commonly covered by the candidate configuration real area; According to the sample coverage area, the sample prediction area k is obtained in the candidate configuration real area. i The corresponding real area.

7. The method according to claim 6, characterized in that According to the sample coverage area, the sample prediction area k is obtained in the candidate configuration real area. i The corresponding real areas include: The candidate is configured in the real area of ​​the sample prediction area k i The uncovered area is taken as the first sample area, and the sample prediction area k is i The area not covered by the candidate configuration real area is used as the second sample area; Determine the candidate configuration true area and the sample prediction area k according to the first sample area, the second sample area and the sample coverage area i The sample area ratio s i ; If the sample area ratio s i If the area ratio is greater than the area ratio threshold, the candidate configuration true area is determined as the sample prediction area k i The corresponding real area.

8. The method according to claim 7, characterized in that The M sample prediction regions also include sample prediction region k j ; j is a positive integer; the sample prediction area k i The sample prediction object category corresponding to the sample object in is based on the sample prediction area k output by the first convolutional network layer. i The sample object existence probability and the sample category prediction probability of the sample object in the sample prediction area k are jointly determined; the sample prediction area k i The probability of existence of a sample object refers to the sample prediction area k i The probability that the sample object exists in the sample prediction area k j The sample prediction object category corresponding to the sample object in is based on the sample prediction area k output by the first convolutional network layer. j The sample object existence probability and the sample category prediction probability of the sample object in the sample prediction area k are jointly determined; the sample prediction area k j The probability of existence of a sample object refers to the sample prediction area k j The probability that the sample object exists in ; The determining of a first loss value according to the M sample prediction regions, the real regions corresponding to each of the sample prediction regions, the real object categories corresponding to each of the real regions, and the sample prediction object categories corresponding to the sample objects in each of the sample prediction regions comprises: Get the sample prediction area k j The corresponding sample area ratio s j , the sample area ratio s i Ratio of sample area to j The sample prediction regions whose area ratio is greater than or equal to the area ratio threshold are determined as positive sample regions, and the sample prediction regions whose area ratio is less than the area ratio threshold are determined as negative sample regions; Generate a positive sample initial loss value according to the positive sample area, the real area corresponding to the positive sample area, the real object category corresponding to the real area corresponding to the positive sample area, and the sample prediction object category corresponding to the sample object in the positive sample area; Generate a positive sample confidence loss value according to the existence probability of the sample object corresponding to the positive sample area; Generate a negative sample confidence loss value according to the existence probability of the sample object corresponding to the negative sample area; The positive sample initial loss value, the positive sample confidence loss value and the negative sample confidence loss value are added together to obtain the first loss value.

9. The method according to claim 8, characterized in that The generating of the positive sample initial loss value according to the positive sample region, the real region corresponding to the positive sample region, the real object category corresponding to the real region corresponding to the positive sample region, and the sample predicted object category corresponding to the sample object in the positive sample region comprises: Determining a regional loss value according to a real region corresponding to the positive sample region and the positive sample region; Determine a category loss value according to a sample predicted object category of the sample object in the positive sample area and a real object category corresponding to a real area corresponding to the positive sample area; The region loss value and the category loss value are added to obtain the initial loss value of the positive sample.

10. The method according to claim 8, characterized in that The M sample prediction regions include a first sample prediction region and a second sample prediction region; the M sample object label features include a first sample object label feature corresponding to the first sample prediction region and a second sample object label feature corresponding to the second sample prediction region; the first sample prediction region and the second sample prediction region belong to the positive sample region; Determining a second loss value according to a predicted classification result between every two sample object label features among the M sample object label features and a true classification result between every two sample object label features includes: Inputting the first sample object label feature and the second sample object label feature into the second convolutional network layer, and performing feature classification on the first sample object label feature and the second sample object label feature through the second convolutional network layer to obtain a predicted classification result between the first sample object label feature and the second sample object label feature; Determine a true classification result between the first sample object label feature and the second sample object label feature according to the real object category corresponding to the real area corresponding to the first sample prediction area and the real object category corresponding to the real area corresponding to the second sample prediction area; The second loss value is determined according to a predicted classification result between the first sample object label feature and the second sample object label feature and a true classification result between the first sample object label feature and the second sample object label feature.

11. A data identification device, characterized in that: include: A region recognition module is used to obtain a target image, input the target image into an image recognition model, and output a candidate prediction region corresponding to the target image, an object existence probability corresponding to the candidate prediction region, and an initial category prediction probability corresponding to the candidate prediction region through the image recognition model; the object existence probability refers to the probability that an object exists in the candidate prediction region; The region identification module is further used to determine the candidate prediction region where the object existence probability is greater than the probability threshold as the prediction region to be identified; obtain the maximum initial category prediction probability from the initial category prediction probabilities corresponding to the prediction region to be identified, and determine the object category corresponding to the maximum initial category prediction probability as the predicted object category of the object contained in the prediction region to be identified; perform region classification on the prediction region to be identified according to the predicted object category of the object contained in the prediction region to be identified, and obtain M region sets; the predicted object categories of the objects contained in the prediction region to be identified included in one region set belong to the same object category; obtain a target region set from the M region sets, determine the object contained in the prediction region to be identified included in the target region set as the target object, and determine the prediction region to be identified included in the target region set as the N prediction regions of the target object in the target image; N is a positive integer; A data acquisition module, used to acquire the predicted object category and predicted object label features respectively corresponding to the target object in each predicted area; A target region acquisition module is used to acquire the maximum category prediction probability among the category prediction probabilities corresponding to the N prediction object categories, and determine the prediction region corresponding to the prediction object category having the maximum category prediction probability as the target prediction region; A coverage area determination module, used to determine a coverage area jointly covered by the remaining prediction area and the target prediction area; the remaining prediction area is a prediction area other than the target prediction area among the N prediction areas; A similarity determination module, used to determine the label feature similarity between the predicted object label feature corresponding to the remaining prediction area and the predicted object label feature corresponding to the target prediction area; The optimal region determination module is used to determine the optimal prediction region of the target object in the target image from among the N prediction regions according to the similarity between the coverage region and the label feature.

12. A computer device, characterized in that: include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide a network communication function, the memory is used to store program codes, and the processor is used to call the program codes to execute the method described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the method according to any one of claims 1 to 10 is executed.

Images