Image recognition method, device and equipment

Abstract

The application provides an image recognition method, device and equipment. After a plurality of to-be-recognized images of a preset target are acquired, first image recognition is performed on the plurality of to-be-recognized images to obtain a plurality of recognition result vectors. Then, the plurality of recognition result vectors are input into a preset image quality judgment model. The image quality judgment model is used to determine the quality of the plurality of to-be-recognized images according to the vector distance between the plurality of recognition result vectors corresponding to the plurality of to-be-recognized images. Thus, second image recognition is performed according to the quality, that is, the quality of the to-be-recognized images is considered when image recognition is performed, thereby solving the problems of missed detection and low accuracy of recognition results in existing image recognition. Moreover, the embodiment of the application improves the accuracy of image recognition, enables relevant personnel to timely and correctly process based on the recognition results, and meets the needs of actual application.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image recognition method, device and equipment. BACKGROUND

[0002] Image recognition technology can recognize the relevant information of the object contained in the image. For example, by recognizing the face image of a person, some information of the person can be determined. Therefore, with the continuous development of economy, image recognition technology is applied more and more widely, such as face recognition has been applied in fields such as security, vehicle, finance, etc.

[0003] Among them, many vehicle models in the vehicle field will be equipped with face recognition function to provide customized services for the vehicle owner. For example, by face recognition, the behavior of the driver is determined, so that when it is determined that the behavior of the driver is not conducive to the safe driving of the vehicle, measures are taken in time to reduce the influence of the behavior of the driver on the safe driving of the vehicle as much as possible.

[0004] However, when the camera captures data, the face image obtained by image recognition such as face recognition has problems such as over-brightness, over-darkness and blurring, as well as problems such as occlusion and too large head posture, resulting in poor quality of the face image (these poor quality images can be collectively referred to as low quality images). If low quality images are used in subsequent face recognition, it may cause more false negatives (FN) and the accuracy of the recognition result is low, so that relevant personnel cannot correctly handle based on the recognition result. For example, in the vehicle field, the accuracy of face recognition of the behavior of the driver that is not conducive to the safe driving of the vehicle is low, which greatly increases the probability of accidents of the vehicle. SUMMARY

[0005] To solve the problems in the prior art, the present application provides an image recognition method, device and equipment.

[0006] In a first aspect, an image recognition method is provided, which comprises:

[0007] obtaining a plurality of to-be-recognized images of a preset target;

[0008] performing first image recognition based on the plurality of to-be-recognized images to obtain a plurality of recognition result vectors corresponding to the plurality of to-be-recognized images;

[0009] inputting the plurality of recognition result vectors into a preset image quality judgment model, wherein the image quality judgment model is used to determine the quality of the plurality of to-be-recognized images according to the vector distance between the plurality of recognition result vectors corresponding to the plurality of to-be-recognized images;

[0010] According to the quality of the plurality of to-be-identified images, second image identification is performed.

[0011] In a possible implementation, before the plurality of identification result vectors are input into the preset image quality judgment model, the method further includes:

[0012] A plurality of reference images are acquired, first image identification is performed on the plurality of reference images, and a plurality of identification result vectors corresponding to the plurality of reference images are obtained.

[0013] A determination result vector F is determined. i A vector distance of each of the remaining identification result vectors except the determination result vector F i from the plurality of identification result vectors corresponding to the plurality of reference images is determined, where the determination result vector F i is any one of the plurality of identification result vectors corresponding to the plurality of reference images, i = 1,..., N, and N represents a number of vectors in the plurality of identification result vectors corresponding to the plurality of reference images.

[0014] Based on the vector distance, an initial image quality judgment model is trained, so that a value of a loss function of the trained initial image quality judgment model satisfies a preset requirement, where the value of the loss function is determined according to predicted qualities of the plurality of reference images and real qualities of the plurality of reference images, and the predicted qualities of the plurality of reference images are determined according to the vector distance.

[0015] The preset image quality judgment model is obtained according to the trained initial image quality judgment model.

[0016] In a possible implementation, before the initial image quality judgment model is trained based on the vector distance, the method further includes:

[0017] A minimum distance is acquired from the vector distance.

[0018] It is determined whether the minimum distance is greater than a preset distance threshold.

[0019] The initial image quality judgment model is trained based on the vector distance, including:

[0020] If the minimum distance is less than or equal to the preset distance threshold, the initial image quality judgment model is trained based on the vector distance.

[0021] In a possible implementation, the initial image quality judgment model is trained based on the vector distance, including:

[0022] From the vector distance, a minimum distance is obtained, and a mean value of a pre-stored distance of a negative sample pair and the recognition result vector F are calculated i a first difference value of the corresponding vector distance, and a second difference value of the mean value of the distance and the minimum distance;

[0023] Based on the first difference value and the second difference value, the initial image quality judgment model is trained, and the predicted quality of the plurality of reference images is determined according to the first difference value and the second difference value.

[0024] In a possible implementation, before the second image recognition according to the quality of the plurality of to-be-recognized images, the method further includes:

[0025] Based on the quality of the plurality of to-be-recognized images, an evaluation of image quality judgment is performed.

[0026] The second image recognition according to the quality of the plurality of to-be-recognized images includes:

[0027] If the evaluation passes, the second image recognition according to the quality of the plurality of to-be-recognized images is performed.

[0028] In a possible implementation, the evaluation of image quality judgment based on the quality of the plurality of to-be-recognized images includes:

[0029] Based on the quality of the plurality of to-be-recognized images, to-be-filtered images in the plurality of to-be-recognized images are determined.

[0030] A positive sample proportion before filtering is determined according to positive sample images in the plurality of to-be-recognized images, and a filtering proportion is determined according to the to-be-filtered images.

[0031] Based on the positive sample proportion before filtering, a change curve of a positive sample proportion after filtering and the filtering proportion is determined.

[0032] The evaluation of the image quality judgment is performed according to the change curve of the positive sample proportion after filtering and the filtering proportion.

[0033] In a possible implementation, the evaluation of the image quality judgment according to the change curve of the positive sample proportion after filtering and the filtering proportion includes:

[0034] A pre-stored change curve of a positive sample proportion after filtering and a filtering proportion is obtained.

[0035] An evaluation index value is determined according to the change curve of the positive sample proportion after filtering and the filtering proportion and the pre-stored change curve of the positive sample proportion after filtering and the filtering proportion.

[0036] If the evaluation index value is greater than a preset evaluation threshold, it is determined that the evaluation passes.

[0037] In a possible implementation, the performing the second image recognition according to the quality of the plurality of to-be-recognized images comprises:

[0038] According to the quality of the plurality of to-be-recognized images and a preset quality requirement, a target image is obtained from the plurality of to-be-recognized images.

[0039] Based on the target image, the second image recognition is performed.

[0040] In a possible implementation, the performing the first image recognition based on the plurality of to-be-recognized images to obtain a plurality of recognition result vectors corresponding to the plurality of to-be-recognized images comprises:

[0041] The plurality of to-be-recognized images are input into a preset image recognition model, wherein the preset image recognition model inputs an image and outputs a recognition result vector.

[0042] According to an output of the preset image recognition model, a plurality of recognition result vectors corresponding to the plurality of to-be-recognized images are obtained.

[0043] In a second aspect, an embodiment of the present application provides an image recognition device, the device comprising:

[0044] An image acquisition module is configured to acquire a plurality of to-be-recognized images of a preset target.

[0045] A first image recognition module is configured to perform a first image recognition based on the plurality of to-be-recognized images to obtain a plurality of recognition result vectors corresponding to the plurality of to-be-recognized images.

[0046] A quality determination module is configured to input the plurality of recognition result vectors into a preset image quality judgment model, wherein the image quality judgment model is configured to determine the quality of the plurality of to-be-recognized images according to vector distances between the plurality of recognition result vectors corresponding to the plurality of to-be-recognized images.

[0047] A second image recognition module is configured to perform a second image recognition according to the quality of the plurality of to-be-recognized images.

[0048] In a possible implementation, the quality determination module is further configured to:

[0049] A plurality of reference images are acquired, a first image recognition is performed on the plurality of reference images to obtain a plurality of recognition result vectors corresponding to the plurality of reference images.

[0050] A recognition result vector F ia plurality of reference images, wherein the vector distance of each of the plurality of reference images is determined according to a distance between the recognition result vector F i , and a vector distance of each of the remaining recognition result vectors, wherein the vector distance of the recognition result vector F i is a distance between the recognition result vector F and any one of the plurality of reference images, i = 1,..., N, N represents a number of vectors in the plurality of reference images corresponding to the plurality of reference images;

[0051] based on the vector distance, training the initial image quality judgment model, so that a value of a loss function of the trained initial image quality judgment model meets a preset requirement, wherein the value of the loss function is determined according to predicted qualities of the plurality of reference images and real qualities of the plurality of reference images, and the predicted qualities of the plurality of reference images are determined according to the vector distance;

[0052] obtaining the preset image quality judgment model according to the trained initial image quality judgment model.

[0053] In a possible implementation, the quality determination module is further configured to:

[0054] obtain a minimum distance from the vector distance;

[0055] determine whether the minimum distance is greater than a preset distance threshold;

[0056] if the minimum distance is less than or equal to the preset distance threshold, train the initial image quality judgment model based on the vector distance.

[0057] In a possible implementation, the quality determination module is specifically configured to:

[0058] obtain a minimum distance from the vector distance, and calculate a first difference value between a mean value of a pre-stored distance of a negative sample pair and a corresponding vector distance of the recognition result vector F i , and a second difference value between the mean value of the distance and the minimum distance;

[0059] train the initial image quality judgment model based on the first difference value and the second difference value, and the predicted qualities of the plurality of reference images are determined according to the first difference value and the second difference value.

[0060] In a possible implementation, the second image recognition module is further configured to:

[0061] perform image quality judgment evaluation based on the qualities of the plurality of to-be-recognized images;

[0062] if the evaluation passes, perform second image recognition according to the qualities of the plurality of to-be-recognized images.

[0063] In a possible implementation, the second image recognition module is specifically configured to:

[0064] determine, based on the quality of the plurality of images to be recognized, an image to be filtered from the plurality of images to be recognized;

[0065] determine, according to a positive sample image in the plurality of images to be recognized, a positive sample ratio before filtering, and determine, according to the image to be filtered, a filtering ratio;

[0066] determine, based on the positive sample ratio before filtering, a change curve of a positive sample ratio after filtering and the filtering ratio;

[0067] perform evaluation of the image quality judgment according to the change curve of the positive sample ratio after filtering and the filtering ratio.

[0068] In a possible implementation, the second image recognition module is specifically configured to:

[0069] obtain a pre-stored change curve of a positive sample ratio after filtering and a filtering ratio;

[0070] determine an evaluation index value according to the change curve of the positive sample ratio after filtering and the filtering ratio and the pre-stored change curve of the positive sample ratio after filtering and the filtering ratio;

[0071] if the evaluation index value is greater than a preset evaluation threshold, determine that the evaluation passes.

[0072] In a possible implementation, the second image recognition module is specifically configured to:

[0073] obtain a target image from the plurality of images to be recognized according to the quality of the plurality of images to be recognized and a preset quality requirement;

[0074] perform second image recognition based on the target image.

[0075] In a possible implementation, the first image recognition module is specifically configured to:

[0076] input the plurality of images to be recognized into a preset image recognition model, where the preset image recognition model inputs an image and outputs a recognition result vector;

[0077] obtain a plurality of recognition result vectors corresponding to the plurality of images to be recognized according to an output of the preset image recognition model.

[0078] In a third aspect, an embodiment of the present application provides an image recognition device, comprising:

[0079] a processor;

[0080] a memory; and

[0081] a computer program;

[0082] wherein the computer program is stored in the memory and configured to be executed by the processor, the computer program comprising instructions for performing the method according to the first aspect.

[0083] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program causing a server to perform the method according to the first aspect.

[0084] In a fifth aspect, an embodiment of the present application provides a computer program product comprising computer instructions for performing the method according to the first aspect when executed by a processor.

[0085] The image recognition method, device and equipment provided by the embodiments of the present application obtain a plurality of to-be-recognized images of a preset target, perform first image recognition on the plurality of to-be-recognized images to obtain a plurality of recognition result vectors, and then input the plurality of recognition result vectors into a preset image quality judgment model, wherein the image quality judgment model is used to determine the quality of the plurality of to-be-recognized images according to the vector distance between the plurality of recognition result vectors corresponding to the plurality of to-be-recognized images, so as to perform second image recognition according to the quality, that is, the quality of the to-be-recognized images is considered when performing image recognition, thereby solving the problems of missing detection and low accuracy of recognition results in the existing image recognition. Moreover, the embodiments of the present application improve the accuracy of image recognition, enable relevant personnel to timely and correctly process based on the recognition result, and meet the needs of actual application. BRIEF DESCRIPTION OF DRAWINGS

[0086] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0087] Figure 1 The image recognition system architecture schematic diagram provided by the embodiments of the present application;

[0088] Figure 2 The flowchart of the image recognition method provided by the embodiments of the present application;

[0089] Figure 3 The flowchart of another image recognition method provided by the embodiments of the present application;

[0090] Figure 4 A schematic diagram showing the change curves of the proportion of positive samples after filtering and the filtering proportion provided in the embodiments of this application;

[0091] Figure 5 This is a schematic diagram of the structure of an image recognition device provided in an embodiment of this application;

[0092] Figure 6 This is a schematic diagram of the basic hardware architecture of an image recognition device provided in an embodiment of this application. Detailed Implementation

[0093] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0094] The terms “first,” “second,” “third,” and “fourth,” etc. (if present), in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0095] In today's intelligent world, image recognition has been applied to multiple fields. Taking facial recognition as an example, it has been used in areas such as security, automotive, and finance. In the automotive field, many car models are equipped with this function to provide customized services for car owners. However, when cameras capture data, some images are not suitable for facial recognition, such as those that are too bright, too dark, or blurry, as well as those with occlusions or excessive head posture, resulting in poor facial image quality (these poor-quality images can be collectively referred to as low-quality images).

[0096] Extensive testing revealed that using low-quality images in subsequent facial recognition processes results in more false positives (FNs) and lower recognition accuracy. For example, in the field of vehicle driving, low-quality images lead to a lower accuracy rate in facial recognition of driver behaviors that are detrimental to safe driving, significantly increasing the probability of vehicle accidents.

[0097] To address the aforementioned issues, this application proposes an image recognition method that considers the quality of the image to be recognized and performs image recognition based on that quality. This solves the problems of missed detections and low accuracy in existing image recognition methods, thereby enabling relevant personnel to process the recognition results correctly and promptly to meet application needs.

[0098] Optionally, the image recognition method provided in this application embodiment can be applied to, for example... Figure 1 The image recognition system shown. In Figure 1 Taking the determination of whether a driver's gaze is normal through image recognition as an example, the image recognition system architecture may include a processing device 11 and multiple acquisition units. Here, the multiple acquisition units are exemplified by a first acquisition unit 12 and a second acquisition unit 13. The first acquisition unit 12 may be located in a first vehicle, and the second acquisition unit 13 may be located in a second vehicle. The first acquisition unit 12 acquires data about the driver in the first vehicle, such as a facial image of the driver. Similarly, the second acquisition unit 13 acquires data about the driver in the second vehicle. For example, the first acquisition unit 12 and the second acquisition unit 13 may be cameras.

[0099] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the image recognition architecture. In other feasible embodiments of this application, the above architecture may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components, which can be determined according to the actual application scenario and is not limited here. Figure 1 The components shown can be implemented in hardware, software, or a combination of both.

[0100] In specific implementation, the first acquisition unit 12 can acquire the facial image of the driver of the first vehicle after the driver of the first vehicle enters the vehicle and begins driving, and send the acquired image to the processing device 11. Similarly, the second acquisition unit 13 can acquire the facial image of the driver of the second vehicle after the driver of the second vehicle enters the vehicle and begins driving, and send the acquired image to the processing device 11.

[0101] After receiving the driver's face images sent by the first acquisition unit 12 and the second acquisition unit 13, the processing device 11 determines the quality of these face images, and then performs image recognition, i.e. face recognition, based on the image quality to determine whether the driver's line of sight in the first vehicle and the second vehicle is normal. The recognition result has a high accuracy rate, enabling relevant personnel to make correct processing based on the recognition result.

[0102] Furthermore, the aforementioned architecture may also include a reminder unit, which can remind the driver when it is determined that the driver's line of sight is abnormal. The reminder unit can be installed in the vehicle; for example, taking the aforementioned architecture as an example with two reminder units, one reminder unit can be installed in each of the first and second vehicles. The processing device 11 can determine the driver's status in the first and second vehicles based on the aforementioned facial recognition results. If it determines that the driver's line of sight in the first vehicle is abnormal, it can send a reminder message to the reminder unit in the first vehicle. The reminder unit in the first vehicle then reminds the driver based on the reminder message, for example, by playing the reminder message aloud.

[0103] The aforementioned architecture may also include a display unit, which can be used to display the driver's image and recognition results.

[0104] The display unit can also be a touch screen, used to receive user commands while displaying the above content, so as to realize interaction with the user.

[0105] It should be understood that the above-mentioned processing device can be implemented by a processor reading instructions from memory and executing those instructions, or it can be implemented by a chip circuit.

[0106] The above system is only an example system. In specific implementation, it can be set up according to application requirements.

[0107] It is understood that the system architecture described in the embodiments of this application is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0108] The technical solutions of this application are described below using several embodiments as examples. The same or similar concepts or processes may not be repeated in some embodiments.

[0109] Figure 2 This is a flowchart illustrating an image recognition method provided in an embodiment of this application. The execution entity of this embodiment can be... Figure 1 The processing device in the process can be specifically executed based on the actual application scenario, and this application embodiment does not impose any particular restrictions on this. For example Figure 2 As shown, the image recognition method provided in this application embodiment may include the following steps:

[0110] S201: Acquire multiple images of a preset target to be recognized.

[0111] The aforementioned preset targets can be determined based on actual circumstances, such as the above. Figure 1a driver in the first vehicle.

[0112] Here, the processing apparatus can acquire a plurality of to-be-recognized images of the preset target through an acquisition unit (e.g., a camera), for example, the number of the plurality of to-be-recognized images is N, and the plurality of to-be-recognized images can be represented as I1…I N .

[0113] S202: Based on the plurality of to-be-recognized images, first image recognition is performed to obtain a plurality of recognition result vectors corresponding to the plurality of to-be-recognized images.

[0114] For example, the processing apparatus can input the plurality of to-be-recognized images into a preset image recognition model, where the preset image recognition model inputs an image and outputs a recognition result vector, and then, according to the output of the preset image recognition model, the plurality of recognition result vectors corresponding to the plurality of to-be-recognized images are obtained.

[0115] For example, the processing apparatus can acquire an image recognition model with a higher usage frequency as the preset image recognition model, for example, an image recognition model with a usage frequency exceeding a preset frequency threshold is acquired as the preset image recognition model. Here, the preset frequency threshold can be determined according to actual conditions, for example, 100 times.

[0116] The processing apparatus inputs the images I1…I N into the preset image recognition model to obtain a plurality of recognition result vectors corresponding to the images I1…I N , which can be denoted as F1…F N .

[0117] S203: The plurality of recognition result vectors are input into a preset image quality judgment model, where the image quality judgment model is used to determine the quality of the plurality of to-be-recognized images according to the vector distances between the plurality of recognition result vectors corresponding to the plurality of to-be-recognized images.

[0118] In the embodiments of the present application, the processing apparatus can acquire a plurality of reference images, perform first image recognition on the plurality of reference images to obtain a plurality of recognition result vectors corresponding to the plurality of reference images, and then determine the vector distance between each recognition result vector and each of the remaining recognition result vectors, for example, the vector distance between the recognition result vector F i and each of the remaining recognition result vectors. Wherein, the vector distance between the recognition result vector F i and each of the remaining recognition result vectors is multiple, and the processing apparatus can calculate the average of the multiple vector distances, and take the average as the vector distance between the recognition result vector F i and each of the remaining recognition result vectors, which can be denoted as D i.

[0119] Further, the processing device can train the initial image quality judgment model based on the vector distances, so that a value of a loss function of the trained initial image quality judgment model satisfies a preset requirement, wherein the value of the loss function is determined according to predicted qualities of the plurality of reference images and true qualities of the plurality of reference images, and the predicted qualities of the plurality of reference images are determined according to the vector distances, so that the preset image quality judgment model is obtained according to the trained initial image quality judgment model.

[0120] The initial image quality judgment model outputs the predicted qualities of the plurality of reference images.

[0121] In addition, the processing device can obtain a minimum distance D i from the vector distances D min = min (D1, D2...D N ), and further determine whether the minimum distance is greater than a preset distance threshold. If the minimum distance is less than or equal to the preset distance threshold, the processing device can train the initial image quality judgment model based on the vector distances. If the minimum distance is greater than the preset distance threshold, it indicates that there is no image with particularly good quality in the plurality of reference images, or there is a case that the true value is incorrect, and the image can be discarded and not processed subsequently. min

[0122] The preset distance threshold can be determined according to actual conditions, for example, according to a minimum value of distances between recognition result vectors corresponding to a plurality of good quality images.

[0123] For example, when the processing device trains the initial image quality judgment model based on the vector distances, the processing device can obtain a minimum distance from the vector distances, and calculate a first difference value between a mean value of the distances and a vector distance D i corresponding to the recognition result vector F i , and a second difference value between the mean value of the distances and the minimum distance D min , so that the initial image quality judgment model is trained based on the first difference value and the second difference value, and the predicted qualities of the plurality of reference images are determined according to the first difference value and the second difference value.

[0124] For example, the processing device determines the predicted qualities Q i of the plurality of reference images through an expression:

[0125]

[0126] ​​represents the mean value of the pre-stored negative sample pair distance. Here, the mean value of the negative sample pair distance can be determined by obtaining the distance between the recognition result vectors corresponding to the plurality of negative samples. The negative sample can be understood as follows: the above-mentioned pre-set target is the above-mentioned Figure 1 For example, the driver in the first vehicle described in the above-mentioned embodiment is taken as an example, the positive sample is an image (such as a face image) of the driver, and the negative sample is not an image of the driver.

[0127] S204: According to the quality of the plurality of to-be-identified images, performing second image identification.

[0128] Here, after determining the quality of the plurality of to-be-identified images, the processing device can obtain a target image from the plurality of to-be-identified images according to the quality of the plurality of to-be-identified images and a pre-set quality requirement, so as to perform second image identification based on the target image, thereby improving the accuracy of the image identification result.

[0129] The pre-set quality requirement can be determined according to actual conditions, for example, the quality of the plurality of to-be-identified images is represented by a value Q i The pre-set quality requirement can be a value Q'. The processing device obtains an image with a value greater than Q' from the plurality of to-be-identified images as the target image, and then performs second image identification based on the target image, such as inputting the target image into the pre-set image identification model, and obtaining a plurality of recognition result vectors corresponding to the target image according to the output of the pre-set image identification model. i

[0130] After obtaining the plurality of to-be-identified images of the pre-set target, the embodiment of the present application performs first image identification on the plurality of to-be-identified images to obtain a plurality of recognition result vectors, and then inputs the plurality of recognition result vectors into a pre-set image quality judgment model, wherein the image quality judgment model is used to determine the quality of the plurality of to-be-identified images according to the vector distance between the plurality of recognition result vectors corresponding to the plurality of to-be-identified images, so as to perform second image identification according to the quality, that is, considering the quality of the to-be-identified image when performing image identification, thereby solving the problem of missing detection and low accuracy of the recognition result in the existing image identification. Moreover, the embodiment of the present application improves the accuracy of image identification, enables relevant personnel to timely and correctly process based on the recognition result, and meets the needs of actual application.

[0131] In addition, before performing second image identification according to the quality of the plurality of to-be-identified images, in order to ensure the accuracy of the determined quality of the plurality of to-be-identified images, the processing device also considers evaluating the image quality judgment based on the quality of the plurality of to-be-identified images, so as to perform subsequent operations only when the evaluation is passed, thereby further improving the accuracy of the image identification result. Figure 3 ​A flowchart of another image recognition method proposed for the embodiments of the present application is shown in FIG. 3. As shown in FIG. 3, the method comprises the following steps. Figure 3

[0132] S301: Obtain a plurality of to-be-recognized images of a preset target.

[0133] S302: Perform first image recognition based on the plurality of to-be-recognized images to obtain a plurality of recognition result vectors corresponding to the plurality of to-be-recognized images.

[0134] S303: Input the plurality of recognition result vectors into a preset image quality judgment model, wherein the image quality judgment model is configured to determine the quality of the plurality of to-be-recognized images according to the vector distance between the plurality of recognition result vectors corresponding to the plurality of to-be-recognized images.

[0135] The steps S301-S303 are the same as the implementation of the steps S201-S203, and thus will not be described here.

[0136] S304: Perform evaluation of image quality judgment based on the quality of the plurality of to-be-recognized images.

[0137] For example, the processing device can determine the images to be filtered from the plurality of to-be-recognized images based on the quality of the plurality of to-be-recognized images, and then determine the positive sample ratio before filtering and the filtering ratio based on the images to be filtered, so as to determine the change curve of the positive sample ratio after filtering and the filtering ratio based on the positive sample ratio before filtering, and perform evaluation of image quality judgment based on the change curve.

[0138] In the determination of the images to be filtered from the plurality of to-be-recognized images, the processing device can first determine the images whose quality does not meet the preset quality requirement, for example, Q i is less than or equal to Q', and then obtain the images to be filtered from the plurality of to-be-recognized images based on Q i is less than or equal to Q'.

[0139] In the evaluation of image quality judgment, the filtering of low-quality images will cause the change of the negative sample ratio, and thus the method of determining the threshold of positive and negative samples is used to compare the change of the positive sample ratio before and after filtering. For example, the processing device can determine the filtering ratio according to the expression:

[0140]

[0141] ​Determine the proportion of positive samples before filtering, t0, where tp represents the number of positive sample images in the above multiple images to be identified, and Rp represents the total number of sample images in the above multiple images to be identified.

[0142] Furthermore, the above-mentioned processing device is based on the expression:

[0143]

[0144] Determine the filtration ratio r, where F i-FN This indicates the number of images to be filtered.

[0145] The above processing device is based on the expression:

[0146]

[0147] Determine the curve y showing the change between the proportion of positive samples after filtering and the aforementioned filtering proportion.

[0148] Therefore, the above-mentioned processing device can perform the above-mentioned image quality judgment based on the curve y.

[0149] As can be seen from the curve above, y increases with the increase of the filtering ratio r, but the actual situation is not so ideal. Therefore, when the processing device evaluates the image quality based on the curve y, it considers both the ideal curve of the change between the positive sample ratio after filtering and the filtering ratio, and the curve of the change between the actual determined positive sample ratio after filtering and the filtering ratio. Based on these two curves, the evaluation index value is determined, and the image quality evaluation is completed.

[0150] For example, the above-mentioned processing device can acquire a pre-stored curve showing the change between the proportion of positive samples after filtering and the filtering ratio (the ideal curve showing the change between the proportion of positive samples after filtering and the filtering ratio), and then determine an evaluation index value based on the curve showing the change between the proportion of positive samples after filtering and the filtering ratio (the actual determined curve showing the change between the proportion of positive samples after filtering and the filtering ratio) and the pre-stored curve showing the change between the proportion of positive samples after filtering and the filtering ratio, and perform the above-mentioned image quality judgment based on the evaluation index value.

[0151] For example, Figure 4 As shown, the proportion of positive samples before filtering, t0, is 0.4. Curve 1 is a pre-stored curve showing the change between the proportion of positive samples after filtering and the filtering proportion (the ideal curve showing the change between the proportion of positive samples after filtering and the filtering proportion):

[0152]

[0153] Curve 2 shows the change curve between the proportion of positive samples after filtering and the filtering ratio (the actual determined change curve between the proportion of positive samples after filtering and the filtering ratio):

[0154]

[0155] Therefore, the processing apparatus determines an evaluation index value based on the two curves. If the evaluation index value is greater than a preset evaluation threshold, the processing apparatus determines that the evaluation passes. For example, the processing apparatus obtains a test index (area under the curve, AUC) ∈ [0, 1] under curve 1 as a denominator and under curve 2 as a numerator on the right side of r = 0 and on the left side of r = 1-t0 under y = t0. Figure 4 Given r = 0.6 and y = 0.4 as two boundaries, the area under curve 1 is as indicated by arrow 1, and the area under curve 2 is as indicated by arrow 2.

[0156] The preset evaluation threshold can be set according to actual conditions, such as 0.8. If the evaluation index value is greater than the preset evaluation threshold, the processing apparatus determines that the evaluation passes, that is, the image quality judgment is effective, and the second image recognition can be further performed according to the quality of the plurality of to-be-recognized images to improve the accuracy of image recognition.

[0157] S305: If the evaluation passes, performing second image recognition according to the quality of the plurality of to-be-recognized images.

[0158] The implementation of step S305 is the same as that of step S204, and will not be described here.

[0159] In the embodiments of the present application, before performing the second image recognition according to the quality of the plurality of to-be-recognized images, the processing apparatus further considers the evaluation of the image quality judgment based on the quality of the plurality of to-be-recognized images to ensure the accuracy of the determined quality of the plurality of to-be-recognized images, so that the subsequent operation is performed only when the evaluation passes, and the accuracy of the image recognition result is further improved. Moreover, the processing apparatus improves the accuracy of image recognition, enables relevant personnel to timely and correctly process based on the recognition result, and meets the needs of actual applications.

[0160] The image recognition method corresponding to the above embodiments, Figure 5 A structural schematic diagram of an image recognition apparatus provided in the embodiments of the present application is shown. For ease of illustration, only parts related to the embodiments of the present application are shown. Figure 5A structural schematic diagram of an image recognition device provided by an embodiment of the present application is shown in FIG. 1. The image recognition device 50 includes an image acquisition module 501, a first image recognition module 502, a quality determination module 503, and a second image recognition module 504. The image recognition device herein can be the processing device itself, or a chip or integrated circuit implementing the functions of the processing device. It should be noted that the division of the image acquisition module, the first image recognition module, the quality determination module, and the second image recognition module is only a logical division, and physically, the two can be integrated or independent.

[0161] The image acquisition module 501 is configured to acquire a plurality of to-be-recognized images of a preset target.

[0162] The first image recognition module 502 is configured to perform first image recognition based on the plurality of to-be-recognized images, and obtain a plurality of recognition result vectors corresponding to the plurality of to-be-recognized images.

[0163] The quality determination module 503 is configured to input the plurality of recognition result vectors into a preset image quality judgment model, where the image quality judgment model is configured to determine the quality of the plurality of to-be-recognized images according to the vector distances between the plurality of recognition result vectors corresponding to the plurality of to-be-recognized images.

[0164] The second image recognition module 504 is configured to perform second image recognition according to the quality of the plurality of to-be-recognized images.

[0165] In a possible implementation, the quality determination module 503 is further configured to:

[0166] acquire a plurality of reference images, perform first image recognition on the plurality of reference images, and obtain a plurality of recognition result vectors corresponding to the plurality of reference images;

[0167] determine a recognition result vector F i vector distance between each of the remaining recognition result vectors and the recognition result vector F i , where the recognition result vector F i is any one of the plurality of recognition result vectors corresponding to the plurality of reference images, i = 1,..., N, and N represents the number of vectors in the plurality of recognition result vectors corresponding to the plurality of reference images.

[0168] train the initial image quality judgment model based on the vector distances, so that a value of a loss function of the trained initial image quality judgment model meets a preset requirement, wherein the value of the loss function is determined according to predicted qualities of the plurality of reference images and real qualities of the plurality of reference images, and the predicted qualities of the plurality of reference images are determined according to the vector distances;

[0169] obtain the preset image quality judgment model according to the trained initial image quality judgment model.

[0170] In a possible implementation, the quality determination module 503 is specifically configured to:

[0171] obtain a minimum distance from the vector distances;

[0172] determine whether the minimum distance is greater than a preset distance threshold;

[0173] train the initial image quality judgment model based on the vector distances if the minimum distance is less than or equal to the preset distance threshold.

[0174] In a possible implementation, the quality determination module 503 is specifically configured to:

[0175] obtain a minimum distance from the vector distances, and calculate a first difference value between a mean value of pre-stored distances of negative samples and the recognition result vector F i a second difference value between the mean value and the minimum distance corresponding to the vector distance;

[0176] train the initial image quality judgment model based on the first difference value and the second difference value, and the predicted qualities of the plurality of reference images are determined according to the first difference value and the second difference value.

[0177] In a possible implementation, the second image recognition module 504 is specifically configured to:

[0178] perform image quality judgment evaluation based on the qualities of the plurality of to-be-recognized images;

[0179] perform second image recognition according to the qualities of the plurality of to-be-recognized images if the evaluation passes.

[0180] In a possible implementation, the second image recognition module 504 is specifically configured to:

[0181] determine images to be filtered from the plurality of to-be-recognized images based on the qualities of the plurality of to-be-recognized images;

[0182] According to the positive sample images in the plurality of to-be-identified images, a positive sample ratio before filtering is determined, and according to the to-be-filtered images, a filtering ratio is determined.

[0183] Based on the positive sample ratio before filtering, a change curve of a positive sample ratio after filtering and the filtering ratio is determined.

[0184] According to the change curve of the positive sample ratio after filtering and the filtering ratio, an evaluation of the image quality judgment is performed.

[0185] In a possible implementation, the second image identification module 504 is specifically configured to:

[0186] obtain a pre-stored change curve of the positive sample ratio after filtering and the filtering ratio;

[0187] According to the change curve of the positive sample ratio after filtering and the filtering ratio, and the pre-stored change curve of the positive sample ratio after filtering and the filtering ratio, an evaluation index value is determined.

[0188] If the evaluation index value is greater than a preset evaluation threshold, it is determined that the evaluation passes.

[0189] In a possible implementation, the second image identification module 504 is specifically configured to:

[0190] According to the quality of the plurality of to-be-identified images and a preset quality requirement, a target image is obtained from the plurality of to-be-identified images.

[0191] Based on the target image, a second image identification is performed.

[0192] In a possible implementation, the first image identification module 502 is specifically configured to:

[0193] input the plurality of to-be-identified images into a preset image identification model, wherein the preset image identification model inputs an image and outputs an identification result vector;

[0194] According to the output of the preset image identification model, a plurality of identification result vectors corresponding to the plurality of to-be-identified images are obtained.

[0195] The device provided by the embodiments of the present application can be used to execute the technical solutions of the above-mentioned method embodiments, and the implementation principles and technical effects are similar, which will not be described here in the embodiments of the present application.

[0196] Optionally, Figure 6 A possible basic hardware architecture schematic diagram of the image identification device described in the present application is schematically provided.

[0197] Referring to Figure 6The image recognition device includes at least one processor 601 and a communication interface 603. Further optionally, a memory 602 and a bus 604 can also be included.

[0198] The number of processors 601 in the image recognition device can be one or more, Figure 6 Only one processor 601 is shown. Optionally, the processor 601 can be a central processing unit (CPU), a graphics processing unit (GPU), or a digital signal processor (DSP). If the image recognition device has multiple processors 601, the types of the multiple processors 601 can be different or can be the same. Optionally, the multiple processors 601 of the image recognition device can also be integrated into a multi-core processor.

[0199] The memory 602 stores computer instructions and data; the memory 602 can store computer instructions and data required to implement the above-mentioned image recognition method provided by the present application, for example, the memory 602 stores instructions for implementing the steps of the above-mentioned image recognition method. The memory 602 can be any one or any combination of the following storage media: non-volatile memory (such as read-only memory (ROM), solid state disk (SSD), hard disk (HDD), optical disk), volatile memory.

[0200] The communication interface 603 can provide information input / output for the at least one processor. It can also include any one or any combination of the following devices: a network interface (such as an Ethernet interface), a wireless network card, and other devices with network access functions.

[0201] Optionally, the communication interface 603 can also be used for data communication between the image recognition device and other computing devices or terminals.

[0202] Further optionally, Figure 6 The bus 604 is represented by a thick line. The bus 604 can connect the processor 601 with the memory 602 and the communication interface 603. In this way, the processor 601 can access the memory 602 and also use the communication interface 603 to interact with other computing devices or terminals through the bus 604.

[0203] In the present application, the image recognition device executes computer instructions in the memory 602, so that the image recognition device implements the above-mentioned image recognition method provided by the present application, or so that the image recognition device deploys the above-mentioned image recognition device.

[0204] From the perspective of logical function division, an example is as follows: Figure 6As shown, the memory 602 can include an image acquisition module 501, a first image recognition module 502, a quality determination module 503, and a second image recognition module 504. The inclusion herein only relates to the instructions stored in the memory, which, when executed, can respectively implement the functions of the image acquisition module, the first image recognition module, the quality determination module, and the second image recognition module, without being limited to a physical structure.

[0205] In addition, the image recognition device described above can be used to perform the image recognition method described above, in addition to the image recognition method described above. Figure 6 In addition, the image recognition device described above can be used to perform the image recognition method described above, in addition to the image recognition method described above.

[0206] The present application provides a computer readable storage medium, which includes computer instructions, and the computer instructions instruct a computing device to execute the image recognition method provided by the present application.

[0207] The present application provides a computer program product, which includes computer instructions, and the computer instructions instruct a processor to execute the image recognition method described above.

[0208] The present application provides a chip, which includes at least one processor and a communication interface, and the communication interface provides information input and / or output for the at least one processor. Further, the chip can further include at least one memory, and the memory is used to store computer instructions. The at least one processor is used to call and run the computer instructions to execute the image recognition method provided by the present application.

[0209] In several embodiments provided by the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the described apparatus embodiments are merely schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0210] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0211] In addition, the various functional units in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of hardware plus software function units.

Claims

1. An image recognition method, characterized in that, include: Acquire multiple images of a preset target to be recognized; Based on the multiple images to be identified, a first image recognition is performed to obtain multiple recognition result vectors corresponding to the multiple images to be identified; Multiple reference images are acquired, and a first image recognition is performed on the multiple reference images to obtain multiple recognition result vectors corresponding to the multiple reference images; Determine the recognition result vector Among the multiple recognition result vectors corresponding to the multiple reference images, excluding the recognition result vector... In addition, the vector distance between the remaining recognition result vectors, wherein the recognition result vectors For any one of the multiple recognition result vectors corresponding to the multiple reference images, , This represents the number of vectors in the multiple recognition result vectors corresponding to the multiple reference images; Based on the vector distance, the initial image quality judgment model is trained so that the value of the loss function of the trained initial image quality judgment model meets the preset requirements. The value of the loss function is determined based on the predicted quality of the multiple reference images and the actual quality of the multiple reference images. The predicted quality of the multiple reference images is determined based on the vector distance. Based on the trained initial image quality judgment model, the preset image quality judgment model is obtained; The plurality of recognition result vectors are input into a preset image quality judgment model, wherein the image quality judgment model is used to determine the quality of the plurality of images to be recognized based on the vector distance between the plurality of recognition result vectors corresponding to the plurality of images to be recognized; A second image recognition is performed based on the quality of the multiple images to be recognized.

2. The method according to claim 1, characterized in that, Before training the initial image quality assessment model based on the vector distance, the method further includes: Obtain the minimum distance from the vector distances; Determine whether the minimum distance is greater than a preset distance threshold; The training of the initial image quality judgment model based on the vector distance includes: If the minimum distance is less than or equal to the preset distance threshold, then the initial image quality judgment model is trained based on the vector distance.

3. The method according to claim 1, characterized in that, The training of the initial image quality judgment model based on the vector distance includes: From the vector distances, obtain the minimum distance, and calculate the mean of the distances between the pre-stored negative sample pairs and the recognition result vector. The first difference between the corresponding vector distances, and the second difference between the mean of the distances and the minimum distance; The initial image quality judgment model is trained based on the first difference and the second difference, and the predicted quality of the multiple reference images is determined according to the first difference and the second difference.

4. The method according to any one of claims 1 to 3, characterized in that, Before performing a second image recognition based on the quality of the plurality of images to be recognized, the method further includes: Based on the quality of the multiple images to be identified, an image quality assessment is performed; The second image recognition based on the quality of the plurality of images to be recognized includes: If the evaluation passes, a second image recognition is performed based on the quality of the multiple images to be recognized.

5. The method according to claim 4, characterized in that, The evaluation of image quality based on the quality of the plurality of images to be identified includes: Based on the quality of the plurality of images to be identified, determine the images to be filtered from the plurality of images to be identified; Based on the positive sample images in the plurality of images to be identified, determine the proportion of positive samples before filtering, and based on the images to be filtered, determine the filtering proportion; Based on the proportion of positive samples before filtering, determine the curve of the change between the proportion of positive samples after filtering and the filtering proportion. The image quality judgment is evaluated based on the curve showing the change between the proportion of positive samples after filtering and the filtering proportion.

6. The method according to claim 5, characterized in that, The evaluation of image quality based on the change curve of the proportion of positive samples after filtering and the filtering proportion includes: Obtain the pre-stored curve showing the change between the proportion of positive samples after filtering and the filtering proportion; The evaluation index value is determined based on the curve of the change between the proportion of positive samples after filtering and the filtering proportion, as well as the pre-stored curve of the change between the proportion of positive samples after filtering and the filtering proportion. If the evaluation index value is greater than the preset evaluation threshold, the evaluation is deemed to have passed.

7. The method according to any one of claims 1 to 3, characterized in that, The second image recognition based on the quality of the plurality of images to be recognized includes: Based on the quality of the plurality of images to be identified and preset quality requirements, a target image is obtained from the plurality of images to be identified; A second image recognition is performed based on the target image.

8. An image recognition device, characterized in that, include: The image acquisition module is used to acquire multiple images of a preset target to be recognized; The first image recognition module is used to perform a first image recognition based on the plurality of images to be recognized, and obtain a plurality of recognition result vectors corresponding to the plurality of images to be recognized; The quality determination module is used to acquire multiple reference images, perform a first image recognition on the multiple reference images, and obtain multiple recognition result vectors corresponding to the multiple reference images; Determine the recognition result vector Among the multiple recognition result vectors corresponding to the multiple reference images, excluding the recognition result vector... In addition, the vector distance between the remaining recognition result vectors, wherein the recognition result vectors For any one of the multiple recognition result vectors corresponding to the multiple reference images, , This represents the number of vectors in the multiple recognition result vectors corresponding to the multiple reference images; Based on the vector distance, the initial image quality judgment model is trained so that the value of the loss function of the trained initial image quality judgment model meets the preset requirements. The value of the loss function is determined based on the predicted quality of the multiple reference images and the actual quality of the multiple reference images. The predicted quality of the multiple reference images is determined based on the vector distance. Based on the trained initial image quality judgment model, the preset image quality judgment model is obtained; The quality determination module is further configured to input the plurality of recognition result vectors into a preset image quality judgment model, wherein the image quality judgment model is configured to determine the quality of the plurality of images to be recognized based on the vector distance between the plurality of recognition result vectors corresponding to the plurality of images to be recognized; The second image recognition module is used to perform a second image recognition based on the quality of the plurality of images to be recognized.

9. An image recognition device, characterized in that, include: processor; Memory; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor, the computer program including instructions for performing the method as described in any one of claims 1-7.

Images