Image recognition method, device and equipment

Abstract

The present invention provides an image recognition method, device and apparatus. [Solution] In this method, after obtaining a plurality of images to be recognized of a predetermined target, a first image recognition is performed on the plurality of images to be recognized to obtain a plurality of recognition result vectors, and the plurality of recognition result vectors are input into a predetermined image quality judgment model for determining the quality of the plurality of images to be recognized based on the vector distance between the plurality of recognition result vectors corresponding to the plurality of images to be recognized, and a second image recognition is performed based on the quality. That is, the quality of the images to be recognized is taken into consideration when performing image recognition, solving the conventional problem that the accuracy rate of the recognition result is low due to the existence of detection misses in image recognition. In addition, in the embodiment of the present invention, the accuracy rate of image recognition is improved, so that the relevant parties can timely perform correct processing based on the recognition result to meet the actual application needs.

Description

Detailed Description of the Invention

[0001] (Related Applications) This application claims priority to a Chinese patent application bearing application number 202111625658.8 and titled "Image recognition method, apparatus and device" filed with the China Patent Office on December 28, 2021, the entire contents of which are incorporated herein by reference. [Technical field]

[0002] The present invention relates to the technical field of image processing, and in particular to an image recognition method, device and apparatus. [Background technology]

[0003] Image recognition technology can recognize relevant information of an object contained in an image, for example, by recognizing a person's face image, it is possible to identify some information of the person. Therefore, with economic development, the application of image recognition technology is becoming more and more widespread, for example, face recognition is being applied in fields such as crime prevention, in-vehicle, and finance.

[0004] Among them, many vehicle models in the automotive field are equipped with facial recognition functions to provide customized services to owners. For example, by identifying the driver's behavior through facial recognition, when it is determined that the driver's behavior is detrimental to the safe driving of the vehicle, it can take timely action to reduce the impact of the driver's behavior on the safe driving of the vehicle as much as possible.

[0005] However, in image recognition, e.g., face recognition, when capturing data by a camera, the captured face image has problems such as being too bright, too dark, blurred, etc., as well as problems such as occlusion and excessive head pose, resulting in poor face image quality (these poor quality images may be collectively referred to as low-quality images). In subsequent face recognition, the use of low-quality images may cause many false negatives (FN), and the accuracy rate of the recognition result is low, so that the relevant parties cannot perform correct processing based on the recognition result. For example, in the automotive field, the accuracy rate of detecting driver behavior that is unfavorable to the safe driving of the vehicle by face recognition is low, which significantly increases the probability of vehicle accidents. Summary of the Invention [Problem to be solved by the invention]

[0006] To solve the problems existing in the prior art, the present invention provides an image recognition method, apparatus and device. [Means for solving the problem]

[0007] In a first aspect, an embodiment of the present invention provides a method of image recognition, the method comprising: acquiring a plurality of images of a predetermined target to be recognized; performing a first image recognition based on the plurality of images to be recognized, and acquiring a plurality of recognition result vectors corresponding to the plurality of images to be recognized; inputting the plurality of recognition result vectors into a predetermined image quality judgment model for determining the quality of the plurality of images to be recognized based on vector distances between the plurality of recognition result vectors corresponding to the plurality of images to be recognized; and performing a second image recognition based on the quality of the plurality of images to be recognized.

[0008] In one possible implementation, before inputting the plurality of recognition result vectors into a predetermined image quality judgment model, acquiring a plurality of reference images, performing a first image recognition on the plurality of reference images, and acquiring a plurality of recognition result vectors corresponding to the plurality of reference images; A recognition result vector F, which is one of the plurality of recognition result vectors corresponding to the plurality of reference images. i (where i=1,...,N, and N represents the number of recognition result vectors corresponding to the reference images), and the recognition result vector F i determining a vector distance between each recognition result vector other than the vector training an initial image quality judgment model based on the vector distance, so that a value of a loss function of the trained initial image quality judgment model satisfies a predetermined requirement; and obtaining the predetermined image quality judgment model based on a trained initial image quality judgment model; The value of the loss function is determined based on predicted qualities of the plurality of reference images and actual qualities of the plurality of reference images, and the predicted qualities of the plurality of reference images are determined based on the vector distance.

[0009] In one possible implementation, before training an initial image quality judgment model based on the vector distance, obtaining a minimum distance among the vector distances; determining whether the minimum distance is greater than a predetermined distance threshold; The step of training an initial image quality judgment model based on the vector distance includes: If the minimum distance is less than or equal to the predetermined distance threshold, training the initial image quality judgment model based on the vector distance.

[0010] In one possible implementation, the step of training an initial image quality judgment model based on the vector distance includes: The minimum distance of the vector distances is obtained, and the average value of the negative sample pair distances stored in advance and the recognition result vector F i and a second difference between the average of the distances and the minimum distance; training the initial image quality judgment model based on the first difference and the second difference; A prediction quality of the plurality of reference images is determined based on the first difference and the second difference.

[0011] In one possible implementation, before performing a second image recognition based on the quality of the plurality of images to be recognized, evaluating an image quality judgment based on the quality of the plurality of images to be recognized; The step of performing a second image recognition based on the quality of the plurality of images to be recognized includes: If the evaluation is successful, a second image recognition is performed based on the quality of the plurality of images to be recognized.

[0012] In one possible implementation, the step of evaluating an image quality judgment based on the quality of the plurality of images to be recognized includes: Identifying an image to be filtered from among the plurality of images to be recognized based on a quality of the plurality of images to be recognized; determining a ratio of positive samples before filtering based on positive sample images among the plurality of images to be recognized, and determining a filtering ratio based on the images to be filtered; determining a rate of positive samples after filtering and a change curve of the filtering rate based on the rate of positive samples before filtering; and evaluating the image quality judgment based on the proportion of positive samples after filtering and the variation curve of the filtering proportion.

[0013] In one possible implementation, the evaluation of the image quality judgment based on the proportion of positive samples after filtering and the change curve of the filtering rate includes: Obtaining a pre-stored change curve of the proportion of positive samples after filtering and the filtering proportion; Identifying an evaluation index value based on the change curve of the ratio of positive samples after filtering and the filtering ratio, and the change curve of the ratio of positive samples after filtering and the filtering ratio stored in advance; and determining that the evaluation has passed if the evaluation index value is greater than a predetermined evaluation threshold.

[0014] In one possible implementation, the step of performing a second image recognition based on the quality of the plurality of images to be recognized includes: obtaining a target image from the plurality of images to be recognized based on the quality of the plurality of images to be recognized and a predetermined quality requirement; and performing a second image recognition based on the target image.

[0015] In one possible implementation form, the step of performing a first image recognition based on the plurality of images to be recognized and obtaining a plurality of recognition result vectors corresponding to the plurality of images to be recognized includes: inputting the plurality of images to be recognized into a predetermined image recognition model which inputs an image and outputs a recognition result vector; and obtaining a plurality of recognition result vectors corresponding to the plurality of images to be recognized based on an output of the predetermined image recognition model.

[0016] In a second aspect, an embodiment of the present invention provides an image recognition apparatus, comprising: an image capture module for capturing a plurality of images of a predetermined target to be recognized; a first image recognition module for performing a first image recognition based on the plurality of images to be recognized and obtaining a plurality of recognition result vectors corresponding to the plurality of images to be recognized; a quality identification module for inputting the plurality of recognition result vectors into a predetermined image quality judgment model for identifying a quality of the plurality of images to be recognized based on vector distances between a plurality of recognition result vectors corresponding to the plurality of images to be recognized; and a second image recognition module for performing a second round of image recognition based on qualities of the plurality of images to be recognized.

[0017] In one possible implementation, the quality identification module further comprises: acquiring a plurality of reference images, performing a first image recognition on the plurality of reference images, and acquiring a plurality of recognition result vectors corresponding to the plurality of reference images; A recognition result vector F, which is one of the plurality of recognition result vectors corresponding to the plurality of reference images. i (where i=1,...,N, and N represents the number of recognition result vectors corresponding to the reference images), and the recognition result vector F i A vector distance between each recognition result vector other than Training an initial image quality judgment model based on the vector distance so that a value of a loss function of the trained initial image quality judgment model satisfies a predetermined requirement; Obtaining the predetermined image quality judgment model based on a trained initial image quality judgment model; The value of the loss function is determined based on predicted qualities of the plurality of reference images and actual qualities of the plurality of reference images, and the predicted qualities of the plurality of reference images are determined based on the vector distance.

[0018] In one possible implementation, the quality identification module further comprises: Obtain a minimum distance among the vector distances; determining whether the minimum distance is greater than a predetermined distance threshold; If the minimum distance is less than or equal to the predetermined distance threshold, the initial image quality judgment model is trained based on the vector distance.

[0019] In one possible implementation, the quality identification module specifically comprises: obtain a minimum distance from among the vector distances, and calculate a first difference between an average value of the negative sample pair distances stored in advance and the vector distance corresponding to the recognition result vector, and a second difference between the average value of the distances and the minimum distance; training the initial image quality judgment model based on the first difference and the second difference; A prediction quality of the plurality of reference images is determined based on the first difference and the second difference.

[0020] In one possible implementation, the second image recognition module further comprises: evaluating image quality based on the quality of the plurality of images to be recognized; If the evaluation is successful, a second image recognition is performed based on the quality of the plurality of images to be recognized.

[0021] In one possible implementation form, the second image recognition module specifically includes: Identifying an image to be filtered from among the plurality of images to be recognized based on a quality of the plurality of images to be recognized; Identifying a ratio of positive samples before filtering based on a positive sample image among the plurality of images to be recognized, and identifying a filtering ratio based on the image to be filtered; determining a post-filtering positive sample ratio and a change curve of the filtering ratio based on the pre-filtering positive sample ratio; The image quality judgment is evaluated based on the proportion of positive samples after filtering and the variation curve of the filtering rate.

[0022] In one possible implementation form, the second image recognition module specifically includes: Obtain the pre-stored change curve of the proportion of positive samples after filtering and the filtering proportion; Identifying an evaluation index value based on the change curve of the ratio of positive samples after filtering and the filtering ratio, and the change curve of the ratio of positive samples after filtering and the filtering ratio stored in advance; If the evaluation index value is greater than a predetermined evaluation threshold, the evaluation is determined to pass.

[0023] In one possible implementation form, the second image recognition module specifically includes: Obtaining a target image from the plurality of images to be recognized according to the quality of the plurality of images to be recognized and a predetermined quality requirement; A second image recognition is performed based on the target image.

[0024] In one possible implementation, the first image recognition module specifically includes: The plurality of images to be recognized are input to a predetermined image recognition model which inputs an image and outputs a recognition result vector; Based on the output of the predetermined image recognition model, a plurality of recognition result vectors corresponding to the plurality of images to be recognized are obtained.

[0025] In a third aspect, an embodiment of the present invention provides an image recognition device, comprising: A processor; Memory, A computer program, including The computer program is stored in the memory and configured to be executed by the processor, the computer program including instructions for performing the image recognition method according to the first aspect.

[0026] In a fourth aspect, an embodiment of the present invention provides a computer readable storage medium having a computer program stored thereon, the computer program causing a server to implement the image recognition method according to the first aspect.

[0027] In a fifth aspect, an embodiment of the present invention provides a computer program product comprising computer instructions which, when executed by a processor, perform the image recognition method according to the first aspect. Effect of the Invention

[0028] In the image recognition method, device and equipment according to the embodiment of the present invention, after obtaining a plurality of images to be recognized of a predetermined target, a first image recognition is performed on the plurality of images to be recognized to obtain a plurality of recognition result vectors, and the plurality of recognition result vectors are input into a predetermined image quality judgment model for determining the quality of the plurality of images to be recognized based on the vector distance between the plurality of recognition result vectors corresponding to the plurality of images to be recognized, and a second image recognition is performed based on the quality. That is, the quality of the images to be recognized is taken into consideration when performing image recognition, solving the conventional problem that there are detection omissions in image recognition and the accuracy rate of the recognition result is low. In addition, the embodiment of the present invention improves the accuracy rate of image recognition, so that the relevant parties can timely perform correct processing based on the recognition result to meet the actual application needs.

[0029] In order to more clearly describe the technical solutions in the embodiments of the present invention or the prior art, the following briefly introduces the drawings necessary for use in the embodiments or the prior art. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can obtain other drawings from these drawings without labor equivalent to inventive step. [Brief description of the drawings]

[0030] [Figure 1]1 is a schematic diagram of the architecture of an image recognition system according to an embodiment of the present invention. [Diagram 2] 1 is a schematic flowchart of an image recognition method according to an embodiment of the present invention. [Diagram 3] 10 is a schematic flowchart of another image recognition method according to an embodiment of the present invention. [Figure 4] FIG. 2 is a schematic diagram showing the change curve of the proportion of positive samples after filtering and the filtering proportion according to an embodiment of the present invention; [Diagram 5] 1 is a structural schematic diagram of an image recognition device according to an embodiment of the present invention; [Figure 6] 1 is a schematic diagram of a basic hardware architecture of an image recognition device according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] In the following, the technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, but not all of the embodiments. All other embodiments made by those skilled in the art based on the embodiments of the present invention without labor worthy of inventive step are all included in the protection scope of the present invention.

[0032] The terms "first," "second," "third," and "fourth," etc. (if present) in the present specification and claims, as well as in the drawings, are not intended to describe a particular order or sequence, but are intended to distinguish between similar objects. Data used in this manner are interchangeable where appropriate, and the embodiments of the present application described herein may be performed in other orders than those shown or described herein. Also, "comprises" and "having" and any variations thereof are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units expressly recited, but may include other steps or units not expressly shown or inherent to those processes, methods, products, or apparatus.

[0033] Nowadays, as the world becomes more intelligent, image recognition is applied to multiple fields. Take face recognition for example. Face recognition is applied to security, in-vehicle, finance, and other fields. Many vehicles in the in-vehicle field are equipped with this function to provide customized services to their owners. However, when a camera captures data, some images are not suitable for face recognition, for example, they are too bright, too dark, blurred, occluded, or the head posture is too large, which leads to poor image quality of the face (these poor quality images may be collectively referred to as low-quality images).

[0034] It can be seen through a large amount of testing that when low-quality images are involved in subsequent face recognition, many FNs will be brought about, and the accuracy rate of the recognition result will be low. For example, in the field of vehicle driving, low-quality images will result in low accuracy rate of detecting driver behaviors that are unfavorable to the safe driving of the vehicle through face recognition, greatly increasing the probability of vehicle accidents.

[0035] In order to solve the above problems, an embodiment of the present invention provides an image recognition method, which considers the quality of an image to be recognized, performs image recognition based on the quality of the image to be recognized, solves the traditional problem of detection omissions in image recognition and low accuracy of recognition results, and enables relevant parties to take corrective action in a timely manner based on the recognition results, thereby meeting application needs.

[0036] Alternatively, the image recognition method according to the embodiment of the present invention can be applied to the image recognition system shown in FIG. 1. In FIG. 1, taking as an example whether the driver's line of sight is normal or not through image recognition, the image recognition system architecture may include a processing device 11 and a plurality of acquisition means. Here, the plurality of acquisition means are exemplified by a first acquisition means 12 and a second acquisition means 13. The first acquisition means 12 may be disposed in a first vehicle, and the second acquisition means 13 may be disposed in a second vehicle. The first acquisition means 12 acquires data of the driver in the first vehicle, for example, a face image of the driver. For the same reason, the second acquisition means 13 acquires data of the driver in the second vehicle. As an example, the first acquisition means 12 and the second acquisition means 13 may be cameras.

[0037] As can be understood, the structure shown in the embodiment of the present application is not a specific limitation of the image recognition architecture. In some other possible embodiments of the present invention, the above architecture may include more or less components than those shown, or may combine some components, divide some components, or perform different component arrangements, which may be specifically determined according to the actual application scenario, and is not limited here. The components shown in FIG. 1 may be implemented by hardware, software, or a combination of software and hardware.

[0038] In a specific implementation process, the first acquisition means 12 may acquire a facial image of the driver of the first vehicle after the driver of the first vehicle enters the vehicle and starts driving, and transmit the acquired image to the processing device 11. Similarly, the second acquisition means 13 may acquire a facial image of the driver of the second vehicle after the driver of the second vehicle enters the vehicle and starts driving, and transmit the acquired image to the processing device 11.

[0039] After receiving the facial images of the drivers transmitted from the first acquisition means 12 and the second acquisition means 13, the processing device 11 identifies the quality of these facial images, and performs image recognition, i.e., facial recognition, based on the image quality to identify whether the line of sight of the drivers on the first vehicle and the second vehicle is normal. Since the accuracy rate of the recognition result is high, the relevant parties perform correct processing based on the recognition result.

[0040] The architecture may further include a reminding means for reminding the driver when it is determined that the line of sight of the driver is not normal. The reminding means may be provided in the vehicle, and for example, in the example where the architecture includes two reminding means, one reminding means is provided in each of the first vehicle and the second vehicle. The processing device 11 may determine the driver states in the first vehicle and the second vehicle based on the face recognition result, and when it is determined that the line of sight of the driver in the first vehicle is not normal, it may transmit reminding information to the reminding means in the first vehicle. The reminding means in the first vehicle calls the driver's attention based on the reminding information, and for example, reproduces the reminding information by voice.

[0041] The architecture may further include a display means for displaying the driver's image, the recognition results, etc.

[0042] The display means may be a touch display screen for receiving user commands simultaneously with the displayed content so that interaction with the user can be achieved.

[0043] It should be understood that the processing device may be implemented by a processor reading instructions in a memory and executing the instructions, or may be implemented by a chip circuit.

[0044] The above system is an exemplary system, and specific implementations may be implemented according to application needs.

[0045] It is understood that the system architecture described in the embodiments of the present invention is intended to more clearly illustrate the technical solutions of the embodiments of the present invention, and is not intended to limit the technical solutions of the embodiments of the present invention. As those skilled in the art can appreciate, with the evolution of system architecture and the emergence of new service scenarios, the technical solutions of the embodiments of the present invention can be similarly applied to similar technical problems.

[0046] In the following, the technical solution of the present invention will be described by taking several embodiments as examples, and the description of the same or similar concepts or processes may be omitted in some embodiments.

[0047] Fig. 2 is a schematic flow chart of an image recognition method according to an embodiment of the present invention. The execution subject of this embodiment may be the processing device in Fig. 1, and a specific execution subject may be specified according to an actual application scene. This is not limited in the embodiment of the present invention. As shown in Fig. 2, the image recognition method according to the embodiment of the present invention may include the following steps S201 to S204.

[0048] In S201, a plurality of images of a predetermined target to be recognized are acquired. The predetermined target may be specified according to the actual situation, for example the driver in the first vehicle in FIG. Here, the processing device may acquire a plurality of images to be recognized of the predetermined target via an acquisition means (e.g., a camera). For example, the number of the plurality of images to be recognized is N, and the plurality of images to be recognized are I 1 ...I N It may be shown as follows.

[0049] In S202, a first image recognition is performed based on the plurality of images to be recognized, and a plurality of recognition result vectors corresponding to the plurality of images to be recognized are obtained.

[0050] For example, the processing device may input the plurality of images to be recognized into a predetermined image recognition model, where the predetermined image recognition model inputs images and outputs recognition result vectors, and the processing device obtains a plurality of recognition result vectors corresponding to the plurality of images to be recognized based on the output of the predetermined image recognition model.

[0051] The processing device may acquire an image recognition model that has been used frequently as the predetermined image recognition model, for example, an image recognition model whose number of uses exceeds a predetermined number threshold as the predetermined image recognition model, where the predetermined number threshold may be specified according to an actual situation, for example, 100 times.

[0052] The processing device processes the image I 1 ...I N is input to the above-mentioned predetermined image recognition model, and the above-mentioned image I 1 ...I N A plurality of recognition result vectors corresponding to F are obtained, and the plurality of recognition results are 1 ...F N It may also be written as:

[0053] In S203, the plurality of recognition result vectors are inputted into a predetermined image quality judgment model, and the image quality judgment model is used to identify the quality of the plurality of images to be recognized based on the vector distances between the plurality of recognition result vectors corresponding to the plurality of images to be recognized.

[0054] In an embodiment of the present invention, the processing device obtains a plurality of reference images, performs a first image recognition on the plurality of reference images, obtains a plurality of recognition result vectors corresponding to the plurality of reference images, and then determines a vector distance between each recognition result vector and each remaining recognition result vector, for example, a recognition result vector F i The vector distance between the recognition result vector F and each of the remaining recognition result vectors is determined. i There are a plurality of vector distances between the vector F and each of the remaining recognition result vectors, and the processing device calculates an average value of the plurality of vector distances and sets the average value to the recognition result vector F. i and the vector distance between each remaining recognition result vector, and i It may also be written as:

[0055] Furthermore, the processing device may train an initial image quality judgment model based on the vector distance, such that a value of a loss function of the trained initial image quality judgment model satisfies a predetermined requirement, where the value of the loss function is determined based on the predicted qualities of the plurality of reference images and the actual qualities of the plurality of reference images, and the predicted qualities of the plurality of reference images are determined based on the vector distance, thereby obtaining the predetermined image quality judgment model based on the trained initial image quality judgment model.

[0056] The initial image quality judgment model outputs a predicted quality of the plurality of reference images.

[0057] The processing device also calculates the vector distance D i The minimum distance D min =min(D 1 ,D 2 ...D N ), and further determine whether the minimum distance is greater than a predetermined distance threshold. If the minimum distance is less than or equal to the predetermined distance threshold, the processing device may train the initial image quality judgment model based on the vector distance. minIf is greater than the predetermined distance threshold, it means that none of the reference images has very good quality, or the true value is incorrect, the image may be discarded without further processing.

[0058] The predetermined distance threshold may be specified according to practical circumstances, for example, based on the minimum distance between the recognition result vectors corresponding to a number of good quality images.

[0059] For example, when training the initial image quality judgment model based on the vector distances, the processing device obtains the minimum distance among the vector distances, and calculates an average value of the distances and the recognition result vector F i The vector distance D corresponding to i The first difference between the average distance and the minimum distance D min and a second difference between the first and second differences, thereby training the initial image quality judgment model based on the first and second differences, and a predicted quality of the plurality of reference images is determined based on the first and second differences.

[0060] For example, the processing device may include:

number

number

[0061] In S204, a second image recognition is performed based on the quality of the plurality of images to be recognized.

[0062] Here, after the processing device identifies the qualities of the plurality of images to be recognized, it may obtain a target image from the plurality of images to be recognized according to the qualities of the plurality of images to be recognized and a predetermined quality requirement, thereby performing a second image recognition based on the target image, and improving the accuracy rate of the image recognition result.

[0063] The predetermined quality requirement may be specified according to the actual situation, for example, the quality of the plurality of images to be recognized is a value Q i and the predetermined quality requirement may be a value Q′. The processing device determines, from the plurality of images to be recognized, Q i After obtaining an image for which Q' is greater than Q' as the target image, a second image recognition is performed based on the target image. For example, the target image is input to the specified image recognition model, and a plurality of recognition result vectors corresponding to the target image are obtained based on the output of the specified image recognition model.

[0064] In the embodiment of the present invention, after obtaining a plurality of images to be recognized of a predetermined target, a first image recognition is performed on the plurality of images to be recognized to obtain a plurality of recognition result vectors, and the plurality of recognition result vectors are input into a predetermined image quality judgment model, where the image quality judgment model is used to determine the quality of the plurality of images to be recognized according to the vector distance between the plurality of recognition result vectors corresponding to the plurality of images to be recognized, and then a second image recognition is performed according to the quality. That is, the quality of the images to be recognized is taken into consideration when performing image recognition, solving the conventional problem that there are detection omissions in image recognition and the accuracy rate of the recognition result is low. In addition, the embodiment of the present invention improves the accuracy rate of image recognition, so that the relevant parties can timely perform correct processing according to the recognition result to meet the actual application needs.

[0065] In addition, the processing device also considers evaluating image quality judgment based on the quality of the plurality of images to be recognized to ensure the accuracy of the identified quality of the plurality of images to be recognized before performing the second image recognition based on the quality of the plurality of images to be recognized. This further improves the accuracy rate of the image recognition result because the subsequent operation is executed only when the evaluation is passed. Figure 3 is a schematic flow chart of another image recognition method according to an embodiment of the present invention. As shown in Figure 3, the method includes steps S301 to S305.

[0066] In S301, a plurality of images of a predetermined target to be recognized are acquired.

[0067] In S302, a first image recognition is performed based on the plurality of images to be recognized, and a plurality of recognition result vectors corresponding to the plurality of images to be recognized are obtained.

[0068] In S303, the plurality of recognition result vectors are input to a predetermined image quality judgment model for identifying quality of the plurality of images to be recognized based on vector distances between the plurality of recognition result vectors corresponding to the plurality of images to be recognized.

[0069] Steps S301-S303 are implemented in the same manner as steps S201-S203 above, and will not be described here repeatedly.

[0070] In S304, image quality is evaluated based on the quality of the plurality of images to be recognized.

[0071] As an example, the processing device may identify an image to be filtered from among the plurality of images to be recognized based on the quality of the plurality of images to be recognized, and further identify a proportion of positive samples before filtering based on a positive sample image from among the plurality of images to be recognized, and identify a filtering proportion based on the image to be filtered, thereby identifying a change curve of the proportion of positive samples after filtering and the filtering proportion based on the proportion of positive samples before filtering, and evaluating image quality judgment based on the change curve.

[0072] Here, when identifying an image to be filtered from among the plurality of images to be recognized, the processing device identifies an image, for example, Q i First, identify those whose Q' is equal to or smaller than Q', and then, i may be obtained as an image to be filtered from among the plurality of images to be recognized based on which the image is smaller than or equal to Q′.

[0073] Here, when the processing device performs the image quality judgment evaluation, since the filtering of low-quality images will cause a change in the proportion of negative samples, the evaluation adopts a method of plotting a positive and negative sample judgment threshold value, and compares the change in the proportion of positive samples before and after filtering. For example, the processing device:

number

[0074] Furthermore, the processing device includes:

number

[0075] The processing device includes:

number

[0076] This enables the processing device to perform the evaluation of the image quality judgment based on the curve y.

[0077] From the above curve, it can be seen that y increases with the increase of filtering ratio r, but the actual situation is not so ideal. Therefore, when the processing device performs the evaluation of the image quality judgment based on the curve y, it considers the ideal post-filtering positive sample ratio and filtering ratio change curve and the actually determined post-filtering positive sample ratio and filtering ratio change curve. Thus, it determines the evaluation index value based on these two curves to complete the evaluation of the image quality judgment.

[0078] As an example, the processing device acquires a pre-stored change curve of the proportion of positive samples after filtering and the filtering ratio (the ideal change curve of the proportion of positive samples after filtering and the filtering ratio), and further identifies an evaluation index value based on the change curve of the proportion of positive samples after filtering and the filtering ratio (the actually identified change curve of the proportion of positive samples after filtering and the filtering ratio) and the pre-stored change curve of the proportion of positive samples after filtering and the filtering ratio, and evaluates the image quality judgment based on the evaluation index value.

[0079] For example, as shown in Figure 4, the proportion of positive samples before filtering, t 0is 0.4, and curve 1 is a pre-stored curve of change in the ratio of positive samples after filtering and the filtering ratio (the ideal curve of change in the ratio of positive samples after filtering and the filtering ratio).

number

[0080] Curve 2 is a curve of change in the proportion of positive samples and the filtering ratio after the filtering (a curve of change in the proportion of positive samples and the filtering ratio after the filtering actually identified),

number

[0081] The processing device then determines an evaluation index value based on the two curves. If the evaluation index value is greater than a predetermined evaluation threshold, the processing device determines that the evaluation has passed. For example, the processing device may determine that y=t 0 Below, to the right of r=0, r=1-t 0 On the left side of the curve, we take the area under the curve 1 as the denominator and the area under the curve 2 as the numerator to obtain the test index (area under the curve, AUC) ∈ [0,1]. Here, Figure 4 gives two bounds of r=0.6, y=0.4, where the area under the curve 1 is pointed to by the arrow 1, and the area under the curve 2 is pointed to by the arrow 2.

[0082] The predetermined evaluation threshold can be set according to actual circumstances, for example, 0.8. If the evaluation index value is greater than the predetermined evaluation threshold, the processing device determines that the evaluation is passed, that is, evaluates that the image quality judgment is valid, and may further perform a second image recognition according to the quality of the plurality of images to be recognized, thereby improving the accuracy rate of image recognition.

[0083] In S305, if the evaluation is successful, a second image recognition is performed based on the quality of the plurality of images to be recognized.

[0084] Step S305 and the above step S204 are implemented in the same manner, and will not be described here repeatedly.

[0085] In an embodiment of the present invention, the processing device also considers evaluating image quality judgment based on the quality of the plurality of images to be recognized before performing the second image recognition based on the quality of the plurality of images to be recognized, so as to ensure the accuracy of the quality of the identified plurality of images to be recognized. In this way, only when the evaluation is passed, the subsequent operation is performed, so that the accuracy rate of the image recognition result is further improved. In addition, the processing device improves the accuracy rate of image recognition, so that the relevant parties can timely perform correct processing based on the recognition result to meet the actual application needs.

[0086] Corresponding to the image recognition method of the above embodiment, FIG. 5 is a structural schematic diagram of an image recognition device according to an embodiment of the present invention. For convenience of explanation, only parts related to the embodiment of the present invention are shown. FIG. 5 is a structural schematic diagram of an image recognition device according to an embodiment of the present invention. The image recognition device 50 includes an image acquisition module 501, a first image recognition module 502, a quality identification module 503 and a second image recognition module 504. The image recognition device here may be the above processing device itself, or a chip or integrated circuit that performs the function of the processing device. It should be explained here that the division of the image acquisition module, the first image recognition module, the quality identification module and the second image recognition module is merely a division of logical functions, and the two may be physically integrated or independent of each other.

[0087] The image acquisition module 501 acquires a number of images of a given target to be recognized.

[0088] The first image recognition module 502 performs a first image recognition based on the plurality of images to be recognized, and obtains a plurality of recognition result vectors corresponding to the plurality of images to be recognized.

[0089] The quality determination module 503 inputs the plurality of recognition result vectors into a predetermined image quality determination model for determining the quality of the plurality of images to be recognized based on the vector distance between the plurality of recognition result vectors corresponding to the plurality of images to be recognized.

[0090] The second image recognition module 504 performs a second image recognition based on the quality of the plurality of images to be recognized.

[0091] In one possible implementation, the quality determination module 503 further obtains a plurality of reference images, performs a first image recognition on the plurality of reference images, obtains a plurality of recognition result vectors corresponding to the plurality of reference images, a recognition result vector F that is any one of the plurality of recognition result vectors corresponding to the plurality of reference images i (where i = 1,..., N, and N represents the number of the plurality of recognition result vectors corresponding to the plurality of reference images), and the recognition result vector F among the plurality of recognition result vectors corresponding to the plurality of reference images i identifies the vector distance from each recognition result vector other than it, trains an initial image quality determination model based on the vector distance such that the value of the loss function of the trained initial image quality determination model meets a predetermined requirement, obtains the predetermined image quality determination model based on the trained initial image quality determination model, The value of the loss function is determined based on the predicted quality and the actual quality of the plurality of reference images, and the predicted quality of the plurality of reference images is determined based on the vector distance.

[0092] In one possible implementation, the quality determination module 503 further obtains the minimum distance among the vector distances, determines whether the minimum distance is greater than a predetermined distance threshold, If the minimum distance is less than or equal to the predetermined distance threshold, the initial image quality judgment model is trained based on the vector distance.

[0093] In one possible implementation, the quality identification module 503 specifically includes: obtain a minimum distance from among the vector distances, and calculate a first difference between an average value of the negative sample pair distances stored in advance and the vector distance corresponding to the recognition result vector, and a second difference between the average value of the distances and the minimum distance; The initial image quality judgment model is trained based on the first difference and the second difference, and a predicted quality of the plurality of reference images is determined based on the first difference and the second difference.

[0094] In one possible implementation, the second image recognition module 504 further comprises: evaluating image quality based on the quality of the plurality of images to be recognized; If the evaluation is successful, a second image recognition is performed based on the quality of the plurality of images to be recognized.

[0095] In one possible implementation, the second image recognition module 504 specifically: Identifying an image to be filtered from among the plurality of images to be recognized based on a quality of the plurality of images to be recognized; Identifying a ratio of positive samples before filtering based on a positive sample image among the plurality of images to be recognized, and identifying a filtering ratio based on the image to be filtered; determining a post-filtering positive sample ratio and a change curve of the filtering ratio based on the pre-filtering positive sample ratio; The image quality judgment is evaluated based on the proportion of positive samples after filtering and the variation curve of the filtering rate.

[0096] In one possible implementation, the second image recognition module 504 specifically: Obtain the pre-stored change curve of the proportion of positive samples after filtering and the filtering proportion; Identifying an evaluation index value based on the change curve of the ratio of positive samples after filtering and the filtering ratio, and the change curve of the ratio of positive samples after filtering and the filtering ratio stored in advance; If the evaluation index value is greater than a predetermined evaluation threshold, the evaluation is determined to pass.

[0097] In one possible implementation, the second image recognition module 504 specifically: Obtaining a target image from the plurality of images to be recognized according to the quality of the plurality of images to be recognized and a predetermined quality requirement; A second image recognition is performed based on the target image.

[0098] In one possible implementation, the first image recognition module 502 specifically: The plurality of images to be recognized are input to a predetermined image recognition model which inputs an image and outputs a recognition result vector; Based on the output of the predetermined image recognition model, a plurality of recognition result vectors corresponding to the plurality of images to be recognized are obtained.

[0099] The apparatus according to the embodiments of the present invention can implement the technical solutions of the above method embodiments, and the implementation principles and technical effects are similar, so they will not be described repeatedly in the embodiments of the present invention.

[0100] Advantageously, FIG. 6 provides a schematic diagram of a possible basic hardware architecture of an image recognition device of the present invention.

[0101] Referring to FIG. 6, the image recognition device may include at least one processor 601 and a communication interface 603, and more preferably, a memory 602 and a bus 604.

[0102] Here, in the image recognition device, the number of processors 601 may be one or more. FIG. 6 simply shows one of the processors 601 in schematic form. Optionally, the processor 601 may be a central processing unit (CPU), a graphics processing unit (GPU), or a digital signal processor (DSP). When the image recognition device has multiple processors 601, the types of the multiple processors 601 may be different or the same. Optionally, the multiple processors 601 of the image recognition device may be integrated as a multi-core processor.

[0103] The memory 602 stores computer instructions and data, and may store computer instructions and data necessary to implement the image recognition method of the present invention, for example, the memory 602 stores instructions for implementing the steps of the image recognition method. The memory 602 may be any one or any combination of non-volatile memory (e.g., read-only memory (ROM), solid-state drive (SSD), hard disk (HDD), optical disk), volatile memory.

[0104] The communication interface 603 can provide input and output of information to at least one processor, and may include any one or any combination of devices having a network access function, such as a network interface (e.g., an Ethernet interface), a wireless network card, etc.

[0105] Optionally, the communication interface 603 may be further configured to enable the image recognition device to perform data communications with other computing devices or terminals.

[0106] More selectively, Fig. 6 shows a bus 604 in bold. The bus 604 can connect the processor 601 to the memory 602 and the communication interface 603. In this way, via the bus 604, the processor 601 can access the memory 602, and can also use the communication interface 603 to perform data interaction with other computing devices or terminals.

[0107] In the present invention, the image recognition device executes the computer instructions in the memory 602 to cause the image recognition device to perform the image recognition method according to the present invention, or the image recognition device is equipped with the image recognition device described above.

[0108] From the perspective of logical function division, as an example, as shown in Fig. 6, the memory 602 may include an image acquisition module 501, a first image recognition module 502, a quality identification module 503, and a second image recognition module 504. The term "include" here refers only to the functions of the image acquisition module, the first image recognition module, the quality identification module, and the second image recognition module that are respectively realized when the instructions stored in the memory are executed, and is not limited to the physical structure.

[0109] The image recognition device described above may be implemented by software as shown in FIG. 6, or may be implemented by hardware as a hardware module or a circuit unit.

[0110] The present invention provides a computer readable storage medium, the computer readable storage medium comprising computer instructions for directing a computing device to perform the image recognition method according to the present application.

[0111] The present invention provides a computer program product, comprising computer instructions which, when executed by a processor, perform the image recognition method described above.

[0112] The present invention provides a chip, the chip including at least one processor and a communication interface, the communication interface providing information input and / or output to the at least one processor, the chip may further include at least one memory for storing computer instructions, the at least one processor configured to call and execute the computer instructions to perform the image recognition method according to the present invention.

[0113] It should be understood that in some embodiments of the present invention, the disclosed apparatus and method may be realized in other ways. For example, the above-mentioned apparatus embodiments are merely schematic, for example, the division of the means is only a division of logical functions, and in actual implementation, there may be other division ways. For example, several means or units may be combined or integrated into another system, or some features may be omitted or not implemented. Meanwhile, the shown or discussed mutual couplings or direct couplings or communication connections may be indirect couplings or communication connections via some interfaces, devices or means, which may be electrical, mechanical or other forms.

[0114] The means described as separate components may or may not be physically separated, and the components shown as means may or may not be physical means, i.e., they may be located in one place or distributed across multiple network means. According to actual needs, some or all of the means can be selected to achieve the purpose of the solution of the embodiment.

[0115] In addition, each functional means in each embodiment of the present invention may be integrated into one processing means, each means may exist physically alone, or two or more means may be integrated into one means. The integrated means may be implemented as hardware, or may be implemented in the form of hardware plus software functional means.

Claims

1. 1. An image recognition method, comprising: acquiring a plurality of images of a predetermined target to be recognized; performing a first image recognition based on the plurality of images to be recognized, and acquiring a plurality of recognition result vectors corresponding to the plurality of images to be recognized; inputting the plurality of recognition result vectors into a predetermined image quality judgment model for determining the quality of the plurality of images to be recognized based on vector distances between the plurality of recognition result vectors corresponding to the plurality of images to be recognized; and performing a second image recognition based on the quality of the plurality of images to be recognized; before inputting the plurality of recognition result vectors into a predetermined image quality judgment model, acquiring a plurality of reference images, performing a first image recognition on the plurality of reference images, and acquiring a plurality of recognition result vectors corresponding to the plurality of reference images; A recognition result vector F i (where i=1 , . . . , N, where N represents the number of recognition result vectors corresponding to the plurality of reference images, and determining a vector distance between each of the recognition result vectors other than the recognition result vector F i among the plurality of recognition result vectors corresponding to the plurality of reference images; training the initial image quality judgment model based on the vector distance so that a value of a loss function of the trained initial image quality judgment model satisfies a predetermined requirement; and obtaining the predetermined image quality judgment model based on the trained initial image quality judgment model, The image recognition method, characterized in that the value of the loss function is determined based on predicted quality of the multiple reference images and actual quality of the multiple reference images, and the predicted quality of the multiple reference images is determined based on the vector distance.

2. An image recognition method, comprising: acquiring a plurality of images of a predetermined target to be recognized; performing a first image recognition based on the plurality of images to be recognized, and acquiring a plurality of recognition result vectors corresponding to the plurality of images to be recognized; inputting the plurality of recognition result vectors into a predetermined image quality judgment model for determining the quality of the plurality of images to be recognized based on vector distances between the plurality of recognition result vectors corresponding to the plurality of images to be recognized; and performing a second image recognition based on the quality of the plurality of images to be recognized; before performing a second image recognition based on the quality of the plurality of images to be recognized, evaluating an image quality judgment based on the quality of the plurality of images to be recognized; The step of performing a second image recognition based on the quality of the plurality of images to be recognized includes: and if the evaluation is successful, performing a second round of image recognition based on the quality of the plurality of images to be recognized.

3. Before training the initial image quality judgment model based on the vector distance, obtaining a minimum distance among the vector distances; determining whether the minimum distance is greater than a predetermined distance threshold; The step of training the initial image quality judgment model based on the vector distance includes:

2. The method of claim 1, further comprising training the initial image quality judgment model based on the vector distance if the minimum distance is less than or equal to the predetermined distance threshold.

4. The step of training an initial image quality judgment model based on the vector distance includes: The minimum distance of the vector distances is obtained, and the average value of the negative sample pair distances stored in advance and the recognition result vector F i and a second difference between the average of the pre-stored negative sample pair distances and the minimum distance; training the initial image quality judgment model based on the first difference and the second difference; 2. The image recognition method according to claim 1, wherein the prediction qualities of the plurality of reference images are determined based on the first difference and the second difference.

5. The step of evaluating image quality judgment based on the quality of the plurality of images to be recognized includes: Identifying an image to be filtered from among the plurality of images to be recognized based on a quality of the plurality of images to be recognized; determining a ratio of positive samples before filtering based on positive sample images among the plurality of images to be recognized, and determining a filtering ratio based on the images to be filtered; determining a rate of positive samples after filtering and a change curve of the filtering rate based on the rate of positive samples before filtering; 3. The method of claim 2, further comprising: evaluating the image quality judgment based on the proportion of positive samples after the filtering and a change curve of the filtering proportion.

6. The evaluation of the image quality judgment based on the proportion of positive samples after filtering and the change curve of the filtering rate is performed by: Obtaining a pre-stored change curve of the proportion of positive samples after filtering and the filtering proportion; Identifying an evaluation index value based on the change curve of the ratio of positive samples after filtering and the filtering ratio, and the change curve of the ratio of positive samples after filtering and the filtering ratio stored in advance; 6. The image recognition method according to claim 5, further comprising: determining that the evaluation has passed if the evaluation index value is greater than a predetermined evaluation threshold value.

7. The step of performing a second image recognition based on the quality of the plurality of images to be recognized includes: obtaining a target image from the plurality of images to be recognized based on the quality of the plurality of images to be recognized and a predetermined quality requirement; 3. The image recognition method according to claim 1, further comprising: performing a second image recognition based on the target image.

8. An image recognition device, an image capture module for capturing a plurality of images of a predetermined target to be recognized; a first image recognition module for performing a first image recognition based on the plurality of images to be recognized and obtaining a plurality of recognition result vectors corresponding to the plurality of images to be recognized; a quality identification module for inputting the plurality of recognition result vectors into a predetermined image quality judgment model for identifying a quality of the plurality of images to be recognized based on vector distances between a plurality of recognition result vectors corresponding to the plurality of images to be recognized; a second image recognition module for performing a second image recognition based on the quality of the plurality of images to be recognized; The quality identification module further comprises: acquiring a plurality of reference images, performing a first image recognition on the plurality of reference images, and acquiring a plurality of recognition result vectors corresponding to the plurality of reference images; A recognition result vector F i (where i=1 , . . . , N, where N represents the number of multiple recognition result vectors corresponding to the multiple reference images, and a vector distance between each recognition result vector other than the recognition result vector F i among the multiple recognition result vectors corresponding to the multiple reference images; Training the initial image quality judgment model based on the vector distance so that a value of a loss function of the trained initial image quality judgment model satisfies a predetermined requirement; Obtaining the predetermined image quality judgment model based on the trained initial image quality judgment model; an image recognition device characterized in that the value of the loss function is determined based on predicted quality of the multiple reference images and actual quality of the multiple reference images, and the predicted quality of the multiple reference images is determined based on the vector distance.

9. An image recognition device, an image capture module for capturing a plurality of images of a predetermined target to be recognized; a first image recognition module for performing a first image recognition based on the plurality of images to be recognized and obtaining a plurality of recognition result vectors corresponding to the plurality of images to be recognized; a quality identification module for inputting the plurality of recognition result vectors into a predetermined image quality judgment model for identifying a quality of the plurality of images to be recognized based on vector distances between a plurality of recognition result vectors corresponding to the plurality of images to be recognized; a second image recognition module for performing a second image recognition based on the quality of the plurality of images to be recognized; The second image recognition module further comprises: evaluating image quality based on the quality of the plurality of images to be recognized; performing a second image recognition based on the quality of the plurality of images to be recognized, and if the evaluation is successful, performing a second round of image recognition based on the quality of the plurality of images to be recognized.

10. The quality identification module further comprises: Obtain a minimum distance among the vector distances; determining whether the minimum distance is greater than a predetermined distance threshold; Training the initial image quality judgment model based on the vector distance includes: The image recognition apparatus of claim 8 , further comprising: training the initial image quality judgment model based on the vector distance if the minimum distance is less than or equal to the predetermined distance threshold.

11. The quality identification module includes: The minimum distance of the vector distances is obtained, and the average value of the negative sample pair distances stored in advance and the recognition result vector F i and a second difference between the average value of the pre-stored negative sample pair distances and the minimum distance; training the initial image quality judgment model based on the first difference and the second difference; 9. The image recognition device according to claim 8, wherein the prediction qualities of the plurality of reference images are determined based on the first difference and the second difference.

12. The second image recognition module is Identifying an image to be filtered from among the plurality of images to be recognized based on a quality of the plurality of images to be recognized; Identifying a ratio of positive samples before filtering based on a positive sample image among the plurality of images to be recognized, and identifying a filtering ratio based on the image to be filtered; determining a post-filtering positive sample ratio and a change curve of the filtering ratio based on the pre-filtering positive sample ratio; The image recognition device according to claim 9 , wherein the image quality judgment is evaluated based on a ratio of positive samples after the filtering and a change curve of the filtering ratio.

13. The second image recognition module is Obtain the pre-stored change curve of the proportion of positive samples after filtering and the filtering proportion; Identifying an evaluation index value based on the change curve of the ratio of positive samples after filtering and the filtering ratio, and the change curve of the ratio of positive samples after filtering and the filtering ratio stored in advance; The image recognition device according to claim 12 , wherein the evaluation is determined to be passed when the evaluation index value is greater than a predetermined evaluation threshold value.

14. The second image recognition module is Obtaining a target image from the plurality of images to be recognized according to the quality of the plurality of images to be recognized and a predetermined quality requirement; 10. The image recognition apparatus according to claim 8, further comprising: a second image recognition step for performing the second image recognition step based on the target image.

15. An image recognition device, A processor; Memory, A computer program, including 3. An image recognition device, comprising: a computer program stored in the memory and configured to be executed by the processor, the computer program including instructions for performing the image recognition method according to claim 1 or 2.

16. 1. A computer-readable storage medium, comprising: The computer-readable storage medium has instructions stored thereon:

3. A computer-readable storage medium comprising instructions, the instructions being executed by a computer, the computer implementing the image recognition method according to claim 1 or 2.

17. A computer program comprising:

3. A computer program, comprising: a computer program for executing, when executed by a computer, the computer carrying out the image recognition method according to claim 1 or 2.

Images