Target recognition method, device and electronic system

Abstract

The present invention provides a target recognition method, device and electronic system, which obtain an image to be processed containing a target to be recognized; input the image to be processed into a trained target recognition model to obtain a recognition result of the image to be processed; the target recognition model is trained in the following manner: determine the sample weight corresponding to each sample image in the sample image set; determine the loss value of the sample image based on the sample weight and loss function corresponding to the sample image, and train the initial model according to the loss value to obtain the target recognition model. In this method, different sample images correspond to different sample weights. During the training process of the target recognition model, the loss value corresponding to each sample image is determined for the sample weight and loss function corresponding to the sample image, and then the model parameters are adjusted based on the loss value. This training method is conducive to improving the overall performance of the target recognition model, thereby improving the target recognition accuracy of the model.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a target recognition method, device and electronic system. Background Art

[0002] Target recognition is a method of identifying an object based on its characteristic information. Using deep learning technology, target recognition has achieved certain results and can be widely used in security, traffic and other scenarios. In related technologies, targets are usually identified through trained deep learning models, which are trained directly on data sets. However, since there are samples in the data set with different degrees of difficulty for model learning, the training of the model is limited, resulting in poor accuracy of model target recognition. Summary of the invention

[0003] The object of the present invention is to provide a target recognition method, device and electronic system to improve the accuracy of model target recognition.

[0004] In a first aspect, the present invention provides a target recognition method, which includes: obtaining an image to be processed containing a target to be recognized; inputting the image to be processed into a pre-trained target recognition model to obtain a recognition result of the image to be processed; the target recognition model is trained in the following manner: determining a sample weight corresponding to each sample image in a sample image set; determining a loss value corresponding to the sample image based on the sample weight corresponding to the sample image and a preset loss function; and training an initial model according to the loss value to obtain a target recognition model.

[0005] In an optional embodiment, the sample image set includes multiple sample pairs, each of which includes two sample images. The step of determining the sample weight corresponding to each sample image in the sample image set includes: determining the similarity of each sample pair in the multiple sample pairs; for each sample pair in the multiple sample pairs, determining the sample weight of the sample pair based on the similarity of the sample pairs; and determining the sample weight of the sample pair as the sample weight corresponding to each sample image in the two sample images included in the sample pair.

[0006] In an optional embodiment, the step of determining the sample weight of the sample pair based on the similarity of the sample pairs for each sample pair in the multiple sample pairs includes: sorting the multiple sample pairs according to the size of the similarity of each sample pair to obtain a sorting result; and determining the sample weight of the sample pair according to the position of the sample pair in the sorting result for each sample pair.

[0007] In an optional embodiment, the above-mentioned multiple sample pairs include similar sample pairs and heterogeneous sample pairs; wherein, the two sample images in the similar sample pairs contain the same target, and the two sample images in the heterogeneous sample pairs contain different targets; the above-mentioned step of sorting the multiple sample pairs according to the size of the similarity of each sample pair to obtain the sorting result includes: sorting the similarities of the similar sample pairs in the multiple sample pairs in order from small to large similarities to obtain a first sorting result; sorting the similarities of the heterogeneous sample pairs in the multiple sample pairs in order from large to small similarities to obtain a second sorting result.

[0008] In an optional embodiment, the above-mentioned step of determining the sample weight of the sample pair according to the position of the sample pair in the sorting result for each sample pair, includes: performing the following operations for the first sorting result and the second sorting result, respectively: determining the sample weight of the sample pair sorted within the first preset range as a first value; determining the sample weight of the sample pair sorted within the second preset range as a second value; determining the sample weight of the sample pair sorted within the third preset range as a third value; the third value is greater than the first value and the second value; for the sample pairs sorted outside the first preset range, the second preset range and the third preset range, determining the sample weight of the sample pair as successively decreasing preset values ​​in accordance with the sorting order; wherein the minimum value of the preset value is greater than the first value and the second value, and the maximum value of the preset value is less than the third value.

[0009] In an optional embodiment, the above-mentioned multiple sample pairs include: the most difficult samples, the second most difficult samples, general samples and simple samples; the above-mentioned method also includes: determining the sample pairs sorted within the first preset range as the most difficult samples; determining the sample pairs sorted within the second preset range as simple samples; determining the sample pairs sorted within the third preset range as the second most difficult samples; and determining the sample pairs sorted outside the first preset range, the second preset range and the third preset range as general samples.

[0010] In an optional embodiment, the sample image set includes a plurality of sample pairs, each sample pair includes two sample images, and the sample weights corresponding to the two sample images are the same; the loss value corresponding to the sample image is determined based on the sample weights corresponding to the sample image and a preset loss function; the initial model is trained according to the loss value to obtain a target recognition model, comprising: determining training sample pairs from the sample image set; inputting the training sample pairs into the initial model to obtain output results of the training sample pairs; determining the loss value of the training sample pairs according to the preset loss function, the sample weights of the training sample pairs and the output results; adjusting the network parameters of the initial model based on the loss values ​​of the training sample pairs, and continuing to execute the step of determining the training sample pairs from the sample image set until the initial model converges or reaches a preset number of training times to obtain a target recognition model.

[0011] In an optional embodiment, the step of determining the loss value of the training sample pair based on a preset loss function, the sample weights of the training sample pair and the output results includes: substituting the output results into the preset loss function to obtain the loss amount; and multiplying the loss amount by the sample weights of the training sample pair to determine the loss value of the training sample pair.

[0012] In a second aspect, the present invention provides a target recognition device, which includes: an image acquisition module, used to acquire an image to be processed containing a target to be recognized; a recognition module, used to input the image to be processed into a pre-trained target recognition model to obtain a recognition result of the image to be processed; a model training module, used to: determine the sample weight corresponding to each sample image in a sample image set; determine the loss value corresponding to the sample image based on the sample weight corresponding to the sample image and a preset loss function; train the initial model according to the loss value to obtain a target recognition model.

[0013] In a third aspect, the present invention provides an electronic system comprising: an image acquisition device, a processing device and a storage device; the image acquisition device is used to obtain preview video frames or image data; a computer program is stored on the storage device, and the computer program executes the above-mentioned target recognition method when the processed device is running.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and the computer program executes the above-mentioned target recognition method when executed by a processing device.

[0015] The embodiments of the present invention bring the following beneficial effects:

[0016] The present invention provides a target recognition method, device and electronic system, which first obtains an image to be processed containing a target to be recognized; then inputs the image to be processed into a pre-trained target recognition model to obtain a recognition result of the image to be processed; the target recognition model is trained in the following manner: determining the sample weight corresponding to each sample image in the sample image set; determining the loss value corresponding to the sample image based on the sample weight corresponding to the sample image and a preset loss function; training the initial model according to the loss value to obtain the target recognition model. In this method, different sample images correspond to different sample weights. During the training of the target recognition model, the loss value corresponding to each sample image is determined for the sample weight and loss function corresponding to the sample image, and then the model parameters are adjusted based on the loss value. This training method is conducive to improving the overall performance of the target recognition model, thereby improving the target recognition accuracy of the model.

[0017] Other features and advantages of the present invention will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present invention.

[0018] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of the structure of an electronic system provided by an embodiment of the present invention;

[0021] Figure 2 A flow chart of a target recognition method provided by an embodiment of the present invention;

[0022] Figure 3 A flowchart of a method for training a target recognition model provided by an embodiment of the present invention;

[0023] Figure 4 A flowchart of another method for training a target recognition model provided by an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of the structure of a target recognition device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0026] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] In the related art, targets are usually identified through a trained deep learning model, which is trained directly on a data set. However, since the data set contains samples with different levels of difficulty in learning the model, the training of the model is limited, resulting in poor accuracy in target recognition by the model. It is difficult to improve the target recognition effect of the model when the target image has problems such as cross-age, large angles, blur, complex lighting, and large-scale occlusion.

[0028] Based on this, an embodiment of the present invention provides a target recognition method, device and electronic system. This technology can be applied to image processing scenarios, especially in scenarios such as target recognition, target similarity calculation and image screening. At the same time, this technology can be implemented using corresponding software and hardware. The embodiments of the present invention are introduced in detail below.

[0029] Embodiment 1:

[0030] First, refer to Figure 1 An exemplary electronic system 100 for implementing the target recognition method, apparatus, and electronic system according to the embodiments of the present invention is described.

[0031] like Figure 1 The electronic system 100 includes one or more processing devices 102, one or more storage devices 104, an input device 106, an output device 108, and one or more image acquisition devices 110. These components are interconnected via a bus system 112 and / or other forms of connection mechanisms (not shown). It should be noted that Figure 1 The components and structure of the electronic system 100 shown are merely exemplary and non-limiting. The electronic system may also have other components and structures as required.

[0032] The processing device 102 may be a gateway, or an intelligent terminal, or a device including a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities. It may process data of other components in the electronic system 100, and may also control other components in the electronic system 100 to perform desired functions.

[0033] The storage device 104 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processing device 102 may run the program instructions to implement the client functions and / or other desired functions in the embodiments of the present invention (implemented by the processing device) described below. Various applications and various data, such as various data used and / or generated by the application, may also be stored in the computer-readable storage medium.

[0034] The input device 106 may be a device used by a user to input instructions, and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.

[0035] The output device 108 may output various information (eg, images or sounds) to the outside (eg, a user), and may include one or more of a display, a speaker, and the like.

[0036] The image acquisition device 110 may acquire preview video frames or image data, and store the acquired preview video frames or image data in the storage device 104 for use by other components.

[0037] Exemplarily, the components in the example electronic system for implementing the target recognition method, device and electronic system according to the embodiment of the present invention can be integrated or dispersed, such as integrating the processing device 102, the storage device 104, the input device 106 and the output device 108 into one body, and setting the image acquisition device 110 at a specified position where the target image can be acquired. When the components in the above electronic system are integrated, the electronic system can be implemented as a smart terminal such as a camera, a smart phone, a tablet computer, a computer, a vehicle-mounted terminal, etc.

[0038] Embodiment 2:

[0039] This embodiment provides a target recognition method, which is executed by a processing device in the above electronic system; the processing device can be any device or chip with data processing capabilities. The processing device can process the received information independently, or it can be connected to a server to jointly analyze and process the information. Figure 2 As shown, the target recognition method includes the following specific steps:

[0040] Step S202, obtaining an image to be processed containing an object to be identified.

[0041] The above-mentioned image to be processed can be a picture or photo taken by a camera or a camera, or a video frame in a specified video file; the image to be processed contains a target to be identified, and the target to be identified can be a human body, a human face, an animal or other scenery object. If the target to be identified is a human face, the face can be a front face, a side face, or a face with various expressions. In specific implementation, the image can be obtained by: shooting and transmitting through a communication-connected camera, a camera or other device, or obtaining from a storage device that stores the image to be processed that has been shot.

[0042] Step S204: input the above-mentioned image to be processed into a pre-trained target recognition model to obtain a recognition result of the image to be processed.

[0043] The above target recognition model can be trained in the following way: first, determine the sample weight corresponding to each sample image in the sample image set; then, based on the sample weight corresponding to the sample image and the preset loss function, determine the loss value corresponding to the sample image; train the initial model according to the loss value to obtain the target recognition model.

[0044] The above sample image set includes a large number of sample images, each of which contains a target and carries a target identifier, which is used to indicate the target contained in the sample image; specifically, if the target identifiers corresponding to two sample images are the same, it means that the targets contained in the two sample images are the same.

[0045] In the specific implementation, each sample image has different training value for the model. The present invention sets the sample weight of the sample image with high training value to a larger value to increase the influence of the sample image on the model training; sets the sample weight of the sample image with low training value to a smaller value to reduce the influence of the sample image on the model training. Then, according to the sample images in the sample image set, as well as the sample weights and loss functions corresponding to the sample images, the initial model is trained to obtain a pre-trained target recognition model. This method can better train the model and improve the recognition accuracy of the model.

[0046] The present invention provides a target recognition method, which first obtains an image to be processed containing a target to be recognized; then inputs the image to be processed into a pre-trained target recognition model to obtain a recognition result of the image to be processed; the target recognition model is trained in the following manner: determining the sample weight corresponding to each sample image in the sample image set; determining the loss value corresponding to the sample image based on the sample weight corresponding to the sample image and a preset loss function; training the initial model according to the loss value to obtain the target recognition model. In this method, different sample images correspond to different sample weights. During the training of the target recognition model, the loss value corresponding to each sample image is determined for the sample weight and loss function corresponding to the sample image, and then the model parameters are adjusted based on the loss value. This training method is conducive to improving the overall performance of the target recognition model, thereby improving the target recognition accuracy of the model.

[0047] Embodiment three:

[0048] The embodiment of the present invention further provides a method for training a target recognition model, wherein the target recognition model obtained by training is used to process the above-mentioned image to be processed; Figure 3 As shown, the method comprises the following specific steps:

[0049] Step S302, obtaining a sample image set; the sample image set includes a plurality of sample pairs, and each sample pair includes two sample images.

[0050] The sample image set may include a part of the sample images in the preset training data set, or may include all the sample images. The training data set includes a large number of sample images, each of which includes a target, and each of which carries a target identifier. In practical applications, it is necessary to combine the sample images in the sample image set to obtain a plurality of sample pairs, wherein each sample pair includes two sample images, and the targets included in the two sample images may be the same or different.

[0051] In a specific implementation, after obtaining the sample image set, the sample images in the sample image set may be preprocessed to obtain sample images that are more conducive to model training. Specifically, the sample images in the sample image set may be subjected to scene enhancement processing (equivalent to the above preprocessing) to obtain enhanced sample images; then the enhanced sample images may be combined into a new sample image set, and the new sample image set may be used as the sample image set for determining the similarity of the sample pairs. The above image enhancement processing may include, but is not limited to, scaling, flipping, blurring, whitening, and the like.

[0052] Step S304: determine the similarity of each sample pair in the sample image set.

[0053] Similarity generally refers to the similarity between the image features of two images. There are many ways to calculate the similarity. You can use the method of calculating similarity provided in the prior art, or you can use the following steps 10-11 to calculate the similarity:

[0054] Step 10: Input the sample image set into a preset neural network model to obtain a feature vector of each sample image in the sample image set. It can also be understood that the preset neural network model is used to extract features from the sample image and output the feature vector of the sample image.

[0055] Step 11: for each sample pair in the sample image set, the Euclidean distance between the feature vectors of the two sample images in the sample pair is calculated to obtain the similarity of the sample pair.

[0056] In a specific implementation, the Euclidean distance between the feature vectors of two sample images in a sample pair is the similarity between the two sample images. Generally, the larger the Euclidean distance, the smaller the similarity; the smaller the Euclidean distance, the greater the similarity.

[0057] Step S306, for each sample pair in the sample image set, determine the sample weight of the sample pair based on the similarity of the sample pair; determine the sample weight of the sample pair as the sample weight corresponding to each sample image in the two sample images included in the sample pair.

[0058] In specific implementation, the sample weight of each sample pair in the sample image set can be determined through the following steps 20-21:

[0059] Step 20, sorting the sample pairs in the sample image set according to the similarity of each sample pair to obtain a sorting result.

[0060] In specific implementation, the sample pairs in the sample image set may be sorted in descending order of similarity, or may be sorted in ascending order of similarity.

[0061] In the specific implementation, through the target identifiers carried by the two sample images in the sample pair, it can be determined whether the two sample images in the sample pair contain the same target. If so, the two images in the sample pair are determined to be of the same type, otherwise they are of different types; among them, the sample pairs corresponding to the image class can be called the same type sample pairs, and the sample pairs corresponding to the different types of images can be called different types of sample pairs. On the basis of determining whether the two sample images in the sample pair are of the same type, combined with the similarity of the sample pair, the learning difficulty of the sample pair can be determined, and then the sample weight of the sample pair can be obtained, which also represents the value of the sample pair for model training. Usually, the model may never learn a sample pair that is too difficult, and a sample pair that is too simple is prone to model overfitting and does not need to be learned, so these two sample pairs have low value for model training. Therefore, it is necessary to determine sample pairs with appropriate learning difficulty (which can also be understood as the sample pairs that are most beneficial to model training) to train samples.

[0062] In practical applications, the similarity between images of the same type is usually relatively large, but in some special scenarios, the similarity between images of the same type is very small. Such images of the same type are difficult to learn during model training, so the training value of such images of the same type is determined to be very low, and the sample weights of the sample pairs corresponding to such images of the same type are adjusted to reduce the impact of such images of the same type on model training. Similarly, the similarity between images of different types is usually relatively small, but in some special scenarios, the similarity between images of different types is very large, that is, the targets in images of different types are very similar. Such images of different types are also difficult to learn during model training, so the training value of such images of different types is determined to be very low, and the sample weights of the sample pairs corresponding to such images of different types are adjusted to reduce the impact of such images of different types on model training.

[0063] At the same time, for images of the same type with very large similarity (for example, greater than a preset threshold), or images of different types with very small similarity, they are very easy to learn during model training, and model overfitting is prone to occur. Therefore, such images are not of great value to model training, and the sample weights of the sample pairs corresponding to such images need to be adjusted to reduce the impact of such images on model training.

[0064] Sample images other than the above cases can be considered as samples with relatively greater value for model training, and the sample weights of sample pairs corresponding to such sample images can be adjusted to increase the influence of such sample pairs on model training.

[0065] Since the rules for determining the learning difficulty of sample pairs are different when the sample pairs belong to the same class and the sample pairs belong to different classes, it is also necessary to sort the sample pairs belonging to the same class and the sample pairs belonging to different classes respectively to obtain two sorting results, that is, the above step 20 can be implemented by the following steps 30-31:

[0066] Step 30 , sorting the similarities of the sample pairs of the same type among the multiple sample pairs in order of similarity from small to large, to obtain a first sorting result.

[0067] Step 31 , sorting the similarities of the heterogeneous sample pairs among the multiple sample pairs in descending order of similarity to obtain a second sorting result.

[0068] In some embodiments, the same sample pairs among the multiple sample pairs may be sorted in descending order of similarity; and the different sample pairs among the multiple sample pairs may be sorted in descending order of similarity. Of course, the different sample pairs among the multiple sample pairs and the different sample pairs among the multiple sample pairs may also be sorted according to the same rules. However, in this sorting method, the rules for determining the learning difficulty and sample weight of the sample pairs will be different for the sorting results corresponding to the different sample pairs among the multiple sample pairs and the sorting results corresponding to the different sample pairs among the multiple sample pairs.

[0069] Step 21, for each sample pair, determine the sample weight of the sample pair according to the position of the sample pair in the sorting result.

[0070] In the specific implementation, the learning difficulty of the sample pair corresponding to each sorting position is preset, and each learning difficulty is also set with a corresponding learning sample weight. Therefore, the learning difficulty and sample weight of the sample pair can be determined according to the position of the sample pair in the sorting result. Specifically, the learning difficulty of the sample pair corresponding to each sorting position preset above can be set according to research and development needs, or can be determined according to experimental exploration; the specific value of the sample weight corresponding to each learning difficulty can be set according to research and development needs.

[0071] In a specific implementation, after obtaining the first sorting result and the second sorting result according to the above steps 30-31, in order to determine the sample weights of the sample pairs, the following steps 40-43 may be performed respectively for the first sorting result and the second sorting result:

[0072] Step 40, the sample weights of the sample pairs ranked within the first preset range are determined as a first value. The first preset range can be set according to research and development requirements, or can be determined according to experimental exploration. For example, the first preset range can be the top 1 / 20 sample pairs ranked, or the top 1 / 30 sample pairs ranked. The first value can also be set according to research and development requirements.

[0073] In specific implementation, sample pairs can be divided into the most difficult samples, second most difficult samples, general samples and simple samples according to the learning difficulty of sample pairs. Among them, the most difficult samples are usually the samples that are most difficult to fit the model. In the same type of sample pairs, it can refer to the part with the smallest similarity; in heterogeneous sample pairs, it refers to the part with the largest similarity. Therefore, the sample pairs within the first preset range that are ranked high in the first sorting result and the second sorting result can be determined as the most difficult samples. For example, if the first preset range is the first 1 / 20, then the most difficult samples can be the first 1 / 20 sample pairs ranked high in the first sorting result, that is, the 1 / 20 sample pairs with the smallest similarity among the same type of sample pairs; or, the first 1 / 20 sample pairs ranked high in the second sorting result, that is, the 1 / 20 sample pairs with the largest similarity among the heterogeneous sample pairs.

[0074] Since it is difficult for the model to fit the most difficult sample, that is, the most difficult sample is not of high value to model training, the sample weight corresponding to the most difficult sample can be determined as a smaller value. For example, the first value can be set to 0.01 or 0.02.

[0075] Step 41, the sample weights of the sample pairs ranked within the second preset range are determined as a second value. The second preset range can be set according to research and development requirements, or can be determined according to experimental exploration. For example, the second preset range can be the last 1 / 20 sample pairs ranked at the back, or the last 1 / 30 sample pairs, etc. The second value can be set according to research and development requirements, and the second value can be the same as or different from the first value.

[0076] In specific implementation, simple samples are usually the samples that are easiest to fit the model, that is, the part with the greatest similarity among the same type of sample pairs and the part with the least similarity among the different types of sample pairs. Therefore, the sample pairs in the second preset range that are ranked later in the first sorting result and the second sorting result can be determined as simple samples. For example, if the second preset range is the last 1 / 20, then the simple samples can be the last 1 / 20 sample pairs ranked later in the first sorting result, that is, the 1 / 20 sample pairs with the greatest similarity among the same type of sample pairs; or, the last 1 / 20 sample pairs ranked later in the second sorting result, that is, the 1 / 20 sample pairs with the least similarity among the different types of sample pairs.

[0077] Since the model can easily fit simple samples, that is, simple samples are not of high value to model training, the sample weights corresponding to simple samples can be determined as smaller values. For example, the second value can be set to 0.01 or 0.02.

[0078] Step 42, the sample weights of the sample pairs ranked within the third preset range are determined as a third value; the third value is greater than the first value and the second value. The third preset range can be set according to research and development requirements, or can be determined according to experimental exploration. For example, the third preset range can be 1 / 20 to 1 / 4 of the top ranked samples. The third value can be set according to research and development requirements.

[0079] In specific implementation, the second most difficult sample belongs to the category of difficult samples, but is not the most difficult sample, that is, the part with the second smallest similarity in the same sample pair; the part with the second largest similarity in the heterogeneous sample pair. Therefore, the sample pairs ranked within the third preset range in the first sorting result and the second sorting result can be determined as the second most difficult samples. For example, if the third preset range is the top 1 / 20 to 1 / 4, then the second most difficult sample can be the top 1 / 20 to 1 / 4 sample pairs in the first sorting result, that is, the top 1 / 20 to 1 / 4 sample pairs with the smallest similarity in the same sample pairs; or, the top 1 / 20 to 1 / 4 sample pairs in the second sorting result, that is, the top 1 / 20 to 1 / 4 sample pairs with the largest similarity in the heterogeneous sample pairs.

[0080] Since the second most difficult samples have moderate difficulty in fitting the model, that is, the second most difficult samples have the highest value for model training, the sample weight corresponding to the second most difficult samples can be determined as a larger value. For example, the third value can be set to 1 or 2.

[0081] Step 43, for sample pairs sorted outside the first preset range, the second preset range and the third preset range, determine the sample weights of the sample pairs as preset values ​​in descending order according to the sorting order; wherein the minimum value of the preset value is greater than the first value and the second value, and the maximum value of the preset value is less than the third value.

[0082] Among multiple sample pairs, sample pairs other than the most difficult sample, the second most difficult sample, and the simple sample are general samples, that is, sample pairs ranked outside the first preset range, the second preset range, and the third preset range are determined as general samples. The above-mentioned preset value can be a value selected from a preset range, and the preset range can be set according to research and development requirements, wherein the minimum value of the preset range is less than the second value, and the maximum value is greater than the first value and the second value. In specific implementation, general samples can select descending preset values ​​from the preset range in the sorting order as sample weights. For example, if the preset range is (0.01, 1), then the general samples in the first sorting result will select descending values ​​from 0.01 to 1 in order of similarity from small to large as sample weights; the general samples in the second sorting result will select descending values ​​from 0.01 to 1 in order of similarity from large to small as sample weights.

[0083] Step S308, based on the sample weight corresponding to the sample image and the preset loss function, determine the loss value corresponding to the sample image; train the initial model according to the loss value to obtain the target recognition model.

[0084] The above-mentioned preset loss function is a pre-set loss function during model training. The preset loss function can adopt a cross entropy loss function, a square error loss function, etc.

[0085] The training method of the above-mentioned target recognition model can determine the sample weight of the sample pair and the sample weight of the sample image in the sample pair according to the similarity of the sample pair when the target recognition model is being trained; then the loss value of the sample image pair is adjusted based on the sample weight so as to obtain different loss values ​​for different sample images to train the model, thereby improving the accuracy of the model's target recognition; at the same time, this method can also better improve the recognition effect of the model in complex lighting, large angles, blur, and large-scale occlusion scenes.

[0086] Embodiment 4:

[0087] The embodiment of the present invention also provides another training method for a target recognition model, which is implemented on the basis of the above training method. The method focuses on describing the specific process of determining the loss value corresponding to the sample image based on the sample weight corresponding to the sample image and the preset loss function; training the initial model according to the loss value to obtain the target recognition model (specifically implemented by the following steps S404-S414); Figure 4 As shown, the method comprises the following specific steps:

[0088] Step S402, obtaining a sample image set, and determining a sample weight of each sample pair in the sample image set; wherein the sample image set includes a plurality of sample pairs, each sample pair includes two sample images, and the sample weights corresponding to the two sample images are the same.

[0089] Step S404: determining training sample pairs from the sample image set.

[0090] When determining the training sample pair from the sample image set for the first time, a sample pair is randomly selected from the sample image set as the training sample pair; when determining the training sample pair subsequently, a sample pair can be randomly selected from the sample pairs in the sample image set that have not been determined as the training sample pair as a new training sample pair.

[0091] Step S406: input the above training sample pairs into the initial model to obtain output results of the training sample pairs.

[0092] The above-mentioned initial model can be a neural network or a deep learning model, etc. The initial model can output the predicted recognition results of the training sample pairs (equivalent to the above-mentioned output results). The predicted recognition results may be inaccurate, and the initial model needs to be subsequently trained to obtain a model that can accurately predict.

[0093] Step S408, determining the loss value of the training sample pair according to the preset loss function, the sample weight of the training sample pair and the output result.

[0094] The above preset loss function is used to calculate the difference between the output result and the standard result, and the difference will guide the optimization direction of the model. In the specific implementation, since both sample images in the training sample pair carry the target identifier, the standard results of the two sample images can be determined according to whether the target identifiers of the two sample images are consistent. For example, if the target identifiers of the two sample images are consistent, the standard results of the two sample images can be set to 1, which can also be understood as the similarity of the training sample pairs corresponding to the two sample images is 1; if the target identifiers of the two sample images are inconsistent, the standard results of the two sample images can be set to 0, which can also be understood as the similarity of the training sample pairs corresponding to the two sample images is 0.

[0095] In the specific implementation, the output results and standard results of the training sample pair are input into the preset loss function of the pair to obtain the loss amount, and then the product of the loss amount and the sample weight of the training sample pair is determined as the loss value of the training sample pair.

[0096] Step S410, adjusting the network parameters of the initial model based on the loss value of the training sample pair.

[0097] In the specific implementation, the loss value is used to guide the training of the network parameters of the initial model, that is, the loss value is ultimately reflected in the gradient of the model, and the gradient is backpropagated to guide the training of the model.

[0098] Step S412, determine whether the adjusted network parameters of the initial model converge; if so, execute step S414; otherwise, execute step S404.

[0099] In some embodiments, the training may be stopped when a preset number of training times is reached, and the adjusted initial model obtained when the training is stopped may be determined as the target recognition model.

[0100] Step S414: determine the adjusted initial model as the target recognition model.

[0101] The training method of the above-mentioned target recognition model characterizes the learning difficulty and learning value of the sample pair through the sample weight of each sample pair in the sample image set, and then determines the loss value according to the sample weight of each sample pair and the preset loss function during the model training process, so as to generate different loss values ​​according to sample pairs with different learning values, thereby improving the recognition accuracy of the model.

[0102] Embodiment five:

[0103] Corresponding to the above target recognition method embodiment, the present invention provides a target recognition device, such as Figure 5 As shown, the device comprises:

[0104] The image acquisition module 50 is used to acquire an image to be processed that contains an object to be identified.

[0105] The recognition module 51 is used to input the image to be processed into a pre-trained target recognition model to obtain a recognition result of the image to be processed.

[0106] The model training module 52 is used to: determine the sample weight corresponding to each sample image in the sample image set; determine the loss value corresponding to the sample image based on the sample weight corresponding to the sample image and a preset loss function; train the initial model according to the loss value to obtain a target recognition model.

[0107] Specifically, the above-mentioned sample image set includes multiple sample pairs; wherein each sample pair includes two sample images; the above-mentioned model training module 52 includes: a similarity determination model, used to determine the similarity of each sample pair in the multiple sample pairs; a weight determination model, used to determine the sample weight of the sample pair for each sample pair in the multiple sample pairs based on the similarity of the sample pairs; and determine the sample weight of the sample pair as the sample weight corresponding to each sample image in the two sample images contained in the sample pair.

[0108] Furthermore, the weight determination model is also used to: sort multiple sample pairs according to the similarity of each sample pair to obtain a sorting result; and for each sample pair, determine the sample weight of the sample pair according to the position of the sample pair in the sorting result.

[0109] Specifically, the above-mentioned multiple sample pairs include similar sample pairs and heterogeneous sample pairs; wherein, the two sample images in the similar sample pairs contain the same target, and the two sample images in the heterogeneous sample pairs contain different targets; the above-mentioned weight determination model is also used to: sort the similarities of the similar sample pairs in the multiple sample pairs in order from small to large similarities to obtain a first sorting result; sort the similarities of the heterogeneous sample pairs in the multiple sample pairs in order from large to small similarities to obtain a second sorting result.

[0110] In a specific implementation, the above-mentioned weight determination model is also used to: perform the following operations for the first sorting result and the second sorting result respectively: determine the sample weight of the sample pairs sorted within the first preset range as a first value; determine the sample weight of the sample pairs sorted within the second preset range as a second value; determine the sample weight of the sample pairs sorted within the third preset range as a third value; the third value is greater than the first value and the second value; for the sample pairs sorted outside the first preset range, the second preset range and the third preset range, determine the sample weights of the sample pairs as successively decreasing preset values ​​in accordance with the sorting order; wherein the minimum value of the preset value is greater than the first value and the second value, and the maximum value of the preset value is less than the third value.

[0111] In practical applications, the above-mentioned multiple sample pairs include: the most difficult samples, the second most difficult samples, general samples and simple samples; the above-mentioned device also includes a sample classification module, which is used to: determine the sample pairs sorted within the first preset range as the most difficult samples; determine the sample pairs sorted within the second preset range as simple samples; determine the sample pairs sorted within the third preset range as the second most difficult samples; and determine the sample pairs sorted outside the first preset range, the second preset range and the third preset range as general samples.

[0112] Furthermore, the above-mentioned sample image set includes multiple sample pairs, each sample pair includes two sample images, and the sample weights corresponding to the two sample images are the same; the above-mentioned model training module 52 is also used to: determine the training sample pairs from the sample image set; input the training sample pairs into the initial model to obtain the output results of the trained sample pairs; determine the loss value of the training sample pairs according to the preset loss function, the sample weights of the training sample pairs and the output results; adjust the network parameters of the initial model based on the loss value of the training sample pairs, and continue to execute the step of determining the training sample pairs from the sample image set until the initial model converges or reaches a preset number of training times to obtain the target recognition model.

[0113] Furthermore, the above-mentioned model training module 52 is also used to: bring the output result into a preset loss function to obtain the loss amount; and determine the product of the loss amount and the sample weight of the training sample pair as the loss value of the training sample pair.

[0114] The above-mentioned target recognition device first obtains an image to be processed containing a target to be recognized; then the image to be processed is input into a pre-trained target recognition model to obtain a recognition result of the image to be processed; the target recognition model is trained in the following manner: the sample weight corresponding to each sample image in the sample image set is determined; based on the sample weight corresponding to the sample image and a preset loss function, the loss value corresponding to the sample image is determined; the initial model is trained according to the loss value to obtain a target recognition model. In this method, different sample images correspond to different sample weights. During the training of the target recognition model, the loss value corresponding to each sample image is determined for the sample weight and loss function corresponding to the sample image, and then the model parameters are adjusted based on the loss value. This training method is conducive to improving the overall performance of the target recognition model, thereby improving the target recognition accuracy of the model.

[0115] Embodiment six:

[0116] An embodiment of the present invention provides an electronic system, which includes: an image acquisition device, a processing device and a storage device; the image acquisition device is used to obtain preview video frames or image data; a computer program is stored on the storage device, and the computer program executes the target recognition method as described above when the processed device is running.

[0117] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the electronic system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0118] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processing device, the target recognition method as described above is executed.

[0119] The computer program products of the target identification method, device and electronic system provided in the embodiments of the present invention include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the method embodiments and will not be repeated here.

[0120] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and / or device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0121] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0122] In the description of the present invention, it should be noted that the terms “first”, “second” and “third” are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0123] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A target recognition method, characterized in that: The method comprises: Acquire an image to be processed containing an object to be identified; Inputting the image to be processed into a pre-trained target recognition model to obtain a recognition result of the image to be processed; The target recognition model is trained in the following way: Determine the sample weight corresponding to each sample image in the sample image set; Based on the sample weight corresponding to the sample image and a preset loss function, determining the loss value corresponding to the sample image; training the initial model according to the loss value to obtain the target recognition model; The sample image set includes a plurality of sample pairs; wherein each of the sample pairs includes two sample images, and the sample weight of the sample pair represents the value of the sample pair for model training; the step of determining the sample weight corresponding to each sample image in the sample image set includes: determining the similarity of each of the sample pairs in the plurality of sample pairs; for each sample pair in the plurality of sample pairs, determining the sample weight of the sample pair based on the similarity of the sample pairs; determining the sample weight of the sample pair as the sample weight corresponding to each of the two sample images included in the sample pair; The step of determining the sample weight of each sample pair in the plurality of sample pairs based on the similarity of the sample pairs comprises: sorting the plurality of sample pairs according to the magnitude of the similarity of each sample pair to obtain a sorting result; and determining the sample weight of each sample pair according to the position of the sample pair in the sorting result; The multiple sample pairs include similar sample pairs and heterogeneous sample pairs; wherein the two sample images in the similar sample pairs include the same target, and the two sample images in the heterogeneous sample pairs include different targets; the step of sorting the multiple sample pairs according to the similarity of each sample pair to obtain the sorting result comprises: sorting the similarities of the similar sample pairs in the multiple sample pairs in an ascending order of the similarities to obtain a first sorting result; sorting the similarities of the heterogeneous sample pairs in the multiple sample pairs in an ascending order of the similarities to obtain a second sorting result; The step of determining the sample weight of each sample pair according to the position of the sample pair in the sorting result includes: performing the following operations on the first sorting result and the second sorting result respectively: determining the sample weight of the sample pair sorted within the first preset range as a first value; determining the sample weight of the sample pair sorted within the second preset range as a second value; determining the sample weight of the sample pair sorted within the third preset range as a third value; the third value is greater than the first value and the second value; for the sample pairs sorted outside the first preset range, the second preset range and the third preset range, determining the sample weight of the sample pair as preset values ​​that decrease in sequence according to the sorting order; wherein the minimum value of the preset value is greater than the first value and the second value, and the maximum value of the preset value is less than the third value; The multiple sample pairs include: the most difficult samples, the second most difficult samples, general samples and simple samples; the method also includes: determining the sample pairs sorted within the first preset range as the most difficult samples; determining the sample pairs sorted within the second preset range as the simple samples; determining the sample pairs sorted within the third preset range as the second most difficult samples; determining the sample pairs sorted outside the first preset range, the second preset range and the third preset range as the general samples; wherein the sample weight corresponding to the most difficult sample is determined to be a smaller value, the sample weight corresponding to the simple sample is determined to be a smaller value, the sample weight corresponding to the second most difficult sample is determined to be a larger value, and the sample weight corresponding to the general sample is determined to be a preset value selected from the preset range in descending order.

2. The method according to claim 1, characterized in that: The sample image set includes a plurality of sample pairs, each of the sample pairs includes two sample images, and the sample weights corresponding to the two sample images are the same; Determining the loss value corresponding to the sample image based on the sample weight corresponding to the sample image and a preset loss function; The step of training the initial model according to the loss value to obtain the target recognition model comprises: Determining training sample pairs from the sample image set; Inputting the training sample pair into the initial model to obtain an output result of the training sample pair; Determining a loss value of the training sample pair according to the preset loss function, the sample weight of the training sample pair and the output result; The network parameters of the initial model are adjusted based on the loss value of the training sample pair, and the step of determining the training sample pair from the sample image set is continued until the initial model converges or reaches a preset number of training times, thereby obtaining the target recognition model.

3. The method according to claim 2, characterized in that The step of determining the loss value of the training sample pair according to the preset loss function, the sample weight of the training sample pair and the output result comprises: Substituting the output result into the preset loss function to obtain the loss amount; The product of the loss amount and the sample weight of the training sample pair is determined as the loss value of the training sample pair.

4. A target recognition device, characterized in that: The device comprises: An image acquisition module is used to acquire an image to be processed containing an object to be identified; A recognition module, used for inputting the image to be processed into a pre-trained target recognition model to obtain a recognition result of the image to be processed; The model training module is used to: determine the sample weight corresponding to each sample image in the sample image set; determine the loss value corresponding to the sample image based on the sample weight corresponding to the sample image and a preset loss function; train the initial model according to the loss value to obtain the target recognition model; The sample image set includes a plurality of sample pairs; each of the sample pairs includes two sample images, and the sample weight of the sample pair represents the value of the sample pair for model training; the model training module is used to: determine the similarity of each of the sample pairs in the plurality of sample pairs; for each sample pair in the plurality of sample pairs, determine the sample weight of the sample pair based on the similarity of the sample pair; determine the sample weight of the sample pair as the sample weight corresponding to each sample image in the two sample images included in the sample pair; The model training module is used to: sort the multiple sample pairs according to the similarity of each sample pair to obtain a sorting result; for each sample pair, determine the sample weight of the sample pair according to the position of the sample pair in the sorting result; The multiple sample pairs include similar sample pairs and heterogeneous sample pairs; wherein the two sample images in the similar sample pairs include the same target, and the two sample images in the heterogeneous sample pairs include different targets; the model training module is used to: sort the similarities of the similar sample pairs in the multiple sample pairs in an ascending order of the similarities to obtain a first sorting result; sort the similarities of the heterogeneous sample pairs in the multiple sample pairs in an ascending order of the similarities to obtain a second sorting result; The model training module is used to: perform the following operations respectively for the first sorting result and the second sorting result: determine the sample weight of the sample pair sorted within the first preset range as a first value; determine the sample weight of the sample pair sorted within the second preset range as a second value; determine the sample weight of the sample pair sorted within the third preset range as a third value; the third value is greater than the first value and the second value; for the sample pairs sorted outside the first preset range, the second preset range and the third preset range, determine the sample weight of the sample pair as preset values ​​that decrease in sequence according to the sorting order; wherein the minimum value of the preset value is greater than the first value and the second value, and the maximum value of the preset value is less than the third value; The multiple sample pairs include: the most difficult samples, the second most difficult samples, general samples and simple samples; the model training module is also used to: determine the sample pairs sorted within the first preset range as the most difficult samples; determine the sample pairs sorted within the second preset range as the simple samples; determine the sample pairs sorted within the third preset range as the second most difficult samples; determine the sample pairs sorted outside the first preset range, the second preset range and the third preset range as the general samples; wherein the sample weight corresponding to the most difficult sample is determined to be a smaller value, the sample weight corresponding to the simple sample is determined to be a smaller value, the sample weight corresponding to the second most difficult sample is determined to be a larger value, and the sample weight corresponding to the general sample is determined to be a preset value selected from the preset range in descending order.

5. An electronic system, characterized in that: The electronic system includes: an image acquisition device, a processing device and a storage device; The image acquisition device is used to obtain preview video frames or image data; The storage device stores a computer program, which, when executed by the processing device, executes the target recognition method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the target recognition method according to any one of claims 1 to 3 is executed.

Images