Article classification method, device, electronic device and computer readable medium

Abstract

Embodiments of the present disclosure disclose an article classification method, device, electronic equipment and computer readable medium. A specific implementation of the method comprises: acquiring an article image; inputting the article image into an article image feature extraction model included in a pre-trained article classification model to obtain an article image global feature vector; inputting the article image global feature vector into an article classification sub-model to obtain article category information; and sending the article category information to a display terminal for classified storage of the article. The implementation can improve the accuracy of article classification.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of computer technology, and more specifically to methods, apparatus, electronic devices, and computer-readable media for classifying articles. Background Technology

[0002] By performing category recognition processing on the acquired object images, category information is obtained, and then the objects are classified and stored. Currently, the common method for classifying objects is to use a full-scale learning model (e.g., ImageClassification, an image classification task model) to identify the category of objects in the image and classify the objects accordingly.

[0003] However, the inventors discovered that when classifying items using the above method, the following technical problems often arise:

[0004] First, the full-learning model uses only one classifier to identify the category information in the image. When there are images of different items with high similarity, the accuracy of the single classifier will decrease, resulting in a decrease in the accuracy of item classification.

[0005] Second, the full-scale learning model has only one classifier, and the model parameters are updated in a full-scale manner. Even parameters that meet the training requirements may be changed, which leads to a decrease in the accuracy of the full-scale recognition model in identifying item categories, and thus a decrease in the accuracy of item classification.

[0006] Third, during the training of the full-scale learning model, each sample image needs to be manually labeled with the correct category. When the manual labeling is incorrect, the sample images need to be labeled repeatedly, which wastes labeling time and thus wastes model training time.

[0007] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0008] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0009] Some embodiments of this disclosure provide methods, apparatuses, electronic devices, and computer-readable media for classifying articles to address one or more of the technical problems mentioned in the background section above.

[0010] In a first aspect, some embodiments of this disclosure provide an item classification method, which includes: acquiring an item image; inputting the item image into an item image feature extraction model included in a pre-trained item classification model to obtain a global feature vector of the item image, wherein the pre-trained item classification model further includes an item classification sub-model; inputting the global feature vector of the item image into the item classification sub-model to obtain item category information; and sending the item category information to a display terminal for item classification and storage.

[0011] Secondly, some embodiments of this disclosure provide an item classification device, the device comprising: an acquisition unit configured to acquire an item image; a first input unit configured to input the item image into an item image feature extraction model included in a pre-trained item classification model to obtain a global feature vector of the item image, wherein the pre-trained item classification model further includes an item classification sub-model; a second input unit configured to input the global feature vector of the item image into the item classification sub-model to obtain item category information; and a sending unit configured to send the item category information to a display terminal for classified storage of items.

[0012] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the implementations of the first aspect above.

[0013] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0014] The above embodiments of this disclosure have the following beneficial effects: the item classification method of some embodiments of this disclosure can improve the accuracy of item classification. Specifically, the reason for the decrease in the accuracy of item classification is that the full-learning model only uses one classifier to identify the category information in the image. When there are images of different items with high similarity, the accuracy of the single classifier will decrease. Based on this, the item classification method of some embodiments of this disclosure firstly acquires an item image. Secondly, the item image is input into the item image feature extraction model included in the pre-trained item classification model to obtain the global feature vector of the item image. The pre-trained item classification model also includes an item classification sub-model. Thus, the feature information of the item image can be obtained to identify the category of the item in the item image. Then, the global feature vector of the item image is input into the item classification sub-model to obtain the item category information. Thus, the feature information of the item image can be classified by the item classification sub-model obtained by splicing multiple binary classification sub-models. Finally, the item category information is sent to the display terminal for classified storage of items. Thus, items can be classified according to the identified item category information. Therefore, some of the item classification methods disclosed herein can identify the category information of items in item images through a pre-trained item classification model. The pre-trained item classification model includes an image feature extraction model and an item classification sub-model synthesized by splicing multiple binary sub-models, which can improve the accuracy of category information identification, thereby improving the accuracy of item classification. Attached Figure Description

[0015] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0016] Figure 1 This is a flowchart of some embodiments of the article classification method according to this disclosure;

[0017] Figure 2 This is a schematic diagram of the structure of some embodiments of the article sorting device according to the present disclosure;

[0018] Figure 3 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0019] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0020] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0021] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0022] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0023] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0024] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] Figure 1 A flow 100 of some embodiments of an article classification method according to the present disclosure is shown. The article classification method includes the following steps:

[0026] Step 101: Obtain the item image.

[0027] In some embodiments, the entity performing the item classification method can acquire the item images from a terminal device via a wired or wireless connection. These item images may be images of perishable goods.

[0028] It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other currently known or future wireless connection methods.

[0029] Step 102: Input the item image into the item image feature extraction model included in the pre-trained item classification model to obtain the global feature vector of the item image.

[0030] In some embodiments, the execution entity can input the item image into a pre-trained item classification model, including an item image feature extraction model, to obtain a global feature vector of the item image. The pre-trained item classification model may further include an item classification sub-model. The item image feature extraction model can be a pre-trained model that takes an image as input and outputs a global feature vector of the image. The item classification sub-model may include, but is not limited to, at least one of the following: a set of binary classification sub-models and a pooling model. The global feature vector of the item image can represent the feature information of the item image. The feature information of the item image may include, but is not limited to, at least one of the following: color features, texture features, or shape features. The binary classification sub-model in the set of binary classification sub-models can be a pre-trained fully connected layer. The binary classification sub-model can be a network model that takes the global feature vector of the image as input and outputs an image classification confidence matrix. Here, the various binary classification sub-models in the set of binary classification sub-models can be concatenated into a fully connected matrix, and then the fully connected matrix can be determined as the item classification sub-model. The item classification sub-model can be a network model that takes the global feature vector of the image as input and outputs an image classification confidence matrix. The pooling model described above can be a max pooling layer. It can also be a network model that takes an image classification confidence matrix as input and the maximum image classification confidence as output.

[0031] Optionally, the above item classification model can be trained through the following steps:

[0032] The first step involves acquiring sample item image sequences, an initial image cache, and an initial item category recognition model. The initial image cache may include an initial set of classification image sequences. The initial item category recognition model may include an initial item image feature extraction model and an initial item classification sub-model. The initial item classification sub-model may include an initial set of binary classification models. Each initial classification image sequence in the initial classification image sequence set corresponds one-to-one with each initial binary classification sub-model in the initial binary classification sub-model set. The sample item images in the sample item image sequences may be images of perishable goods. The initial classification image sequences in the initial classification image sequence set may include at least one initial classification image. The initial binary classification model in the initial binary classification model set may be a fully connected layer without bias parameters. The initial classification images may be classified images of perishable goods.

[0033] As an example, the initial item image feature extraction model mentioned above can be, but is not limited to, at least one of the following: ResNet18 (residual neural network) model, MobileNetV2 (lightweight neural network) model, or ViT (VisionTransformer) model.

[0034] The second step involves selecting sample item images from the above sample item image sequence and performing the following training sub-steps on the initial item category recognition model:

[0035] The first sub-step involves inputting the sample item image into the initial item image feature extraction model included in the initial item category recognition model to obtain the global feature vector of the sample item image.

[0036] The second sub-step generates a set of positive sample classification image sequences and a set of negative sample classification image sequences based on the global feature vector of the sample item images and the initial image cache library.

[0037] The third sub-step involves identifying the initial binary classification sub-models in the initial binary classification sub-model set included in the initial item category recognition model that correspond to the positive sample classification image sequence as the target item classification sub-model.

[0038] The fourth sub-step involves adjusting the initial item image feature extraction model and the target item classification sub-model based on a preset loss function, a set of positive sample classification image sequences, and a set of negative sample classification image sequences, to obtain the adjusted initial item image feature extraction model and target item classification sub-model.

[0039] As an example, the aforementioned preset loss function could be a binary cross-entropy loss function.

[0040] The fifth sub-step involves determining the initial item category recognition model as the item category recognition model in response to the determination that the sample item image sequence meets preset conditions. The preset conditions may be that all sample item images in the sample item image sequence have undergone the aforementioned training steps.

[0041] Optionally, in response to determining that the above sample item image sequence does not meet the above preset conditions, the above execution entity may also perform the above training steps again.

[0042] In some optional implementations of certain embodiments, the execution entity generates a set of positive sample classification image sequences and a set of negative sample classification image sequences based on the global feature vector of the sample item image and an initial image cache library, which may include the following steps:

[0043] The first step involves determining the global feature vector of the sample item image and the initial classification average similarity corresponding to each initial classification image sequence in the initial classification image sequence set included in the initial image cache library, thus obtaining an initial classification average similarity set. The initial classification average similarity in this set characterizes the degree of similarity between the sample item image and the initial classification image sequences in the initial classification image sequence set included in the initial image cache library. A smaller initial classification average similarity indicates a greater degree of similarity between the sample item image and the initial classification image sequence corresponding to the initial classification average similarity.

[0044] The second step is to sort the initial average similarity set of categories to obtain the initial average similarity sequence. Specifically, the initial average similarities of each category in the initial average similarity set can be sorted in ascending order to obtain the initial average similarity sequence.

[0045] The third step is to determine the first initial classification average similarity in the above initial classification average similarity sequence as the positive sample average similarity.

[0046] The fourth step involves adding the sample item images to the initial classification image sequence corresponding to the average similarity of the positive samples in the initial classification image sequence set included in the initial image cache library, thereby obtaining the positive sample classification image sequence. Each positive sample classification image in the positive sample classification image sequence can be the initial classification image with the highest similarity to the sample item images among the initial classification images included in the initial image cache library.

[0047] The fifth step is to determine the average similarity of each initial classification from the second to the preset number in the above initial classification average similarity sequence as the negative sample average similarity set.

[0048] As an example, the preset number mentioned above could be 6.

[0049] Step 6: The initial classification image sequence set corresponding to the average similarity of each negative sample in the initial classification image sequence set included in the initial image cache library and the average similarity set of negative samples is determined as the negative sample classification image sequence set. Each negative sample classification image in the negative sample classification image sequence set can be an initial classification image in the initial classification image set included in the initial image cache library whose similarity to the sample item image is greater than a certain threshold.

[0050] As an example, the aforementioned threshold could be the value of the sixth initial classification average similarity in the aforementioned initial classification average similarity sequence.

[0051] Optionally, the aforementioned implementing entity may also perform the following steps:

[0052] The first step, in response to determining that the number of positive sample classification images in the above positive sample classification image sequence is greater than a preset maximum number of images, is to delete the first positive sample classification image from the above positive sample classification image sequence, thereby obtaining a deleted positive sample classification image sequence. The value of the preset maximum number of images is not limited here.

[0053] As an example, the preset maximum number of images could be 10.

[0054] The second step is to determine the positive sample classification image sequence after deletion as the positive sample classification image sequence. This allows control over the number of positive sample classification images in the aforementioned positive sample classification image sequence, reducing storage space and resource waste.

[0055] Optionally, in response to determining that the initial classification average similarity in the above initial classification average similarity sequence is less than a preset similarity value, the execution entity may perform the following sub-steps:

[0056] The first sub-step involves creating a new initial classification image sequence. This can be achieved by creating an empty sequence.

[0057] The second sub-step involves adding the aforementioned sample item images to the newly initialized classification image sequence to obtain a positive sample classification image sequence.

[0058] The third sub-step involves adding the positive sample classification image sequence to the initial classification image sequence set included in the initial image cache library.

[0059] The fourth sub-step involves creating a new initial binary classification sub-model corresponding to the aforementioned new initial classification image sequence. This can be achieved by defining a fully connected layer without bias parameters as the new initial binary classification sub-model.

[0060] The fifth sub-step involves adding the new initial binary sub-model to the set of initial binary sub-models.

[0061] In practice, when a new item category appears, the above sub-steps can promptly increase the number of initial classification image sequences in the initial classification image sequence set included in the initial image cache library, and can promptly increase the number of initial binary classification sub-models, so that the initial item classification model can promptly identify the new item category.

[0062] In some optional implementations of certain embodiments, the execution entity determining the initial classification average similarity corresponding to each initial classification image sequence in the initial classification image sequence set included in the initial image cache library may include the following steps:

[0063] The first step involves extracting features from each initial classification image in the aforementioned initial classification image sequence to generate a global feature vector for the initial classification image, resulting in a sequence of global feature vectors for the initial classification images. This can be achieved using a pre-defined image feature extraction model.

[0064] As an example, the preset image feature extraction model mentioned above can be, but is not limited to, at least one of the following: ResNet18 (residual neural network) model, MobileNetV2 (lightweight neural network) model, or ViT (VisionTransformer) model.

[0065] The second step involves performing the following determination sub-step for each initial classification image global feature vector in the above initial classification image global feature vector sequence to generate initial classification similarities, resulting in an initial classification similarity sequence:

[0066] The first sub-step involves determining the classification similarity vector product by multiplying the global feature vector of the sample item image with the global feature vector of the initial classification image.

[0067] The second sub-step involves determining the classification similarity modulus by multiplying the modulus of the global feature vector of the sample item image with the modulus of the global feature vector of the initial classification image.

[0068] The third sub-step is to determine the ratio of the above classification similarity vector product to the above classification similarity modulus product as the above initial classification similarity.

[0069] The third step is to determine the mean of each initial classification similarity in the above initial classification similarity sequence as the above initial classification average similarity.

[0070] In practice, the aforementioned content regarding the generation of positive sample classification image sequences and negative sample classification image sequence sets serves as an inventive point of this disclosure, solving the third technical problem mentioned in the background art: "waste of model training time." Factors leading to increased model training time often include: during the training of a full-scale learning model, the correct category of each sample image needs to be manually labeled. When manual labeling is incorrect, the sample images need to be relabeled, resulting in wasted labeling time. Solving these factors can reduce the waste of model training time. To achieve this, this disclosure can determine the similarity between the sample image and each initial classification image in the initial image cache using the feature vectors of the sample image and each initial classification image in the initial image cache. Then, it labels the positive sample image sequence corresponding to the sample image category from the initial image cache, reducing the waste of labeling time and thus reducing the waste of model training time.

[0071] In some optional implementations of certain embodiments, the execution entity adjusts the initial item image feature extraction model and the target item classification sub-model based on a preset loss function, a set of positive sample classification image sequences, and a set of negative sample classification image sequences to obtain adjusted initial item image feature extraction models and target item classification sub-models. This may include the following steps:

[0072] The first step involves selecting a predetermined number of positive and negative sample classification images from the aforementioned positive and negative sample classification image sequences, respectively, as the target positive and negative sample classification image sequences. The following adjustment sub-steps are then performed on the initial item image feature extraction model and the target item classification sub-model:

[0073] The first sub-step involves inputting the target positive sample classification image sequence and the target negative sample classification image sequence into the initial item image feature extraction model to obtain the target positive sample image feature sequence and the target negative sample classification image feature sequence.

[0074] As an example, the preset number could be 5.

[0075] The second sub-step involves inputting the aforementioned target positive sample image feature sequence and the aforementioned target negative sample classification image feature sequence into the aforementioned target item classification sub-model to obtain a set of sample classification prediction values.

[0076] The third sub-step involves generating a set of correctly classified sample values ​​based on the aforementioned target positive sample classification image sequence and the aforementioned target negative sample classification image sequence. Specifically, a first preset score can be determined as the correctly classified sample value corresponding to the target positive sample classification image in the aforementioned target positive sample classification image sequence, and a second preset score can be determined as the correctly classified sample value corresponding to the target negative sample classification image in the aforementioned target negative sample classification image sequence, thus obtaining the aforementioned set of correctly classified sample values.

[0077] As an example, the first preset score value mentioned above can be 1. The second preset score value mentioned above can be 0.

[0078] The third sub-step involves determining the classification loss values ​​for the predicted sample classification values ​​and the correct sample classification values, based on a preset loss function.

[0079] The fourth sub-step involves determining, in response to the determination that the classification loss value is less than a preset threshold, the initial item image feature extraction model and the target item classification sub-model are identified as the adjusted initial item image feature extraction model and target item classification sub-model. The value of the preset threshold is not limited.

[0080] Optionally, the aforementioned implementing entity may also perform the following steps:

[0081] The first step is to determine the gradient value of the activation parameter corresponding to the classification loss value based on the preset loss function, in response to the determination that the classification loss value is greater than or equal to the preset threshold.

[0082] The second step involves adjusting the relevant parameters in the initial item image feature extraction model and the target item classification sub-model based on the aforementioned activation parameter gradient values, the preset learning rate of the feature extraction model, and the preset learning rate of the target binary classification sub-model. This yields adjusted initial item image feature extraction models and target item classification sub-models, which are then used to perform the adjustment steps again. Specifically, a preset adjustment algorithm can be used to adjust the relevant parameters in the initial item image feature extraction model and the target item classification sub-model.

[0083] As an example, the learning rate of the preset feature extraction model can be 0.0002. The learning rate of the preset target binary classification sub-model can be 0.00002. The preset adjustment algorithm can be the AdamW optimizer algorithm.

[0084] The relevant content of step 102, as an inventive point of this disclosure, solves the second technical problem mentioned in the background art: "reduced accuracy of item classification." The factors leading to reduced accuracy in item classification are often as follows: the full-scale learning model has only one classifier, and the model parameters are updated completely; even parameters that meet the training requirements may be changed, resulting in reduced accuracy of the full-scale recognition model in identifying item categories. Solving these factors can improve the accuracy of item classification. To achieve this, this disclosure can adjust and optimize only the parameters of the initial image feature extraction model and the parameters of the initial binary classification sub-model corresponding to the category of the sample item image during a single model training, while maintaining the parameters of other initial binary classification sub-models. This improves the accuracy of optimizing the parameters required for item category recognition, thereby increasing the accuracy of the trained item classification model in identifying item categories, and ultimately improving the accuracy of item classification.

[0085] Step 103: Input the global feature vector of the item image into the item classification sub-model to obtain item category information.

[0086] In some embodiments, the execution entity may input the global feature vector of the item image into the item classification sub-model to obtain item category information. The item category information represents the category of the item in the item image.

[0087] Step 104: Send the item category information to the display terminal for categorized storage of items.

[0088] In some embodiments, the aforementioned execution entity may send the aforementioned item category information to a display terminal for the categorized storage of items.

[0089] The above embodiments of this disclosure have the following beneficial effects: the item classification method of some embodiments of this disclosure can improve the accuracy of item classification. Specifically, the reason for the decrease in the accuracy of item classification is that the full-learning model only uses one classifier to identify the category information in the image. When there are images of different items with high similarity, the accuracy of the single classifier will decrease. Based on this, the item classification method of some embodiments of this disclosure firstly acquires an item image. Secondly, the item image is input into the item image feature extraction model included in the pre-trained item classification model to obtain the global feature vector of the item image. The pre-trained item classification model also includes an item classification sub-model. Thus, the feature information of the item image can be obtained to identify the category of the item in the item image. Then, the global feature vector of the item image is input into the item classification sub-model to obtain the item category information. Thus, the feature information of the item image can be classified by the item classification sub-model obtained by splicing multiple binary classification sub-models. Finally, the item category information is sent to the display terminal for classified storage of items. Thus, items can be classified according to the identified item category information. Therefore, some of the item classification methods disclosed herein can identify the category information of items in item images through a pre-trained item classification model. The pre-trained item classification model includes an image feature extraction model and an item classification sub-model synthesized by splicing multiple binary sub-models, which can improve the accuracy of category information identification, thereby improving the accuracy of item classification.

[0090] Further reference Figure 2 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of an item sorting device, which are similar to... Figure 1 Corresponding to the method embodiments shown, this item sorting device can be specifically applied to various electronic devices.

[0091] like Figure 2 As shown, some embodiments of the item classification device 200 include: an acquisition unit 201, a first input unit 202, a second input unit 203, and a sending unit 204. The acquisition unit 201 is configured to acquire an item image; the first input unit 202 is configured to input the item image into a pre-trained item classification model, which includes an item image feature extraction model, to obtain a global feature vector of the item image; the pre-trained item classification model further includes an item classification sub-model; the second input unit 203 is configured to input the global feature vector of the item image into the item classification sub-model to obtain item category information; and the sending unit 204 is configured to send the item category information to a display terminal for item classification and storage.

[0092] It is understandable that the units described in the item sorting device 200 are consistent with the reference. Figure 1 The steps in the described item classification method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the item classification method also apply to the item classification device 200 and the units contained therein, and will not be repeated here.

[0093] The following is for reference. Figure 3 This document illustrates a structural schematic of an electronic device 300 suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 3 The terminal device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0094] like Figure 3 As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0095] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 3 Each box shown can represent a device or multiple devices as needed.

[0096] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined in the methods of some embodiments of this disclosure.

[0097] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0098] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0099] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: acquire an image of an object; input the image of the object into a pre-trained object classification model including an object image feature extraction model to obtain a global feature vector of the object image, wherein the pre-trained object classification model further includes an object classification sub-model; input the global feature vector of the object image into the object classification sub-model to obtain object category information; and send the object category information to a display terminal for categorized storage of the objects.

[0100] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0101] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0102] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including an acquisition unit, a first input unit, a second input unit, and a transmission unit. The names of these units do not necessarily limit the specific unit; for example, the acquisition unit may also be described as a "unit for acquiring an image of an object."

[0103] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0104] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. An article classification method, comprising: acquiring an article image; inputting the article image into an article image feature extraction model included in a pre-trained article classification model to obtain an article image global feature vector, wherein the pre-trained article classification model further comprises an article classification sub-model, and the article classification model is trained by the following steps: acquiring a sample article image sequence, an initial image cache library and an initial article category recognition model, wherein the initial image cache library comprises an initial classification image sequence set, the initial article category recognition model comprises an initial article image feature extraction model and an initial article classification sub-model, the initial article classification sub-model comprises an initial binary classification model set, and the initial classification image sequence in the initial classification image sequence set corresponds to the initial binary classification model in the initial binary classification model set in a one-to-one manner; selecting a sample article image from the sample article image sequence, and performing the following training steps on the initial article category recognition model: inputting the sample article image into the initial article image feature extraction model included in the initial article category recognition model to obtain a sample article image global feature vector; generating a positive sample classification image sequence and a negative sample classification image sequence set based on the sample article image global feature vector and the initial image cache library, comprising: determining an initial classification average similarity corresponding to the sample article image global feature vector and each initial classification image sequence in the initial classification image sequence set included in the initial image cache library to obtain an initial classification average similarity set; performing sorting processing on the initial classification average similarity set to obtain an initial classification average similarity sequence; determining the first initial classification average similarity in the initial classification average similarity sequence as a positive sample average similarity; adding the sample article image to the initial classification image sequence corresponding to the positive sample average similarity in the initial classification image sequence set included in the initial image cache library to obtain the positive sample classification image sequence; determining each initial classification average similarity from the second to the preset sequence number in the initial classification average similarity sequence as a negative sample average similarity set; determining the initial classification image sequence corresponding to each negative sample average similarity in the negative sample average similarity set in the initial classification image sequence set included in the initial image cache library as the negative sample classification image sequence set; determining the initial binary classification model corresponding to the positive sample classification image sequence in the initial binary classification model set included in the initial article category recognition model as a target article classification sub-model; adjusting the initial article image feature extraction model and the target article classification sub-model based on a preset loss function, the positive sample classification image sequence and the negative sample classification image sequence set to obtain an adjusted initial article image feature extraction model and the target article classification sub-model; in response to determining that the sample article image sequence satisfies a preset condition, determining the initial article category recognition model as an article category recognition model; and inputting the article image global feature vector into the article classification sub-model to obtain article category information. The article category information is sent to a display terminal for classified storage of the article.

2. The method of claim 1, wherein, The training step further includes: In response to determining that the sample article image sequence does not satisfy the preset condition, the training step is performed again.

3. The method of claim 1, wherein the adjusting of the initial article image feature extraction model and the target article classification sub-model based on the preset loss function, the positive sample classification image sequence and the negative sample classification image sequence set comprises: selecting a preset number of positive sample classification images and negative sample classification images from the positive sample classification image sequence and the negative sample classification image sequence set respectively as a target positive sample classification image sequence and a target negative sample classification image sequence; and performing the following adjustment steps on the initial article image feature extraction model and the target article classification sub-model: inputting the target positive sample classification image sequence and the target negative sample classification image sequence into the initial article image feature extraction model to obtain a target positive sample image feature sequence and a target negative sample classification image feature sequence; inputting the target positive sample image feature sequence and the target negative sample classification image feature sequence into the target article classification sub-model to obtain a sample classification prediction value set; generating a sample classification correct value set based on the target positive sample classification image sequence and the target negative sample classification image sequence; determining a classification loss value of the sample classification prediction value set and the sample classification correct value set based on a preset loss function; and in response to determining that the classification loss value is less than a preset threshold, determining the initial article image feature extraction model and the target article classification sub-model as an adjusted initial article image feature extraction model and an adjusted target article classification sub-model. The adjustment step further includes: in response to determining that the classification loss value is greater than or equal to the preset threshold, determining an activation parameter gradient value corresponding to the classification loss value based on the preset loss function; and adjusting related parameters in the initial article image feature extraction model and the target article classification sub-model based on the activation parameter gradient value, a preset feature extraction model learning rate and a preset target binary classification sub-model learning rate to obtain an adjusted initial article image feature extraction model and an adjusted target article classification sub-model, so as to perform the adjustment step again.

5. An article classification apparatus, comprising: an acquisition unit configured to acquire an article image; a first input unit configured to input the article image into an article image feature extraction model included in a pre-trained article classification model to obtain an article image global feature vector, wherein the pre-trained article classification model further comprises an article classification sub-model, and the pre-trained article classification model is trained by the following steps: training an initial article image feature extraction model and an initial article classification sub-model based on a preset loss function, a positive sample classification image sequence and a negative sample classification image sequence set; inputting a preset number of positive sample classification images and negative sample classification images from the positive sample classification image sequence and the negative sample classification image sequence set respectively into the initial article image feature extraction model to obtain a positive sample image feature sequence and a negative sample classification image feature sequence; inputting the positive sample image feature sequence and the negative sample classification image feature sequence into the initial article classification sub-model to obtain a sample classification prediction value set; generating a sample classification correct value set based on the positive sample classification image sequence and the negative sample classification image sequence; determining a classification loss value of the sample classification prediction value set and the sample classification correct value set based on a preset loss function; in response to determining that the classification loss value is less than a preset threshold, determining the initial article image feature extraction model and the initial article classification sub-model as an adjusted initial article image feature extraction model and an adjusted initial article classification sub-model; in response to determining that the classification loss value is greater than or equal to the preset threshold, determining an activation parameter gradient value corresponding to the classification loss value based on the preset loss function; and adjusting related parameters in the initial article image feature extraction model and the initial article classification sub-model based on the activation parameter gradient value, a preset feature extraction model learning rate and a preset target binary classification sub-model learning rate to obtain an adjusted initial article image feature extraction model and an adjusted initial article classification sub-model, so as to perform the adjustment step again. ​ ​ ​ 4. The method of claim 3, wherein, ​ ​ ​ ​ ​ ​ acquire a sample object image sequence, an initial image cache library and an initial object category recognition model, wherein the initial image cache library comprises an initial classification image sequence set, and the initial object category recognition model comprises an initial object image feature extraction model and an initial object classification sub-model, the initial object classification sub-model comprises an initial binary classification model set, and each initial classification image sequence in the initial classification image sequence set corresponds to an initial binary classification sub-model in the initial binary classification sub-model set; select a sample object image from the sample object image sequence, and perform the following training steps on the initial object category recognition model: input the sample object image into the initial object image feature extraction model included in the initial object category recognition model to obtain a sample object image global feature vector; based on the sample object image global feature vector and the initial image cache library, generate a positive sample classification image sequence and a negative sample classification image sequence set, comprising: determine an initial classification average similarity corresponding to the sample object image global feature vector and each initial classification image sequence in the initial classification image sequence set included in the initial image cache library to obtain an initial classification average similarity set; perform sorting processing on the initial classification average similarity set to obtain an initial classification average similarity sequence; determine the first initial classification average similarity in the initial classification average similarity sequence as a positive sample average similarity; add the sample object image to an initial classification image sequence corresponding to the positive sample average similarity in the initial classification image sequence set included in the initial image cache library to obtain the positive sample classification image sequence; determine each initial classification average similarity from the second to the preset sequence number in the initial classification average similarity sequence as a negative sample average similarity set; determine the initial classification image sequence corresponding to each negative sample average similarity in the negative sample average similarity set in the initial classification image sequence set included in the initial image cache library as the negative sample classification image sequence set; determine the initial binary classification sub-model corresponding to the positive sample classification image sequence in the initial binary classification sub-model set included in the initial object category recognition model as a target object classification sub-model; based on a preset loss function, the positive sample classification image sequence and the negative sample classification image sequence set, adjust the initial object image feature extraction model and the target object classification sub-model to obtain an adjusted initial object image feature extraction model and target object classification sub-model; in response to determining that the sample object image sequence meets a preset condition, determine the initial object category recognition model as an object category recognition model; a second input unit configured to input the object image global feature vector into the object classification sub-model to obtain object category information; a sending unit configured to send the object category information to a display terminal for classified storage of the object.

6. An electronic device, comprising: one or more processors; storage having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement a method as claimed in any of claims 1-4.

7. A computer readable medium having stored thereon a computer program, wherein, The computer program, which when executed by a processor, implements a method as claimed in any of claims 1-4.

Images