Article attribute discrimination method and device
Through the artificial intelligence model, the video, image and text data of items are processed, and the item attributes are automatically detected, which solves the problems of manual marking data errors and low efficiency in the logistics system, and achieves efficient and accurate item attribute data acquisition and verification.
Patent Information
- Application Number
- CN202411088978.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-12
AI Technical Summary
The marking of item attribute data in existing logistics systems depends on manual methods, which makes it difficult to ensure data accuracy, and manual regular sampling is inefficient and easy to miss errors.
Using the object attribute discrimination method based on the artificial intelligence model, the item attribute data is automatically determined through multimodal data of video, images and text, and the final attribute data is obtained in combination with multiple initial attribute data, instead of manual marking methods, improving efficiency and ensuring correctness , and use this data to perform full verification.
It realizes automatic detection of item attribute data, improves the accuracy and efficiency of data, avoids the omission of manual marking errors, and improves the accuracy and efficiency of the logistics process.
Smart Images

Figure CN118940217A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of smart logistics technology, and in particular to a method and device for determining the attributes of an item. Background Art
[0002] In the logistics process such as order production and goods delivery, the system generally performs cost calculation, vehicle dispatch and other matters based on the attribute data of the length, width, height, volume, weight and other attributes marked in advance for the goods, so it is necessary to ensure the correctness of the attribute data of the goods. In the prior art, the attribute data of the goods is generally marked manually, and the correctness is difficult to guarantee. In addition, the manually marked data is generally verified by regular manual sampling, which is inefficient and easy to miss the wrongly marked data. Summary of the invention
[0003] In view of this, an embodiment of the present invention provides a method and device for distinguishing item attributes, which can automatically determine item attribute data based on an artificial intelligence model and multimodal data such as video, image, text, etc. of the item. The data can also be used as a basis for verifying manually marked data.
[0004] To achieve the above objective, according to one aspect of the present invention, a method for identifying object attributes is provided.
[0005] The object attribute identification method of the embodiment of the present invention includes: obtaining video information, image information and text information of the object to be identified in the preset logistics process; inputting the video information into a pre-trained, video-based attribute identification model to obtain first preliminary attribute data of at least one preset attribute of the object to be identified; inputting the image information into a pre-trained, image-based attribute identification model to obtain second preliminary attribute data of the attribute of the object to be identified; inputting the text information into a pre-trained, text-based attribute identification model to obtain third preliminary attribute data of the attribute of the object to be identified; and combining the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data to obtain the final attribute data of the attribute of the object to be identified.
[0006] Optionally, the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data respectively include: probability values of each attribute value interval corresponding to any attribute of the object to be identified.
[0007] Optionally, the final attribute data of the attribute of the object to be judged is obtained by combining the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data, including: obtaining probability values for the same attribute and the same attribute value range of the attribute in the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data; determining the weighted average of the probability values obtained from the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data according to the weight values pre-configured for the video information, the image information and the text information respectively; determining the attribute value range with the largest weighted average in the same attribute as the target range of the attribute of the object to be judged; the target range belongs to the final attribute data of the attribute of the object to be judged.
[0008] Optionally, the final attribute data of the attribute of the item to be identified is obtained by combining the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data, including: for the same attribute, scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the first preliminary attribute data; scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the second preliminary attribute data; scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the third preliminary attribute data; wherein the score of each scoring is equal; and the attribute value interval with the highest score for the attribute is determined as the target interval of the attribute for the item to be identified.
[0009] Optionally, the method further comprises: comparing a target interval of any attribute of the item to be identified with a pre-marked attribute value of the attribute; if the pre-marked attribute value does not belong to the target interval, determining the pre-marked attribute value as abnormal data.
[0010] Optionally, the method further includes: using a video-based attribute discrimination model to obtain first preliminary attribute data of at least one specific item in an attribute, using an image-based attribute discrimination model to obtain second preliminary attribute data of the specific item in an attribute, and using a text-based attribute discrimination model to obtain third preliminary attribute data of the specific item in an attribute; wherein the specific item and the item to be discriminated belong to the same item type, and items of the same item type have the same attribute value in the attribute; traversing the items to be discriminated and the specific items: for the same attribute, scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the first preliminary attribute data of the same item, scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the second preliminary attribute data of the same item, and scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the third preliminary attribute data of the same item; wherein the score of each scoring is equal; and the attribute value interval with the highest score for the attribute is determined as the target interval of the item type for the attribute.
[0011] Optionally, the attributes include at least one of the following: length, width, height, volume, weight, density; the logistics process includes at least one of the following: picking process, sorting process, review process, and packaging process in order production.
[0012] To achieve the above objective, according to another aspect of the present invention, a device for determining the attributes of an object is provided.
[0013] The device for identifying the property of an item in the embodiment of the present invention may include: a multimodal information acquisition unit, a model calculation unit and a data combination unit.
[0014] Among them, the multimodal information acquisition unit is used to obtain video information, image information and text information of the item to be identified in the preset logistics process; the model calculation unit is used to input the video information into a pre-trained, video-based attribute discrimination model to obtain the first preliminary attribute data of the item to be identified in at least one preset attribute; the image information is input into a pre-trained, image-based attribute discrimination model to obtain the second preliminary attribute data of the item to be identified in the attribute; the text information is input into a pre-trained, text-based attribute discrimination model to obtain the third preliminary attribute data of the item to be identified in the attribute; the data combination unit is used to combine the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data to obtain the final attribute data of the item to be identified in the attribute.
[0015] Optionally, the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data respectively include: probability values of each attribute value interval corresponding to any attribute of the object to be identified.
[0016] Optionally, the data combination unit can be further used to: obtain probability values for the same attribute and the same attribute value range of the attribute in the first preliminary judgment attribute data, the second preliminary judgment attribute data and the third preliminary judgment attribute data; determine the weighted average of the probability values obtained from the first preliminary judgment attribute data, the second preliminary judgment attribute data and the third preliminary judgment attribute data according to the weight values pre-configured for the video information, the image information and the text information respectively; determine the attribute value range with the largest weighted average in the same attribute as the target range of the attribute for the item to be judged; the target range belongs to the final attribute data of the attribute for the item to be judged.
[0017] Optionally, the data combination unit can be further used to: for the same attribute, perform scoring for the attribute value interval corresponding to the maximum probability value of the attribute in the first preliminary attribute data; perform scoring for the attribute value interval corresponding to the maximum probability value of the attribute in the second preliminary attribute data; perform scoring for the attribute value interval corresponding to the maximum probability value of the attribute in the third preliminary attribute data; wherein the score of each scoring is equal; and determine the attribute value interval with the highest score for the attribute as the target interval of the attribute for the item to be identified.
[0018] Optionally, the device may further include a verification unit for comparing a target interval of any attribute of the item to be identified with a pre-marked attribute value of the attribute; if the pre-marked attribute value does not belong to the target interval, determining the pre-marked attribute value as abnormal data.
[0019] Optionally, the device may also include a type attribute calculation unit, which is used to: use a video-based attribute discrimination model to obtain first preliminary attribute data of at least one specific item in an attribute, use an image-based attribute discrimination model to obtain second preliminary attribute data of the specific item in an attribute, and use a text-based attribute discrimination model to obtain third preliminary attribute data of the specific item in an attribute; wherein the specific item and the item to be discriminated belong to the same item type, and items of the same item type have the same attribute value in the attribute; traverse the items to be discriminated and the specific items: for the same attribute, score the attribute value interval corresponding to the maximum probability value of the attribute in the first preliminary attribute data of the same item, score the attribute value interval corresponding to the maximum probability value of the attribute in the second preliminary attribute data of the same item, and score the attribute value interval corresponding to the maximum probability value of the attribute in the third preliminary attribute data of the same item; wherein the score of each scoring is equal; determine the attribute value interval with the highest score for the attribute as the target interval of the item type for the attribute.
[0020] Optionally, the attributes include at least one of the following: length, width, height, volume, weight, density; the logistics process includes at least one of the following: picking process, sorting process, review process, and packaging process in order production.
[0021] To achieve the above objective, according to another aspect of the present invention, an electronic device is provided.
[0022] An electronic device of the present invention includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the object attribute identification method provided by the present invention.
[0023] To achieve the above objective, according to yet another aspect of the present invention, a computer-readable storage medium is provided.
[0024] A computer-readable storage medium of the present invention stores a computer program, which, when executed by a processor, implements the object attribute identification method provided by the present invention.
[0025] To achieve the above objective, according to yet another aspect of the present invention, a computer program product is provided.
[0026] A computer program product of the present invention includes a computer program, and when the computer program is executed by a processor, the object attribute identification method provided by the present invention is implemented.
[0027] According to the technical solution of the present invention, the embodiments of the above invention have the following advantages or beneficial effects: The embodiment of the present invention processes multimodal data such as video, image, text, etc. of the object to be identified based on an artificial intelligence model, and obtains the final attribute data of the object to be identified by combining the three preliminary attribute data output by the model, thereby realizing automatic detection of the attribute data of the object to be identified. The above method can replace the existing manual marking method to obtain the attribute data of the object, which is efficient and can ensure correctness; the data obtained by the above method can also be used as a verification basis for the manually marked data to perform full and efficient verification of the manually marked data, which can avoid missing the data with manual marking errors.
[0028] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention. Figure 1 Schematic diagram of the main steps of the method for determining the attributes of an object in an embodiment of the present invention; Figure 2 is a schematic diagram of specific execution steps of the method for determining the attributes of an item in an embodiment of the present invention; Figure 3 It is a schematic diagram of components of an object attribute determination device in an embodiment of the present invention; Figure 4 is an exemplary system architecture diagram in which embodiments of the present invention may be applied; Figure 5 It is a schematic diagram of the structure of an electronic device used to implement the object attribute determination method in an embodiment of the present invention. DETAILED DESCRIPTION
[0030] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0031] It should be pointed out that the embodiments of the present invention and the technical features therein may be combined with each other without conflict.
[0032] Figure 1Schematic diagram of the main steps of the method for distinguishing the attributes of an object in an embodiment of the present invention.
[0033] like Figure 1 As shown, the object attribute identification method of the embodiment of the present invention can be executed by the server, and the specific execution steps are as follows: Step S101: Obtaining video information, image information and text information of the item to be identified during the preset logistics process. In this step, when the server performs attribute detection on the item to be identified, it first obtains video information, image information and text information of the item to be identified during the preset logistics process. This information is related to the attribute data of the item to be identified, and the attribute data of the item to be identified can be detected by analyzing this information.
[0034] Step S102: Input the video information into a pre-trained, video-based attribute discrimination model to obtain first preliminary attribute data of at least one preset attribute of the item to be discriminated; input the image information into a pre-trained, image-based attribute discrimination model to obtain second preliminary attribute data of the attribute of the item to be discriminated; input the text information into a pre-trained, text-based attribute discrimination model to obtain third preliminary attribute data of the attribute of the item to be discriminated.
[0035] In this step, the server can input the video information, image information and text information of the object to be identified into the pre-trained attribute discrimination models for video, image and text respectively, and obtain three preliminary attribute data based on video, image and text output by each attribute discrimination model.
[0036] Step S103: Combine the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data to obtain the final attribute data of the item to be identified. In this step, the server fuses the three preliminary attribute data to obtain the final prediction value, that is, the final attribute data of the item to be identified in the above attributes, thereby realizing the automatic detection of the attribute data of the item to be identified based on multimodal item information of video, image and text and artificial intelligence model.
[0037] Figure 2 is a schematic diagram of specific execution steps of the method for determining the attributes of an item in an embodiment of the present invention, see Figure 2 .
[0038] Step S201: Execute the training of the attribute discrimination model based on the training samples.
[0039] In this step, three attribute discrimination models can be trained based on the training samples, namely, a video-based attribute discrimination model, an image-based attribute discrimination model, and a text-based attribute discrimination model. Each model can adopt any applicable, known artificial intelligence algorithm for classification or regression. The training input data of the video-based attribute discrimination model is the video information of the object, the training input data of the image-based attribute discrimination model is the image information of the object, and the training input data of the text-based attribute discrimination model is the text information of the object. The training labels of the three attribute discrimination models are all predetermined real attribute data of the object. Exemplarily, the real attribute data of the object can be a continuous true value of the attribute of the object, or a discrete true value of the interval of the attribute value of the object.
[0040] Step S202: The server obtains multimodal information of the item to be identified during the preset logistics process.
[0041] The above logistics process can be any logistics process of the item to be identified, such as one or more of the following in order production: picking process, sorting process, review process, packaging process, any process in the distribution link or any process in the collection link. The above attributes can be any applicable physical attributes of the item, such as one or more of the following attributes related to the logistics link: length, width, height, volume, weight, density.
[0042] It can be understood that, from the perspective of video information, when items with different attribute data appear in a video, the attribute data is correlated with the video content. For example, when a picker in the picking process carries a heavier item, a more laborious and time-consuming video content will be presented (for example, carrying with both hands, bending over, and taking more than 2 seconds), and when carrying a lighter item, a more relaxed video content will be presented (for example, carrying with one hand, standing straight, and taking less than 1.5 seconds); image information can include images that reflect the relative position and posture of the item and the surrounding environment, such as an image of an item placed on a conveyor line. This image information can indicate the value of certain attributes of the item; text information can be information such as the rules and dimensions of any material or object involved in the logistics process of the item, such as the number of items or packaging boxes or packaging bags used or the specification information, which can be obtained from a database that stores relevant material or object information. In this way, the item attributes can be detected by analyzing the above video, image, and text information related to the item attributes.
[0043] Step S203: The server inputs the multimodal information of the item to be identified into the attribute identification model for calculation.
[0044] In this step, each modal information of the object is input into the corresponding model for calculation, that is, the server inputs the video information into a pre-trained, video-based attribute discrimination model, inputs the image information into a pre-trained, image-based attribute discrimination model, and inputs the text information into a pre-trained, text-based attribute discrimination model.
[0045] Step S204: The server obtains the preliminary attribute data of the object to be identified output by each attribute identification model.
[0046] In this step, the server can obtain the first preliminary attribute data of each attribute of the item to be identified output by the video-based attribute discrimination model, obtain the second preliminary attribute data of each attribute of the item to be identified output by the image-based attribute discrimination model; and obtain the third preliminary attribute data of each attribute of the item to be identified output by the text-based attribute discrimination model. The above three preliminary attribute data can only represent the preliminary attribute judgment of a certain mode, and the three preliminary attribute data need to be combined to obtain the final attribute data as the final model calculation value.
[0047] As a preferred solution, when the final attribute data of the item is the attribute value interval to which the attribute value of the item belongs (which may be referred to as the target interval), the various preliminary attribute data in this step may be the probability values of the attribute value intervals of the item corresponding to any attribute. For example, the first preliminary attribute data of the length attribute of the item to be identified is: {0.01, 0.1, 0.88, 0.01}, which indicates the probability values of the item to be identified corresponding to four length intervals A, B, C, and D arranged in ascending order. In one embodiment, when the final attribute data of the item is the attribute value as continuous data, the various preliminary attribute data in this step may be the probability values of the attribute value intervals of the item corresponding to any attribute, or may be a single attribute value as continuous data.
[0048] Step S205: The server combines a plurality of preliminary attribute data to obtain final attribute data of the object to be identified.
[0049] In this step, when the various preliminary attribute data of the item to be identified are single attribute values as continuous data, the server can calculate the average value of each attribute value of the same attribute as the final attribute data of the item to be identified at the attribute. When the various preliminary attribute data of the item to be identified are probability values corresponding to each attribute value interval of any attribute, the final attribute data of the item to be identified can be calculated in the following two ways.
[0050] In the first method, the server first obtains the probability values for the same attribute and the same attribute value interval of the attribute in the first preliminary attribute data, the second preliminary attribute data, and the third preliminary attribute data. Continuing with the above example, for example, the server obtains three probability values for the attribute value interval C of the length attribute, which are 0.88 in the first preliminary attribute data, 0.80 in the second preliminary attribute data, and 0.89 in the third preliminary attribute data. Then, the server determines the weighted average of the probability values obtained from the first preliminary attribute data, the second preliminary attribute data, and the third preliminary attribute data according to the weight values pre-configured for the video information, the image information, and the text information, respectively. The above weight values can be fixed weight values for videos, images, and texts of global items, or dynamic weight values for videos, images, and texts formulated for each item. Continuing with the above example, the weight values of video, image, and text are 0.4, 0.3, and 0.3, respectively, then the weighted average of the three probability values is 0.4*0.88+0.3*0.80+0.3*0.89. After that, the server determines the attribute value interval with the largest weighted average value in the same attribute as the target interval of the item to be identified in the attribute, and the target interval belongs to the final attribute data of the item to be identified in the attribute. Continuing with the above example, the server determines the attribute value interval C with the largest weighted average value in the length attribute as the target interval of the item to be identified in the length attribute.
[0051] In the second method, for the same attribute, the server performs scoring for the attribute value interval corresponding to the maximum probability value of the attribute in the first preliminary attribute data, performs scoring for the attribute value interval corresponding to the maximum probability value of the attribute in the second preliminary attribute data, and performs scoring for the attribute value interval corresponding to the maximum probability value of the attribute in the third preliminary attribute data, and the scores of each scoring are equal. Continuing with the above example, taking one point each time as an example, the server gives 1 point to the attribute value interval C corresponding to the maximum probability value of the length attribute in the first preliminary attribute data, 1 point to the attribute value interval C corresponding to the maximum probability value of the length attribute in the second preliminary attribute data, and 1 point to the attribute value interval C corresponding to the maximum probability value of the length attribute in the third preliminary attribute data. Finally, the attribute value interval C has a total of 3 points, and the other attribute value intervals are all zero points. After that, the server determines the attribute value interval with the highest score for the attribute as the target interval of the item to be identified in the attribute. In the above example, the server directly determines the attribute value interval C as the target interval of the length attribute of the item to be identified.
[0052] In this way, automatic model detection of the attribute data of the item to be identified can be achieved through a variety of methods, and the detection results can replace the existing manual labeling method to obtain the attribute data of the item, which is highly efficient and can ensure accuracy.
[0053] Step S206: The server verifies the pre-marked attribute value according to the final attribute data of the item to be identified.
[0054] In this step, the server can compare the final attribute data obtained through the above steps with the pre-marked attribute value (which can be manually marked), thereby verifying the pre-marked attribute value of the same item. Preferably, the server compares the target interval of any attribute of the item to be identified with the pre-marked attribute value of the attribute; when the pre-marked attribute value does not belong to the target interval, the server determines the pre-marked attribute value as abnormal data. In another implementation, the server can also input the final attribute data and the pre-marked attribute value of the item to be identified into a pre-trained consistency detection model, and determine whether the two are consistent by comparing whether the consistency detection value output by the model is greater than a preset threshold. For example, when the consistency detection value is greater than the preset threshold, it means that the two are consistent, that is, the pre-marked attribute value is correct. When the consistency detection value is not greater than the preset threshold, it means that the two are inconsistent, that is, the pre-marked attribute value is abnormal. In this way, the manually marked data can be efficiently verified based on the automatic detection results of the model, thereby avoiding the omission of manually marked erroneous data.
[0055] The following describes the attribute detection method and verification method for the same item type. The above same item type refers to a type of items with the same attribute data in each attribute. For example, a stock keeping unit SKU can be used as an item type, or a category that meets the above requirements can be used as an item type. The following takes SKU as an example.
[0056] Step S207: The server performs model calculation on the specific items to obtain preliminary attribute data. In this step, the server performs processing similar to step S203 and step S204 on each specific item to obtain the first preliminary attribute data, the second preliminary attribute data, and the third preliminary attribute data of each specific item. Specifically, the server uses the video-based attribute discrimination model to obtain the first preliminary attribute data of each specific item in the above attributes, uses the image-based attribute discrimination model to obtain the second preliminary attribute data of each specific item in the above attributes, and uses the text-based attribute discrimination model to obtain the third preliminary attribute data of each specific item in the above attributes.
[0057] Step S208: The server determines the target interval of the item type based on the preliminary attribute data of the item to be identified and the specific item.
[0058] In this step, if the final attribute data of each item is a continuous value, the server can determine the average value of the final attribute data of each specific item and the item to be identified as the final attribute data of the item type. If the final attribute data of each item is a target interval, the server can determine the target interval of the item type in two ways.
[0059] In the first method, the server first performs a process similar to step S205 to obtain the target interval for each specific item, and then counts the final attribute data of each specific item and the item to be identified to obtain the target interval of the item type. For example, if the number of specific items is 99, the target intervals of 100 items including the item to be identified are counted, and if interval C has the largest number of target intervals, it is determined as the target interval of the corresponding item type.
[0060] In the second method, the server traverses the items to be identified and the specific items: for the same attribute, scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the first preliminary attribute data of the same item, scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the second preliminary attribute data of the same item, and scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the third preliminary attribute data of the same item, and the score of each scoring is equal. For example, if the score of each scoring is 1 point, the server first judges specific item 1, and gives 1 point to the attribute value interval C corresponding to the maximum probability value of the length attribute in its first preliminary judgment attribute data, gives 1 point to the attribute value interval D corresponding to the maximum probability value of the length attribute in its second preliminary judgment attribute data, and gives 1 point to the attribute value interval C corresponding to the maximum probability value of the length attribute in its third preliminary judgment attribute data; then the server judges specific item 2, and gives 1 point to the attribute value interval C corresponding to the maximum probability value of the length attribute in its first preliminary judgment attribute data, gives 1 point to the attribute value interval B corresponding to the maximum probability value of the length attribute in its second preliminary judgment attribute data, and gives 1 point to the attribute value interval C corresponding to the maximum probability value of the length attribute in its third preliminary judgment attribute data. In this way, each specific item and item to be judged are traversed and scoring is performed. After the scoring is completed, the score of each attribute value interval is judged, and the attribute value interval with the highest score for the attribute is determined as the target interval of the item type in that attribute.
[0061] Step S209: The server verifies the pre-marked attribute value according to the target interval of the item type. In this step, the server can verify the pre-marked attribute value of the item to be identified and each specific item based on the final attribute data (including the target interval) of the item type. Exemplarily, for any item in the item type, the target interval of any attribute of the item type is compared with the pre-marked attribute value of the item in the attribute; if the pre-marked attribute value does not belong to the target interval, the pre-marked attribute value is determined as abnormal data.
[0062] Through the above steps, it is possible to obtain attribute data at the item type level based on the model calculation combined with the statistical analysis of multiple items of the same type, thereby generating more accurate item attribute data. Subsequently, batch automatic marking and batch verification can be performed on items of the same type based on the final attribute data of the same type, thereby achieving more efficient attribute marking and abnormal verification. It can be understood that step S207 may be executed after step S206 or not. Step S207 may be executed after step S201 and in parallel with steps S202 to S204.
[0063] In the technical solution of the embodiment of the present invention, the attribute discrimination model is trained by multimodal data such as video, image, text, etc., and then efficient and accurate automatic detection of item attribute data is achieved based on the attribute discrimination model. Thereafter, the automatically identified attribute data and the manually marked data can be compared to achieve automatic verification, identification and early warning of abnormal marked data, thereby overcoming the shortcomings of the manual periodic sampling method in the prior art.
[0064] It should be noted that the collection, collection, update, analysis, processing, use, transmission, storage and other aspects of user personal information that may be involved in the technical solution of the present invention are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.
[0065] For the above-mentioned method embodiments, for the convenience of description, they are expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the order of the actions described, and some steps can actually be performed in other orders or simultaneously. In addition, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily necessary to implement the present invention.
[0066] In order to better implement the above-mentioned solution of the embodiment of the present invention, relevant devices for implementing the above-mentioned solution are also provided below.
[0067] See also Figure 3 As shown, the device 300 for determining the property of an item provided by the embodiment of the present invention may include: a multimodal information acquisition unit 301 , a model calculation unit 302 , and a data combination unit 303 .
[0068] Among them, the multimodal information acquisition unit 301 is used to obtain video information, image information and text information of the item to be identified in the preset logistics process; the model calculation unit 302 is used to input the video information into a pre-trained, video-based attribute discrimination model to obtain first preliminary attribute data of the item to be identified in at least one preset attribute; input the image information into a pre-trained, image-based attribute discrimination model to obtain second preliminary attribute data of the item to be identified in the attribute; input the text information into a pre-trained, text-based attribute discrimination model to obtain third preliminary attribute data of the item to be identified in the attribute; the data combination unit 303 is used to combine the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data to obtain the final attribute data of the item to be identified in the attribute.
[0069] Preferably, the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data respectively include: probability values of each attribute value interval corresponding to any attribute of the object to be identified.
[0070] In an embodiment of the present invention, the data combination unit 303 can be further used to: obtain probability values for the same attribute and the same attribute value range of the attribute in the first preliminary judgment attribute data, the second preliminary judgment attribute data and the third preliminary judgment attribute data; determine the weighted average value of the probability values obtained from the first preliminary judgment attribute data, the second preliminary judgment attribute data and the third preliminary judgment attribute data according to the weight values pre-configured for the video information, the image information and the text information respectively; determine the attribute value range with the largest weighted average value in the same attribute as the target range of the item to be judged in the attribute; the target range belongs to the final attribute data of the item to be judged in the attribute.
[0071] As a preferred solution, the data combination unit 303 can be further used to: for the same attribute, score the attribute value interval corresponding to the maximum probability value of the attribute in the first preliminary attribute data; score the attribute value interval corresponding to the maximum probability value of the attribute in the second preliminary attribute data; score the attribute value interval corresponding to the maximum probability value of the attribute in the third preliminary attribute data; wherein the score of each scoring is equal; and determine the attribute value interval with the highest score for the attribute as the target interval of the attribute for the item to be judged.
[0072] Preferably, the device 300 may also include a verification unit for comparing a target interval of any attribute of the item to be identified with a pre-marked attribute value of the attribute; if the pre-marked attribute value does not belong to the target interval, determining the pre-marked attribute value as abnormal data.
[0073] In one embodiment, the device 300 may also include a type attribute calculation unit, which is used to: use a video-based attribute discrimination model to obtain first preliminary attribute data of at least one specific item in an attribute, use an image-based attribute discrimination model to obtain second preliminary attribute data of the specific item in an attribute, and use a text-based attribute discrimination model to obtain third preliminary attribute data of the specific item in an attribute; wherein the specific item and the item to be discriminated belong to the same item type, and items of the same item type have the same attribute value in the attribute; traverse the items to be discriminated and the specific items: for the same attribute, score the attribute value interval corresponding to the maximum probability value of the attribute in the first preliminary attribute data of the same item, score the attribute value interval corresponding to the maximum probability value of the attribute in the second preliminary attribute data of the same item, and score the attribute value interval corresponding to the maximum probability value of the attribute in the third preliminary attribute data of the same item; wherein the score of each scoring is equal; determine the attribute value interval with the highest score for the attribute as the target interval of the item type in the attribute.
[0074] In addition, in an embodiment of the present invention, the attributes include at least one of the following: length, width, height, volume, weight, density; the logistics process includes at least one of the following: picking process, sorting process, review process, and packaging process in order production.
[0075] According to the technical solution of the embodiments of the present invention, multimodal data such as video, image, text, etc. of the object to be identified are processed based on an artificial intelligence model, and the three preliminary attribute data output by the model are combined to obtain the final attribute data of the object to be identified, thereby realizing automatic detection of the attribute data of the object to be identified. The above method can replace the existing manual marking method to obtain the attribute data of the object, which is efficient and can ensure correctness; the data obtained by the above method can also be used as a verification basis for the manually marked data to perform full and efficient verification of the manually marked data, which can avoid missing the data with manual marking errors.
[0076] Figure 4 An exemplary system architecture 400 is shown to which the method or device for determining the attribute of an item according to an embodiment of the present invention can be applied.
[0077] like Figure 4 As shown, the system architecture 400 may include terminal devices 401, 402, 403, a network 404 and a server 405 (this architecture is only an example, and the components included in the specific architecture may be adjusted according to the specific application). The network 404 is used to provide a medium for a communication link between the terminal devices 401, 402, 403 and the server 405. The network 404 may include various connection types, such as wired, wireless communication links or optical fiber cables.
[0078] The user may use the terminal devices 401 , 402 , 403 to interact with the server 405 via the network 404 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 401 , 402 , 403 , such as an attribute detection application (only an example).
[0079] The terminal devices 401 , 402 , and 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.
[0080] The server 405 may be a server that provides various services, such as a background server (only an example) that provides support for the attribute detection application operated by the user using the terminal devices 401, 402, 403. The background server may process the received attribute detection request, etc., and feed back the processing result (such as the final attribute data of the item - only an example) to the terminal devices 401, 402, 403.
[0081] It should be noted that the item attribute determination method provided in the embodiment of the present invention is generally executed by the server 405 , and accordingly, the item attribute determination device is generally disposed in the server 405 .
[0082] It should be understood that Figure 4 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.
[0083] The present invention also provides an electronic device. The electronic device of the embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the object attribute identification method provided by the present invention.
[0084] Reference below Figure 5 , which shows a schematic diagram of the structure of a computer system 500 of an electronic device suitable for implementing an embodiment of the present invention. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0085] like Figure 5As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the computer system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0086] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read therefrom is installed into the storage section 508 as needed.
[0087] In particular, according to the embodiments disclosed in the present invention, the process described in the main step diagram above can be implemented as a computer software program. For example, the embodiments of the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the main step diagram. In the above embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit 501, the above functions defined in the system of the present invention are executed.
[0088] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0089] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0090] The units involved in the embodiments of the present invention may be implemented by software or hardware. The units described may also be provided in a processor, for example, may be described as: a processor comprising: a multimodal information acquisition unit, a model calculation unit, and a data combination unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the multimodal information acquisition unit may also be described as a "unit that provides video information, image information, and text information to the model calculation unit."
[0091] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the device, the steps executed by the device include: obtaining video information, image information and text information of the item to be identified in the preset logistics process; inputting the video information into a pre-trained, video-based attribute discrimination model to obtain the first preliminary attribute data of the item to be identified in at least one preset attribute; inputting the image information into a pre-trained, image-based attribute discrimination model to obtain the second preliminary attribute data of the item to be identified in the attribute; inputting the text information into a pre-trained, text-based attribute discrimination model to obtain the third preliminary attribute data of the item to be identified in the attribute; combining the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data to obtain the final attribute data of the item to be identified in the attribute.
[0092] The present invention also provides a computer program product, including a computer program, which implements the object attribute identification method provided by the present invention when the computer program is executed by a processor.
[0093] In the technical solution of the embodiment of the present invention, multimodal data such as video, image, text, etc. of the object to be identified are processed based on an artificial intelligence model, and the three preliminary attribute data output by the model are combined to obtain the final attribute data of the object to be identified, thereby realizing automatic detection of the attribute data of the object to be identified. The above method can replace the existing manual marking method to obtain the attribute data of the object, which is efficient and can ensure correctness; the data obtained by the above method can also be used as a verification basis for the manually marked data to perform full and efficient verification of the manually marked data, which can avoid missing the data with manual marking errors.
[0094] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for determining the attributes of an item, characterized in that: include: Obtain video information, image information and text information of the items to be identified during the preset logistics process; Inputting the video information into a pre-trained, video-based attribute discrimination model to obtain first preliminary attribute data of the object to be discriminated in at least one preset attribute; Inputting the image information into a pre-trained, image-based attribute discrimination model to obtain second preliminary attribute data of the attribute of the object to be discriminated; Inputting the text information into a pre-trained, text-based attribute discrimination model to obtain third preliminary attribute data of the attribute of the item to be discriminated; The final attribute data of the attribute of the object to be identified is obtained by combining the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data.
2. The method according to claim 1, characterized in that The first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data respectively include: probability values of each attribute value interval corresponding to any attribute of the object to be identified.
3. The method according to claim 2, characterized in that The step of combining the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data to obtain the final attribute data of the item to be identified in the attribute includes: Obtaining probability values for the same attribute and the same attribute value interval of the attribute in the first preliminary judged attribute data, the second preliminary judged attribute data, and the third preliminary judged attribute data; Determining a weighted average of probability values obtained from the first preliminary judgement attribute data, the second preliminary judgement attribute data, and the third preliminary judgement attribute data according to weight values pre-configured for the video information, the image information, and the text information, respectively; The attribute value interval with the largest weighted average value in the same attribute is determined as the target interval of the attribute of the item to be identified; the target interval belongs to the final attribute data of the item to be identified in the attribute.
4. The method according to claim 2, characterized in that: The step of combining the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data to obtain the final attribute data of the item to be identified in the attribute includes: For the same attribute, scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the first preliminary attribute data; scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the second preliminary attribute data; scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the third preliminary attribute data; wherein the score of each scoring is equal; The attribute value interval with the highest score for the attribute is determined as the target interval of the attribute for the object to be identified.
5. The method according to claim 3 or 4, characterized in that: The method further comprises: Comparing the target interval of any attribute of the object to be identified with the pre-marked attribute value of the attribute; In the case that the pre-marked attribute value does not belong to the target interval, the pre-marked attribute value is determined as abnormal data.
6. The method according to claim 4, characterized in that The method further comprises: The video-based attribute discrimination model is used to obtain first preliminary attribute data of at least one specific item in the attribute, the image-based attribute discrimination model is used to obtain second preliminary attribute data of the specific item in the attribute, and the text-based attribute discrimination model is used to obtain third preliminary attribute data of the specific item in the attribute; wherein the specific item and the item to be discriminated belong to the same item type, and items of the same item type have the same attribute value in the attribute; Traversing the items to be identified and the specific items: for the same attribute, scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the first preliminary attribute data of the same item, scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the second preliminary attribute data of the same item, and scoring is performed for the attribute value interval corresponding to the maximum probability value of the attribute in the third preliminary attribute data of the same item; wherein the score of each scoring is equal; The attribute value interval with the highest score for the attribute is determined as the target interval for the attribute of the item type.
7. The method according to claim 1, characterized in that The attribute includes at least one of the following: length, width, height, volume, weight, density; The logistics process includes at least one of the following: picking process, sorting process, review process, and packaging process in order production.
8. An object attribute determination device, characterized in that: include: A multimodal information acquisition unit, used to acquire video information, image information and text information of the item to be identified during the pre-placed logistics process; A model calculation unit, used for inputting the video information into a pre-trained, video-based attribute discrimination model to obtain first preliminary attribute data of the object to be discriminated in at least one preset attribute; Inputting the image information into a pre-trained, image-based attribute discrimination model to obtain second preliminary attribute data of the attribute of the object to be discriminated; Inputting the text information into a pre-trained, text-based attribute discrimination model to obtain third preliminary attribute data of the attribute of the item to be discriminated; The data combining unit is used to combine the first preliminary attribute data, the second preliminary attribute data and the third preliminary attribute data to obtain the final attribute data of the object to be judged at the attribute.
9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
