Method and apparatus for storing product information
By performing information quality verification and matching on product information, the accuracy problem of storing domestic product information in foreign repositories has been solved, achieving a more efficient and accurate storage process.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- QI-ANXIN LEGENDSEC INFORMATION TECH (BEIJING) INC
- Filing Date
- 2022-11-04
- Publication Date
- 2026-06-05
AI Technical Summary
When using foreign data repositories to store domestic product information, there are issues with storage results being abnormal or failing due to differences in language, affecting the accuracy of the stored results.
By performing information quality verification on product information, we ensure that it complies with the rules required by Chinese specifications. After passing the verification, the information is matched with standard information, and stored according to the matching results and the degree of change.
It improves the accuracy of product information storage, avoids the storage of erroneous or missing data, optimizes storage space utilization, and ensures the effectiveness of storage activities.
Smart Images

Figure CN115687380B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of storage technology, and in particular to a method and apparatus for storing product information. Background Technology
[0002] With the development of information technology, the amount of product information for manufacturers is gradually increasing. During the storage of product information, for subsequent product analysis and troubleshooting needs, it is often necessary to store the product information in a certain way.
[0003] Currently, foreign data repositories are generally used for storing product information. However, in practice, there are significant differences between foreign and Chinese languages in terms of grammar and text format. This makes it easy for the product information to become abnormal or even fail to be stored due to differences in the text types of the information. Therefore, ensuring the accuracy of storage results when storing domestic product information has become an urgent problem to be solved in this field. Summary of the Invention
[0004] This application provides a method and apparatus for storing product information, the main purpose of which is to solve the problem of low accuracy of storage results in the process of storing domestic product information.
[0005] To address the aforementioned technical problems, this application provides the following technical solutions:
[0006] In a first aspect, this application provides a method for storing product information, the method being applied to a client, the method comprising:
[0007] Obtain product information;
[0008] The product information is subjected to information quality verification, which is used to determine whether the data of the product information in different dimensions conforms to information quality rules, and the information quality rules include rules that conform to Chinese specification requirements;
[0009] When the product information passes the information quality verification, the product information is matched with standard information to obtain a matching result. The standard information is product information pre-stored in the product information database according to the storage standard of the product information database.
[0010] When the matching result is a complete match, the degree of product change is determined based on the product information.
[0011] The product information is added to the product information database based on the degree of change.
[0012] Optionally, before performing information quality verification on the product information, the method further includes:
[0013] The information quality rules are set based on the instruction information, wherein the information quality rules are used to limit the data filling specifications of the product information in each dimension.
[0014] Optionally, after performing information quality verification on the product information, the method further includes:
[0015] When it is determined that the product information fails the information quality verification, the product information is sent to the first manual reviewer, and a first target operation is performed based on the feedback information from the first manual reviewer. The first target operation includes a first operation, a second operation, and a third operation. The first operation is used to modify the abnormal content in the product information, the second operation is used to modify the information quality rules, and the third operation is used to delete the product information.
[0016] When the first target operation is the first operation, the information quality verification is performed again on the modified product information;
[0017] When the first target operation is the second operation, the information quality verification is performed again on the product information using the modified information quality rules.
[0018] Optionally, before performing information quality verification on the product information, the method further includes:
[0019] The product information is cleaned so that the format of the cleaned product information conforms to the data format of the product information database.
[0020] The step of performing information quality verification on the product information includes:
[0021] Perform the information quality verification on the cleaned product information.
[0022] Optionally, the product information includes data corresponding to at least one dimension;
[0023] The step of matching the product information with the standard information to obtain the matching result includes:
[0024] The product information is matched with the standard information according to the data pairs corresponding to each dimension to obtain the matching result.
[0025] Optionally, the step of matching the product information with the standard information according to the data pairs corresponding to each dimension to obtain the matching result includes:
[0026] The product information and the standard information are matched according to the data corresponding to each dimension using regular expressions and similarity algorithms to obtain the matching results.
[0027] Optionally, the step of matching the product information with the standard information according to the data corresponding to each dimension, using regular expressions and similarity algorithms, to obtain the matching result includes:
[0028] Regular expressions are used to determine whether the product information and the standard information are completely consistent in the data corresponding to the common dimension.
[0029] If they are completely consistent, then the product information is determined to be a complete match with the standard information.
[0030] If they are not completely identical, a similarity algorithm is used to calculate the similarity between the product information and the standard information in the same dimension, and the matching result is determined based on the similarity and a preset threshold.
[0031] Optionally, the common dimensions include primary dimensions and secondary dimensions, with the primary dimension being more important than the secondary dimension; the matching results also include complete mismatch and suspected match.
[0032] The step of calculating the similarity between the product information and the standard information in the same dimension using a similarity algorithm, and determining the matching result based on the similarity and a preset threshold, includes:
[0033] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and when the similarity exceeds the preset threshold, the matching result is determined to be the suspected match.
[0034] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and if the similarity does not exceed the preset threshold, then the matching result is determined to be a complete mismatch.
[0035] Optionally, the preset threshold includes a first threshold and a second threshold, wherein the first threshold is greater than the second threshold;
[0036] The step of calculating the similarity between the product information and the standard information in the same dimension using a similarity algorithm, and determining the matching result based on the similarity and a preset threshold, includes:
[0037] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and it is also determined that the product information and the standard information have the same data in the secondary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and when the similarity exceeds the first threshold, the matching result is determined to be a complete match.
[0038] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and it is also determined that the product information and the standard information have the same data in the secondary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and when the similarity is between the first threshold and the second threshold, the matching result is determined to be the suspected match.
[0039] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and that the product information and the standard information also have the same data in the secondary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated. If the similarity is lower than the second threshold, the matching result is determined to be a complete mismatch.
[0040] Optionally, the step of calculating the similarity between the product information and the standard information in the same dimension using a similarity algorithm, and determining the matching result based on the similarity and a preset threshold, includes:
[0041] If, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, then the similarity between the product information and the standard information in all dimensions corresponding to the primary dimension is calculated, and when the similarity exceeds the first threshold, the matching result is determined to be a complete match.
[0042] If, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, then the similarity between the product information and the standard information in all dimensions corresponding to the primary dimension is calculated, and when the similarity is between the first threshold and the second threshold, the matching result is determined to be the suspected match.
[0043] If, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, then the similarity between the product information and the standard information in all dimensions corresponding to the primary dimension is calculated, and when the similarity is lower than the second threshold, the matching result is determined to be a complete mismatch.
[0044] Optionally, after determining whether the product information and the standard information are completely consistent in the common dimension using regular expressions, the method further includes:
[0045] If, based on the regular expression, it is determined that the product information and the standard information are inconsistent only in the data corresponding to the secondary dimension, then the matching result is determined to be a suspected match.
[0046] Optionally, after matching the product information with standard information to obtain a matching result when the product information passes the information quality verification, the method further includes:
[0047] When the matching result is determined to be a suspected match, the product information is sent to a second human reviewer, and a second target operation is performed based on the feedback information from the second human reviewer. The second target operation includes the first operation and the third operation.
[0048] When the second target operation is the first operation, the information quality verification is performed on the product information modified based on the second target operation.
[0049] When the product information modified based on the second target operation passes the information quality verification, a matching operation is performed between the product information modified based on the second target operation and the standard information to obtain the matching result. The matching operation is used to match the data corresponding to each dimension according to regular expressions and similarity algorithms.
[0050] Optionally, after matching the product information with standard information to obtain a matching result when the product information passes the information quality verification, the method further includes:
[0051] When the matching result is determined to be a complete mismatch, the product information is sent to a third human reviewer, and a third target operation is performed based on the feedback information from the third human reviewer. The third target operation includes the first operation, the second operation, the third operation, and a fourth operation, wherein the fourth operation is used to directly add the product information to the product information database.
[0052] When the third target operation is determined to be the first operation, the information quality verification will be performed based on the product information modified by the third target operation. When the information quality verification is passed, the matching operation will be performed based on the product information modified by the third target operation and the standard information to obtain the matching result.
[0053] When the third target operation is determined to be the second operation, the information quality verification is performed again on the product information using the information quality rules modified based on the third target operation. When the information quality verification is passed, the matching operation is performed between the product information modified based on the third target operation and the standard information to obtain the matching result.
[0054] When the third target operation is determined to be the fourth operation, the product information is directly added to the product information database.
[0055] Optionally, the primary dimension includes product identifier and manufacturer identifier, and the secondary dimension includes product version; wherein, the product identifier includes Chinese product identifier and / or foreign language product identifier, and the manufacturer identifier includes Chinese manufacturer identifier and / or foreign language manufacturer identifier.
[0056] Optionally, when the matching result is a perfect match, determining the degree of product change based on the product information includes:
[0057] The dimensions in which the product information differs from the standard information are identified and denoted as the difference dimensions.
[0058] Obtain the data corresponding to the difference dimension, and determine the frequency of occurrence, change frequency, and confidence level of the data corresponding to the difference dimension in the same source information based on the data corresponding to the difference dimension; wherein, the same source information is information that represents the same product as the product information and the standard information; the confidence level is used to characterize the accuracy of the information determined based on the source of the product information;
[0059] The degree of change is calculated based on the occurrence frequency, the change frequency, and the confidence level.
[0060] Optionally, before calculating the degree of change based on the occurrence frequency, the change frequency, and the confidence level, the method further includes:
[0061] The weights of the occurrence frequency, the change frequency, and the confidence level are obtained respectively; wherein the weights of the occurrence frequency and the confidence level are higher than the weight of the change frequency.
[0062] The calculation of the degree of change based on the occurrence frequency, the change frequency, and the confidence level includes:
[0063] The occurrence frequency score is calculated based on the occurrence frequency and the weight of the occurrence frequency;
[0064] The change frequency score is calculated based on the change frequency and its weight.
[0065] The confidence score is calculated based on the confidence level and its weight.
[0066] The sum of the occurrence frequency score, the change frequency score, and the execution score is determined as the degree of change.
[0067] Optionally, the calculation of the occurrence frequency score based on the occurrence frequency and the weight of the occurrence frequency includes:
[0068] The frequency of occurrence is obtained by quotienting the number of times the data corresponding to the difference dimension appears in the source information and the number of times the difference dimension appears in the source information.
[0069] The occurrence frequency score is obtained by multiplying the occurrence frequency by its weight.
[0070] Optionally, the step of calculating the change frequency score based on the change frequency and the weight of the change frequency includes:
[0071] The frequency of change is obtained by quotienting the time it takes for the data corresponding to the difference dimension to be updated in the source information and the time it takes for the difference dimension to appear in the source information.
[0072] The change frequency score is obtained by multiplying the change frequency by its weight.
[0073] Secondly, this application also provides a product information storage device, comprising:
[0074] The acquisition unit is used to acquire product information;
[0075] The verification unit is used to perform information quality verification on the product information. The information quality verification is used to determine whether the data of the product information in different dimensions conforms to the information quality rules. The information quality rules include rules that conform to Chinese specification requirements.
[0076] The matching unit is used to match the product information with standard information when the product information passes the information quality verification, and obtain a matching result. The standard information is product information pre-stored in the product information database according to the storage standard of the product information database.
[0077] A determining unit is used to determine the degree of product change based on the product information when the matching result is a complete match.
[0078] An adding unit is used to add the product information to the product information database based on the degree of change.
[0079] Optionally, the device further includes:
[0080] The setting unit is used to set the information quality rules based on the instruction information, wherein the information quality rules are used to limit the data filling specifications of the product information in each dimension.
[0081] Optionally, the device further includes:
[0082] The first operation unit is configured to send the product information to a first manual reviewer when it is determined that the product information has failed the information quality verification, and to perform a first target operation based on the feedback information of the first manual reviewer. The first target operation includes a first operation, a second operation, and a third operation. The first operation is used to modify the abnormal content in the product information, the second operation is used to modify the information quality rules, and the third operation is used to delete the product information.
[0083] The first execution unit is configured to perform the information quality verification again on the modified product information when the first target operation is the first operation.
[0084] The first execution unit is further configured to, when the first target operation is the second operation, re-execute the information quality verification on the product information using the modified information quality rules.
[0085] Optionally, the device further includes:
[0086] A cleaning unit is used to perform a cleaning operation on the product information so that the format of the cleaned product information conforms to the data format of the product information database.
[0087] The verification unit is also used to perform the information quality verification on the cleaned product information.
[0088] Optionally, the product information includes data corresponding to at least one dimension;
[0089] The matching unit is further configured to match the product information with the standard information according to the data pairs corresponding to each dimension, and obtain the matching result.
[0090] Optionally, the matching unit is further configured to match the product information with the standard information according to the data corresponding to each dimension, using regular expressions and similarity algorithms, to obtain a matching result.
[0091] Optionally, the matching unit includes:
[0092] The judgment module is used to determine, through regular expressions, whether the product information and the standard information are completely consistent in the data corresponding to the common dimension;
[0093] The first determining module is used to determine that the product information and the standard information are completely matched if the product information and the standard information are completely consistent in the data corresponding to the common dimension.
[0094] The calculation module is used to calculate the similarity between the product information and the standard information in the same dimension if the data corresponding to the product information and the standard information are not completely consistent, and to determine the matching result based on the similarity and a preset threshold.
[0095] Optionally, the common dimensions include primary dimensions and secondary dimensions, with the primary dimension being more important than the secondary dimension; the matching results also include complete mismatch and suspected match.
[0096] The computing module includes:
[0097] The first calculation submodule is used to calculate the similarity between the product information and the standard information in the other dimension of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension. When the similarity exceeds the preset threshold, the matching result is determined to be the suspected match.
[0098] The second calculation submodule is used to calculate the similarity between the product information and the standard information in the other dimension of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension. If the similarity does not exceed the preset threshold, the matching result is determined to be a complete mismatch.
[0099] Optionally, the preset threshold includes a first threshold and a second threshold, wherein the first threshold is greater than the second threshold;
[0100] The computing module includes:
[0101] The third calculation submodule is used to calculate the similarity between the product information and the standard information in the other dimension of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and it is also determined that the product information and the standard information have the same data in the secondary dimension. When the similarity exceeds the first threshold, the matching result is determined to be the complete match.
[0102] The fourth calculation submodule is used to calculate the similarity between the product information and the standard information in the other dimension of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and it is also determined that the product information and the standard information have the same data in the secondary dimension. When the similarity is between the first threshold and the second threshold, the matching result is determined to be the suspected match.
[0103] The fifth calculation submodule is used to calculate the similarity between the product information and the standard information in the other dimension of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and that the product information and the standard information also have the same data in the secondary dimension. When the similarity is lower than the second threshold, the matching result is determined to be a complete mismatch.
[0104] Optionally, the computing module includes:
[0105] The sixth calculation submodule is used to calculate the similarity between the product information and the standard information in all dimensions of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, and when the similarity exceeds the first threshold, determine that the matching result is the complete match.
[0106] The seventh calculation submodule is used to calculate the similarity between the product information and the standard information in all dimensions of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension. When the similarity is between the first threshold and the second threshold, the matching result is determined to be the suspected match.
[0107] The eighth calculation submodule is used to calculate the similarity between the product information and the standard information in all dimensions of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, and when the similarity is lower than the second threshold, the matching result is determined to be a complete mismatch.
[0108] Optionally, the matching unit further includes:
[0109] The second determining module is used to determine the matching result as a suspected match if, based on the regular expression, it is determined that the product information and the standard information are inconsistent only in the data corresponding to the secondary dimension.
[0110] Optionally, the device further includes:
[0111] The second operation unit is used to send the product information to a second human review when the matching result is determined to be a suspected match, and to perform a second target operation based on the feedback information of the second human review, wherein the second target operation includes the first operation and the third operation;
[0112] The second execution unit is used to perform the information quality verification on the product information modified based on the second target operation when the second target operation is the first operation.
[0113] The second execution unit is further configured to perform a matching operation between the product information modified based on the second target operation and the standard information when the information quality verification passes the information quality verification, and obtain the matching result. The matching operation is configured to match the data corresponding to each dimension according to regular expressions and similarity algorithms.
[0114] Optionally, the device further includes:
[0115] The third operation unit is used to send the product information to a third human review when the matching result is determined to be a complete mismatch, and to perform a third target operation based on the feedback information of the third human review. The third target operation includes the first operation, the second operation, the third operation, and the fourth operation, wherein the fourth operation is used to directly add the product information to the product information database.
[0116] The third execution unit is configured to, when the third target operation is determined to be the first operation, perform the information quality verification based on the product information modified by the third target operation, and when the information quality verification is passed, perform the matching operation based on the product information modified by the third target operation and the standard information to obtain the matching result;
[0117] The third execution unit is further configured to, when the third target operation is determined to be the second operation, perform the information quality verification on the product information again using the information quality rules modified based on the third target operation, and when the information quality verification is passed, perform the matching operation on the product information modified based on the third target operation and the standard information to obtain the matching result;
[0118] The third execution unit is further configured to directly add the product information to the product information database when the third target operation is determined to be the fourth operation.
[0119] Optionally, the primary dimension includes product identifier and manufacturer identifier, and the secondary dimension includes product version; wherein, the product identifier includes Chinese product identifier and / or foreign language product identifier, and the manufacturer identifier includes Chinese manufacturer identifier and / or foreign language manufacturer identifier.
[0120] Optionally, the determining unit includes:
[0121] The first acquisition module is used to acquire the dimensions that differ between the product information and the standard information, denoted as the difference dimension;
[0122] The determination module is used to acquire data corresponding to the difference dimensions, and based on the data corresponding to the difference dimensions, determine the frequency of occurrence, frequency of change, and confidence level of the data corresponding to the difference dimensions in the same source information; wherein, the same source information is information that represents the same product as the product information and the standard information; the confidence level is used to characterize the accuracy of the information determined based on the source of the product information;
[0123] The calculation module is used to calculate the degree of change based on the occurrence frequency, the change frequency, and the confidence level.
[0124] Optionally, the determining unit further includes:
[0125] The second acquisition module is used to acquire the weights of the occurrence frequency, the change frequency, and the confidence level, respectively; wherein the weights of the occurrence frequency and the confidence level are higher than the weight of the change frequency;
[0126] The computing module includes:
[0127] The first calculation submodule is used to calculate the occurrence frequency score based on the occurrence frequency and the weight of the occurrence frequency;
[0128] The second calculation submodule is used to calculate the change frequency score based on the change frequency and the weight of the change frequency;
[0129] The third calculation submodule is used to calculate the confidence score based on the confidence level and the weight of the confidence level;
[0130] The determination submodule is used to determine the degree of change by summing the occurrence frequency score, the change frequency score, and the execution score.
[0131] Optionally, the first calculation submodule is further configured to perform a quotient calculation on the number of times the data corresponding to the difference dimension appears in the source information and the number of times the difference dimension appears in the source information to obtain the occurrence frequency; and to multiply the occurrence frequency with the weight of the occurrence frequency to obtain the occurrence frequency score.
[0132] Optionally, the second calculation submodule is further configured to perform a quotient calculation on the time interval for updating the data corresponding to the difference dimension in the source information and the time interval for the difference dimension to appear in the source information to obtain the change frequency; and to multiply the change frequency with the weight of the change frequency to obtain the change frequency score.
[0133] Thirdly, embodiments of this application provide a storage medium including a stored program, wherein, when the program is executed, the device on which the storage medium is located executes the product information storage method of the terminal device described in the first aspect.
[0134] Fourthly, embodiments of this application provide a product information storage device, the device including a storage medium; and one or more processors, the storage medium being coupled to the processors, the processors being configured to execute program instructions stored in the storage medium; the program instructions, when executed, perform the product information storage method of the terminal device described in the first aspect.
[0135] By employing the above-described technical solution, the technical solution provided in this application has at least the following advantages:
[0136] This application provides a method and apparatus for storing product information. This application can acquire product information and perform information quality verification on the product information. When the product information passes the information quality verification, it is matched with standard information to obtain a matching result. If the matching result is a complete match, the degree of product change is determined based on the product information, and the product information is added to the product information database based on the degree of change, thereby realizing the product information storage function. Compared with the prior art, since the information quality verification is used to determine whether the product information data in different dimensions conforms to information quality rules, and the information quality rules include rules that conform to Chinese standard requirements, this ensures that the product information stored in the product information database is adapted to the domestic Chinese product information. This solves the problem that existing product information databases, which rely on foreign technologies for storage, have poor adaptability and affect the accuracy of the storage results. Simultaneously, information quality verification is performed during storage, based on information quality rules. These rules ensure the "quality" of product information across different dimensions, guaranteeing verification when the product information itself contains errors or omissions. This avoids storing product information with errors or missing data in certain dimensions into the product information database, further improving the accuracy of information during storage. Furthermore, the standard information refers to product information pre-stored in the product information database according to its storage standards. The storage process involves not only storing the product information itself but also storing it based on its degree of change. This not only avoids wasting storage space by storing identical product information but also ensures that information is stored according to changes, guaranteeing the effectiveness of each storage action and avoiding invalid storage.
[0137] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description
[0138] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, with the same or corresponding reference numerals denoteing the same or corresponding parts, wherein:
[0139] Figure 1 A flowchart illustrating a product information storage method according to an embodiment of this application is shown.
[0140] Figure 2-A A flowchart of another product information storage method provided in an embodiment of this application is shown;
[0141] Figure 2-B A flowchart of another product information storage method provided in an embodiment of this application is shown;
[0142] Figure 2-C A flowchart of another product information storage method provided in an embodiment of this application is shown;
[0143] Figure 3 This illustration shows a block diagram of a product information storage device according to an embodiment of this application;
[0144] Figure 4 A block diagram of another product information storage device provided in an embodiment of this application is shown. Detailed Implementation
[0145] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.
[0146] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.
[0147] This application provides a method for storing product information, which is applied to a client-side application, specifically as follows: Figure 1 As shown, the method includes:
[0148] 101. Obtain product information.
[0149] In this embodiment, the product information can be understood as descriptive information about a certain product. This product can be a physical product or a virtual product, such as software or design information. Here, there is no specific limitation on the product corresponding to this product information; it can be selected according to the user's actual needs.
[0150] Furthermore, in this embodiment, the process of obtaining product information can be performed through a pre-set interactive interface. That is, after the interactive interface is displayed to the user, the user can input product information based on this interface. Alternatively, it can be obtained directly from the user's database through a pre-set interactive interface.
[0151] In some cases, the data format in a user's database may not be consistent with the format required by the storage process described in this embodiment. Therefore, after obtaining the original data, a cleaning operation can be performed on the original data to obtain the cleaned data as the product information. This cleaning operation can be understood as a data processing method under a code value conversion rule. Since different databases have different formats when storing data, this cleaning operation ensures that the original data is converted into data recognizable in the subsequent product information database of this embodiment after cleaning. Furthermore, this cleaning operation can also convert certain information and data in Chinese form into numerical form. Since much of the original data is recorded in text form, such as the Chinese characters for "eleven centimeters," this format differs from data like "11cm," which is represented by numbers and letters, in subsequent data storage. The former consumes more storage device performance. Therefore, the cleaning operation in this embodiment ensures that the cleaned product information saves more storage device or database performance compared to the original data.
[0152] 102. Perform information quality verification on the product information.
[0153] The information quality verification is used to determine whether the product information data in different dimensions conforms to the information quality rules, which include rules that conform to Chinese standard requirements.
[0154] In actual use, product information may contain missing data, incorrect data content, formatting errors, or missing dimensions. This may be due to data loss during the cleaning process from the original data, or the aforementioned problems may have existed in the original data. Therefore, this embodiment also requires an analysis of the overall "quality" of the product information, i.e., information quality verification. This verification process is based on information quality rules, which are user-preset rules requiring product information to conform to Chinese language standards. This can be understood as the various dimensions (fields) involved in the product information needing to comply with Chinese language standards. Furthermore, this information quality verification also needs to ensure that the data in different dimensions of the product information also conforms to these information quality rules. Table 1 below serves as an example:
[0155] Information serial number name Manufacturers Version Record date Update? source 1 Storm Translation Han and Tang Dynasties 1.0 2022.7.6 yes Manual input 2 Thunder Translation Edo Technology Thunder Null no Manual input 3 Kuangfeng Translation 84.75.2 1.2 2021.9.37 yes Automatic input
[0156] As shown in Table 1, for product information 1 (information sequence number 1), its name is "Baofeng Translate," indicating that this product information corresponds to a software called "Baofeng Translate," the manufacturer is "Han Tang Times" company, the version is "1.0," the record date is "2022.7.6," indicating that product information 1 was obtained on July 6, 2022. "Updated" indicates that this product information has been updated, and "Source" indicates that the product information was obtained manually through an interactive interface similar to the steps described above. Therefore, the information quality rule described in this embodiment can be understood as limiting the conformity of the names of each dimension of the above product information—information sequence number, name, manufacturer, version, record date, whether updated, and source—to Chinese language standards. Simultaneously, this information quality rule is also used to determine whether the data content of each product information under different dimensions conforms to the format of that dimension and whether the content under that dimension is missing. Therefore, based on the method of this embodiment and referring to Table 1, for product information 2 (information number 2), the data corresponding to its "Version" dimension is "Jing Lei," which clearly does not meet the requirements for data content under the "Version" dimension. Simultaneously, the "Record Date" dimension is "Null," indicating missing data in this dimension. Therefore, it can be determined that product information 2 does not meet the information quality rules and will not pass the information quality verification. Similarly, for product information 3 (information number 3), the data corresponding to its "Manufacturer" dimension is "84.75.2," which is clearly not the data content that should be filled in for this dimension. Furthermore, the data corresponding to the "Record Date" dimension is "2021.9.37," and September certainly does not have a 37th day; therefore, this data is also incorrect. Thus, product information 3 also does not meet the information quality rules and will not pass the information quality verification. Conversely, for product information 1, since the data corresponding to each dimension meets the data requirements for that dimension, that is, meets the information quality rules, it can pass the information quality verification.
[0157] 103. When the product information passes the information quality verification, the product information is matched with the standard information to obtain the matching result.
[0158] The standard information refers to product information pre-stored in the product information database according to the storage standards of the product information database.
[0159] In this embodiment, since the information quality verification process in the product information has already determined that the "quality" of the product information meets the data storage requirements, the next step is to match the product information with standard information. This standard information can be understood as information already stored in the product information database, and it is product information that meets the storage requirements. Therefore, it can be used as a standard for matching, that is, to determine whether the product corresponding to the current product information is the same as the product determined by the standard information.
[0160] Specifically, during the matching process, a similarity algorithm can be used to calculate whether the product information and the standard information are identical across each dimension. Of course, to determine whether the product information and the standard information correspond to the same product, only dimensions that clearly indicate what product the information corresponds to can be selected, such as the product name in the previous example. In practical applications, this includes, but is not limited to, the methods described above. Users can also use other algorithms and select their desired dimensions for matching; this is not limited to these methods.
[0161] 104. When the matching result is a perfect match, the degree of product change is determined based on the product information.
[0162] When the matching result is determined to be a perfect match, it means that the product corresponding to the product information and the product corresponding to the standard information are the same product. This also means that the product information is likely to be information about a product that has undergone some changes based on the standard information, such as updated information. Therefore, in the process of storing this product information, it is actually necessary to determine how much the current product has changed from the standard information, that is, the degree of change.
[0163] Of course, determining the extent of product changes can be done by analyzing the differences between product information and standard information. This involves identifying the dimensions where the product information and the representative information differ, and analyzing the significance of those dimensions. It should be noted that other methods can also be used to determine the extent of product information changes; this is not limited to these methods and can be based on the user's actual needs.
[0164] 105. Add the product information to the product information database based on the degree of change.
[0165] Once the degree of change is determined, the product information can be added to the product information database based on the information change procedure. For example, when the degree of change is severe, it means that a product has undergone significant changes based on the standard information, i.e., the product information. In this case, the product information is likely to better represent the current status of the product, so it is necessary to add the product information to the product information database.
[0166] This application provides a method for storing product information. Compared with existing technologies, the information quality verification determines whether the product information conforms to information quality rules in different dimensions. These information quality rules include those conforming to Chinese language standards. This ensures that the product information stored in the product information database is compatible with domestic Chinese product information, solving the problem of poor compatibility and inaccuracy in existing product information databases that rely on foreign technologies. Furthermore, the storage process is based on information quality verification, which is performed according to information quality rules. Since these rules ensure the "quality" of product information in different dimensions, they also ensure the function of verifying the product information when it contains errors or omissions. This avoids storing product information with errors or missing data in certain dimensions in the product information database, further improving the accuracy of the stored information. In addition, the standard information refers to product information pre-stored in the product information database according to the storage standards of the product information database. Moreover, the process of storing product information is not just about storing the product information itself, but also about storing it based on the degree of change of the product information. This not only avoids the problem of wasting storage space by storing the same product information in the product information database, but also ensures that the product information is stored according to the changes, thus ensuring the effectiveness of each storage behavior and avoiding invalid storage behavior.
[0167] To provide a more detailed explanation, this application provides another method for storing product information, as detailed below. Figure 2-A As shown, the method includes:
[0168] 201. Obtain product information.
[0169] In this embodiment, the process of obtaining product information can be performed through a pre-set interactive interface. That is, after the interactive interface is displayed to the user, the user can input product information based on this interface. Alternatively, it can be obtained directly from the user's database through a pre-set interactive interface. This automates the process of obtaining product information. The user can set a retrieval cycle, and the generated data will be retrieved directly from the user's database at regular intervals. In this embodiment, the product information can be understood as descriptive information about a certain product. This product can be a physical product or a virtual product, such as software or design data. Here, there is no specific limitation on the product corresponding to this product information; it can be set according to the user's actual needs.
[0170] 202. Set the information quality rules based on the instruction information.
[0171] Among them, the information quality rule is used to define the filling specifications of the data of the product information in each dimension.
[0172] In this embodiment, during the process of storing product information in the product information library, it is also necessary to verify the "quality" of the product information, that is, information quality verification. To ensure the accuracy of the verification results, it is necessary for the user to set the verification basis in this process, that is, the information quality rule. Of course, during the setting process, interactive information can be sent to the user in the form of a template, and then the user can set the requirements for verification in each dimension like filling in a blank and send the corresponding instruction information, so as to determine the information quality rule. In addition, the user can preset multiple different information quality rules, so that when the user needs to verify different types of product information or product information of different projects, the appropriate information quality rule can be selected, thereby improving the effect of subsequent information quality verification.
[0173] 203. Perform a cleaning operation on the product information so that the format of the cleaned product information conforms to the data format of the product information library.
[0174] Since product information itself is a kind of data, the formats and character forms when different databases store data may be different. To ensure the accuracy of subsequent storage of product information, it is also necessary to perform a cleaning operation on the product information here. This cleaning operation can be understood as converting the code values of the product information in a manner that conforms to the subsequent product information library. In this way, the accuracy of the subsequent storage result is ensured. At the same time, since some product information itself contains a large amount of Chinese character content, and some of this Chinese character content can actually be recorded based on numbers or letters, and Chinese character content often consumes more processor resources in the computer system, the cleaning operation in this step can also convert such Chinese character content into corresponding numbers or letters at the same time. For example, "3:00 p.m., July 6, 2022" can be converted into "2022.07.06-15:00" through the cleaning operation.
[0175] 2:04. Perform information quality verification on the product information.
[0176] Among them, the information quality verification is used to determine whether the data of the product information in different dimensions conforms to the information quality rule, and the information quality rule includes rules that meet the requirements of Chinese norms.
[0177] Based on the method of the foregoing steps, this step can specifically be: performing the information quality verification on the cleaned product information.
[0178] The process of performing information quality verification on the product information is essentially based on whether the data content of each dimension in the product information is formatted correctly, whether any data is missing from each dimension, whether any important dimensions are missing, and whether the form of the data in each dimension meets the requirements of that dimension, etc. Examples include spelling errors, invalid values, integrity constraints (data corresponding to core dimensions cannot be empty), redundant underscores, full-width / half-width characters being mixed up, invisible characters before or after data, and newlines before or after strings / data. Of course, the information quality verification process is based on information quality rules; that is, the information quality rules set during the establishment of the rules determine the requirements for each dimension of the product information, and the verification process determines whether the product information can pass the verification according to these requirements.
[0179] Based on the information quality verification results, there are two situations. One is that the "quality" of the product information is good and there are no missing or incorrect issues as mentioned above. In this case, the product information can pass the information quality verification and step 208 can be executed directly. When it is determined that the product information has failed the information quality verification, there may be a problem with the product information or a problem with the information quality rules, etc. In this case, step 205 can be executed.
[0180] 205. When it is determined that the product information has failed the information quality verification, the product information is sent to the first manual reviewer, and the first target operation is performed based on the feedback information from the first manual reviewer.
[0181] The first target operation includes a first operation, a second operation, and a third operation; the first operation is used to modify the abnormal content in the product information, the second operation is used to modify the information quality rules, and the third operation is used to delete the product information.
[0182] When product information fails the information quality check, there are three possibilities: first, the product information does indeed have a problem; second, the product information may be fine, but the information quality rules might be flawed; and third, the product information may be missing core data in a very important dimension. Therefore, to differentiate between these three scenarios, this embodiment sends the product information to a human reviewer—the first human review. During this process, after the review is completed, the reviewer will provide corresponding feedback, which may contain different instructions. Upon receiving this feedback, different initial target operations need to be executed.
[0183] 206. When the first target operation is the first operation, perform the information quality verification again on the modified product information.
[0184] When the first target operation is performed, it indicates that there is indeed a problem with the product information, and the product information has already been modified manually. Therefore, it is necessary to re-perform information quality verification on the modified product information during this process.
[0185] 207. When the first target operation is the second operation, the information quality verification is performed again on the product information using the modified information quality rules.
[0186] When the first target operation becomes the second operation, it means that the first manual reviewer believes there may be a problem with the information quality rules. In this case, the modified information quality rules can be used to re-perform information quality verification on the product information.
[0187] In addition, when the first target operation is the third operation, it means that the product information is missing core dimension data content. Therefore, it is meaningless to store the product information in the product information database. The product information has already been discarded and deleted after the first manual review, so there is no need to perform any operation on the product information.
[0188] 208. When the product information passes the information quality verification, the product information is matched with the standard information to obtain the matching result.
[0189] The standard information refers to product information pre-stored in the product information database according to the storage standards of the product information database.
[0190] In some embodiments, the product information includes data corresponding to at least one dimension; therefore, in the matching process, it is necessary to match according to different dimensions separately to determine the matching result.
[0191] Specifically, this step can be performed as follows: matching the product information with the standard information according to the data pairs corresponding to each dimension to obtain the matching results.
[0192] Furthermore, during the matching process, specific matching can be performed based on regular expressions and similarity algorithms. Therefore, in this step, the product information and the standard information are matched according to the data pairs corresponding to each dimension to obtain the matching results, including:
[0193] The product information and the standard information are matched according to the data corresponding to each dimension using regular expressions and similarity algorithms to obtain the matching results.
[0194] Specifically, in the process of matching product information and standard information using regular expressions and similarity algorithms, regular expressions can be used for matching first, followed by similarity algorithms. Based on this, the product information and standard information are matched according to the data corresponding to each dimension using regular expressions and similarity algorithms to obtain the matching results. The execution of this process may include:
[0195] Regular expressions are used to determine whether the product information and the standard information are completely consistent in the shared dimensions. Based on this determination, there are two cases, as follows:
[0196] Aspect 1: If they are completely consistent, then the product information is determined to be a complete match with the standard information.
[0197] 2. If they are not completely identical, a similarity algorithm is used to calculate the similarity between the product information and the standard information in the same dimension, and the matching result is determined based on the similarity and a preset threshold.
[0198] Because regular expressions are a relatively strict matching method, the process of matching product information and standard information according to regular expressions can be divided into two cases: exact match and incomplete match. The former means that the data in the common dimensions of the product information and standard information are the same. The latter means that the product information and standard information have different data in several of the common dimensions.
[0199] For aspect 1, this means that the product information and standard information share common dimensions, i.e., the data under each of these shared dimensions is identical. Therefore, both belong to the same product, and they are a perfect match. Conversely, for aspect 2, this means that among the multiple dimensions shared by the product information and standard information, some dimensions have different data. In this case, a similarity algorithm is needed to calculate the similarity between the data under these different dimensions, and then determine the matching result based on this similarity. In this embodiment, the similarity algorithm can utilize the Jaccard similarity algorithm. The Jaccard similarity algorithm, also known as the Jaccard index or Jaccard similarity coefficient algorithm, is mainly used to compare the similarity and differences between finite sample sets. The larger the Jaccard similarity algorithm result, the higher the similarity.
[0200] In practical applications, the shared dimensions can be divided into two categories: primary dimensions and secondary dimensions. As shown in the text, primary dimensions are more important than secondary dimensions. Furthermore, during the execution of aspect 2 mentioned above, the matching results can further include complete mismatches and potential matches.
[0201] Based on this, the method described in aspect 2 above can also be specifically implemented as follows:
[0202] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and when the similarity exceeds the preset threshold, the matching result is determined to be the suspected match.
[0203] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and if the similarity does not exceed the preset threshold, then the matching result is determined to be a complete mismatch.
[0204] In this embodiment, the primary dimension can be a dimension that indicates what product the product information specifically refers to, such as the product name. This type of dimension can intuitively reflect the product corresponding to the product information and can be used as the primary dimension for analysis during matching in this step. Therefore, when it is determined that one of the important dimensions among the common dimensions in the product information and standard information is consistent according to a regular expression, it means that the two pieces of information may correspond to the same product. Then, the similarity algorithm can be directly used to calculate whether the similarity between the data of the other dimension in the primary dimension exceeds a preset threshold. If it exceeds the threshold, it means that the two are still highly similar, and the two pieces of information (product information and standard information) are actually information about the same product, but there is a small difference in the data of a certain primary dimension. For example, one of the product name dimensions in the two pieces of information is the full name, and the other is the abbreviation. At this time, the matching result between the two is determined to be a suspected match. On the other hand, when the data similarity of the other dimension is lower than the preset threshold, it means that the two are not information about the same product. Therefore, the matching result between the two is determined to be a complete mismatch. For example, the data of the product under the manufacturer dimension in the primary dimension is the same, but the data under the product name in the primary dimension is different. At this time, the two are actually two different products from the same manufacturer, so the two are a complete mismatch.
[0205] Furthermore, in practical applications, the preset threshold includes a first threshold and a second threshold, wherein the first threshold is greater than the second threshold;
[0206] Based on this, the method described in aspect 2 above can also be specifically implemented as follows:
[0207] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and it is also determined that the product information and the standard information have the same data in the secondary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and when the similarity exceeds the first threshold, the matching result is determined to be a complete match.
[0208] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and it is also determined that the product information and the standard information have the same data in the secondary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and when the similarity is between the first threshold and the second threshold, the matching result is determined to be the suspected match.
[0209] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and that the product information and the standard information also have the same data in the secondary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated. If the similarity is lower than the second threshold, the matching result is determined to be a complete mismatch.
[0210] In the three scenarios described above, the product information and standard information share similar data across some dimensions of the primary dimension and also share similar data across secondary dimensions. Therefore, they may be completely matched, possibly matched, or completely mismatched. In this process, it's necessary to analyze the similarity between the data in the inconsistent primary dimension. Specifically, we need to determine whether the similarity is higher than the first threshold, between the first and second thresholds, or lower than the second threshold. This ensures that, given the similarity across most dimensions, we can determine whether the product information and standard information correspond to the same product based on the similarity of data in a specific primary dimension.
[0211] Furthermore, the method described in aspect 2 above may further include, during execution:
[0212] If, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, then the similarity between the product information and the standard information in all dimensions corresponding to the primary dimension is calculated, and when the similarity exceeds the first threshold, the matching result is determined to be a complete match.
[0213] If, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, then the similarity between the product information and the standard information in all dimensions corresponding to the primary dimension is calculated, and when the similarity is between the first threshold and the second threshold, the matching result is determined to be the suspected match.
[0214] If, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, then the similarity between the product information and the standard information in all dimensions corresponding to the primary dimension is calculated, and when the similarity is lower than the second threshold, the matching result is determined to be a complete mismatch.
[0215] In practical applications, as in the three situations mentioned above, there may be instances where product information and standard information are completely identical only in secondary dimensions. In such cases, it is necessary to perform similarity calculations on all dimensions in the primary dimension and compare the calculated similarity with the first and second thresholds to achieve analysis between product information and standard information.
[0216] In practical applications, after determining whether the product information and the standard information are completely consistent in the common dimension using regular expressions, another scenario may also be included:
[0217] Aspect 3: If, based on the regular expression, it is determined that the product information and the standard information are inconsistent only in the data corresponding to the secondary dimension, then the matching result is determined to be a suspected match.
[0218] Since the primary dimension reflects the characteristics of the product or indicates the main dimension of which product it is, if it is determined that the product information and the standard information are inconsistent only in the secondary dimension, then it is very likely that the two are data updates of the same product in a certain dimension. However, it cannot be completely determined that the two correspond to the same product, and further analysis is required. Therefore, the matching result can be determined as a suspected match.
[0219] It should be noted that the primary dimension may include product identifier and manufacturer identifier, and the secondary dimension may include product version; wherein, the product identifier may include the product's Chinese identifier and / or the product's foreign language identifier, and the manufacturer identifier may include the manufacturer's Chinese identifier and / or the manufacturer's foreign language identifier.
[0220] Based on the judgment result, if the matching result is determined to be a complete match, proceed to steps 209A-210A; if the matching result is determined to be a suspected match, proceed to steps 209B to 211B; if the matching result is determined to be a complete mismatch, proceed to step...
[0221] 209A. When the matching result is a perfect match, the degree of product change is determined based on the product information.
[0222] Specifically, this step can be performed based on the following steps:
[0223] Step 1: Obtain the dimensions that differ between the product information and the standard information, and denote them as the difference dimensions;
[0224] Step 2: Obtain the data corresponding to the difference dimensions, and determine the frequency of occurrence, frequency of change, and confidence level of the data corresponding to the difference dimensions in the same source information based on the data corresponding to the difference dimensions; wherein, the same source information is information that represents the same product as the product information and the standard information; the confidence level is used to characterize the accuracy of the information determined based on the source of the product information;
[0225] Step 3: Calculate the degree of change based on the occurrence frequency, the change frequency, and the confidence level.
[0226] In this embodiment, the degree of change can be performed according to steps 1 to 3 above. This allows for quantitative calculation based on the frequency of occurrence, frequency of change, and confidence level of data in the dimensions of the product information, ensuring a more accurate degree of change obtained subsequently.
[0227] Furthermore, since the occurrence frequency, change frequency, and confidence level have different effects on the degree of change in practical applications, the occurrence frequency, change frequency, and confidence level weight can be determined before step 3 is executed.
[0228] Based on this, before executing step 3, the weights of the occurrence frequency, the change frequency, and the confidence level can be obtained respectively; wherein, the weights of the occurrence frequency and the confidence level are higher than the weight of the change frequency.
[0229] Based on this, step 3 can be executed as follows:
[0230] The occurrence frequency score is calculated based on the occurrence frequency and the weight of the occurrence frequency;
[0231] The change frequency score is calculated based on the change frequency and its weight.
[0232] The confidence score is calculated based on the confidence level and its weight.
[0233] The sum of the occurrence frequency score, the change frequency score, and the execution score is determined as the degree of change.
[0234] Because the weights of occurrence frequency, change frequency, and confidence level are determined in the process of calculating the degree of change, the different impacts of the three factors on the degree of change are taken into account, thus ensuring the accuracy of the determination of the degree of change.
[0235] Based on this, the aforementioned step of calculating the frequency of occurrence score based on the frequency of occurrence and the weight of the frequency of occurrence includes:
[0236] The frequency of occurrence is obtained by quotienting the number of times the data corresponding to the difference dimension appears in the source information and the number of times the difference dimension appears in the source information.
[0237] The occurrence frequency score is obtained by multiplying the occurrence frequency by its weight.
[0238] Since the frequency of occurrence is mainly determined by the number of times the product information data under the difference dimension appears in the entire source information and the number of times the difference dimension appears in the source information, it can reflect the occurrence of the difference dimension data in the product information among multiple information of the same product. It can objectively reflect the occurrence of the data, thereby ensuring that the frequency of occurrence is more scientific and accurate.
[0239] Meanwhile, in the aforementioned steps, calculating the change frequency score based on the change frequency and its weight includes:
[0240] The frequency of change is obtained by quotienting the time it takes for the data corresponding to the difference dimension to be updated in the source information and the time it takes for the difference dimension to appear in the source information.
[0241] The change frequency score is obtained by multiplying the change frequency by its weight.
[0242] In this embodiment, the change frequency can be understood as how long the product information in the difference dimension has been updated in the same source information, and how long this difference dimension has appeared in the same source information. Calculating the quotient between the two can reflect the proportion of the data appearing in the overall difference dimension, thus reflecting the update frequency of the data. This can objectively reflect the update status of the data, thereby ensuring that the obtained update frequency is more scientific and accurate.
[0243] 210A. Add the product information to the product information database based on the degree of change.
[0244] Once the extent of the change is determined, the specific differences between the product information and the source information are identified. If the differences are significant, it indicates that the product information is an updated version of the source information and therefore needs to be stored.
[0245] Furthermore, when the matching result is determined to be a suspected match, the following steps can be performed, such as... Figure 2-B As shown, it includes:
[0246] 209B. When the matching result is determined to be a suspected match, the product information is sent to a second human reviewer, and a second target operation is performed based on the feedback information from the second human reviewer.
[0247] The second target operation includes the first operation and the third operation.
[0248] When a match result is determined to be a "suspected match," it actually indicates that some information in the product information is likely problematic, causing discrepancies between the data and the standard information in certain dimensions. Therefore, the product information needs to undergo a second manual review. This second manual review falls into two categories: one where the product information itself does indeed have data issues, in which case it will be manually modified; and another where the product information not only has data issues but also lacks core data, in which case the product information will be discarded and deleted, and no further action needs to be taken on it.
[0249] 210B. When the second target operation is the first operation, perform the information quality verification on the product information modified based on the second target operation.
[0250] Once the second target operation is determined to be the first operation, which means that the product information has been manually modified, then the modified product information can be subjected to information quality verification to determine whether the modified product information can pass the verification.
[0251] 211B. When the product information modified based on the second target operation passes the information quality verification, a matching operation is performed between the product information modified based on the second target operation and the standard information to obtain the matching result.
[0252] The matching operation is used to match data according to each dimension using regular expressions and similarity algorithms.
[0253] Once it is confirmed that the modified product information passes the information quality verification, it means that the product information is of "quality" qualified. Then it can be matched with the standard information again, that is, a matching operation is performed. This matching operation is the same as the matching process in the previous steps, and will not be repeated here.
[0254] In some embodiments, when the matching result is determined to be a complete mismatch, the following steps can be performed, such as... Figure 2-C As shown, it includes:
[0255] 209C. When the matching result is determined to be a complete mismatch, the product information is sent to a third human reviewer, and a third target operation is performed based on the feedback information from the third human reviewer.
[0256] The third target operation includes the first operation, the second operation, the third operation, and the fourth operation, wherein the fourth operation is used to directly add the product information to the product information database.
[0257] When a complete mismatch is determined, it indicates a potential data issue with the product information itself. In this case, a third-party manual review will trigger the first step. It could also be due to problems with the quality information rules, such as incorrect dimensional specifications, leading to a complete mismatch conclusion based on dimensions. Alternatively, the product information might be missing core data, which would correspond to the third step. Furthermore, the product information could be from a completely different product than the previously identified standard information, making a mismatch highly likely.
[0258] 210C. When the third target operation is determined to be the first operation, the information quality verification is performed on the product information modified based on the third target operation. When the information quality verification is passed, the matching operation is performed on the product information modified based on the third target operation and the standard information to obtain the matching result.
[0259] Based on the results of the aforementioned analysis, when the third target operation is determined to be the first operation, it indicates that there is a problem with the product information and it needs to be modified. Therefore, the product information after manual modification needs to be re-verified for information quality. When the information quality verification is passed, the product information is re-matched with the standard information.
[0260] 211C. When the third target operation is determined to be the second operation, the information quality verification is performed again on the product information using the information quality rules modified based on the third target operation. When the information quality verification is passed, the matching operation is performed between the product information modified based on the third target operation and the standard information to obtain the matching result.
[0261] When the third target operation is determined to be the second operation, it indicates a problem with the quality information rules, resulting in a complete mismatch. Therefore, the modified information quality rules can be used to validate the product information. Once the product information passes the quality information validation, a matching operation is performed again with the characterization information.
[0262] 212C. When the third target operation is determined to be the fourth operation, the product information is directly added to the product information database.
[0263] When the third target operation is determined to be the fourth operation, it means that the product information is information about a product that is different from the standard information, that is, new information, and then the information can be directly added to the product information database.
[0264] Furthermore, as a response to the above Figure 1 , Figure 2-A , Figure 2-B as well as Figure 2-C In addition to the implementation of the method shown, another embodiment of this application also provides a product information storage device. This product information storage device embodiment corresponds to the foregoing method embodiment. For ease of reading, this product information storage device embodiment will not repeat the details of the foregoing method embodiment, but it should be understood that the device in this embodiment can correspondingly implement all the contents of the foregoing method embodiment. Specifically, as follows... Figure 3 As shown, the storage device for the product information includes:
[0265] The acquisition unit 301 can be used to acquire product information;
[0266] The verification unit 302 can be used to perform information quality verification on the product information. The information quality verification can be used to determine whether the data of the product information in different dimensions conforms to the information quality rules. The information quality rules include rules that conform to Chinese specification requirements.
[0267] The matching unit 303 can be used to match the product information with standard information when the product information passes the information quality verification, and obtain a matching result. The standard information is product information pre-stored in the product information database according to the storage standard of the product information database.
[0268] The determining unit 304 can be used to determine the degree of product change based on the product information when the matching result is a complete match.
[0269] The adding unit 305 can be used to add the product information to the product information database based on the degree of change.
[0270] Furthermore, such as Figure 4As shown, the device further includes:
[0271] Setting unit 306 can be used to set the information quality rules based on instruction information, wherein the information quality rules can be used to limit the data filling specifications of the product information in each dimension.
[0272] Furthermore, such as Figure 4 As shown, the device further includes:
[0273] The first operation unit 307 can be used to send the product information to a first manual review when it is determined that the product information has failed the information quality verification, and to perform a first target operation based on the feedback information of the first manual review. The first target operation includes a first operation, a second operation, and a third operation. The first operation can be used to modify the abnormal content in the product information, the second operation can be used to modify the information quality rules, and the third operation can be used to delete the product information.
[0274] The first execution unit 308 can be used to perform the information quality verification again on the modified product information when the first target operation is the first operation.
[0275] The first execution unit 308 can also be used to perform the information quality verification on the product information again using the modified information quality rules when the first target operation is the second operation.
[0276] Furthermore, such as Figure 4 As shown, the device further includes:
[0277] The cleaning unit 309 can be used to perform a cleaning operation on the product information so that the format of the cleaned product information conforms to the data format of the product information database.
[0278] The verification unit 302 can also be used to perform the information quality verification on the cleaned product information.
[0279] Furthermore, such as Figure 4 As shown, the product information includes data corresponding to at least one dimension;
[0280] The matching unit 303 can also be used to match the product information with the standard information according to the data pairs corresponding to each dimension to obtain the matching result.
[0281] Furthermore, such as Figure 4 As shown, the matching unit 303 can also be used to match the product information with the standard information according to the data corresponding to each dimension, using regular expressions and similarity algorithms, to obtain the matching result.
[0282] Furthermore, such as Figure 4 As shown, the matching unit 303 includes:
[0283] The judgment module 3031 can be used to determine, through regular expressions, whether the product information and the standard information are completely consistent in the data corresponding to the common dimension;
[0284] The first determining module 3032 can be used to determine that the product information and the standard information are completely matched if the product information and the standard information are completely consistent in the data corresponding to the common dimension.
[0285] The calculation module 3033 can be used to calculate the similarity between the product information and the standard information in the same dimension if the data corresponding to the product information and the standard information are not completely consistent, and to determine the matching result based on the similarity and a preset threshold.
[0286] Furthermore, such as Figure 4 As shown, the common dimensions include primary dimensions and secondary dimensions, with the primary dimension being more important than the secondary dimension; the matching results also include complete mismatch and suspected match.
[0287] The computing module 3033 includes:
[0288] The first calculation submodule 30331 can be used to calculate the similarity between the product information and the standard information in the other dimension of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information are consistent in only one dimension of the primary dimension; and when the similarity exceeds the preset threshold, the matching result is determined to be the suspected match.
[0289] The second calculation submodule 30332 can be used to calculate the similarity between the product information and the standard information in the other dimension of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension. If the similarity does not exceed the preset threshold, the matching result is determined to be a complete mismatch.
[0290] Furthermore, such as Figure 4 As shown, the preset threshold includes a first threshold and a second threshold, wherein the first threshold is greater than the second threshold;
[0291] The computing module 3033 includes:
[0292] The third calculation submodule 30333 can be used to calculate the similarity between the product information and the standard information in the other dimension of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and it is also determined that the product information and the standard information have the same data in the secondary dimension. When the similarity exceeds the first threshold, the matching result is determined to be the complete match.
[0293] The fourth calculation submodule 30334 can be used to calculate the similarity between the product information and the standard information in the other dimension of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and it is also determined that the product information and the standard information have the same data in the secondary dimension. When the similarity is between the first threshold and the second threshold, the matching result is determined to be the suspected match.
[0294] The fifth calculation submodule 30335 can be used to calculate the similarity between the product information and the standard information in the other dimension of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and that the product information and the standard information also have the same data in the secondary dimension. When the similarity is lower than the second threshold, the matching result is determined to be a complete mismatch.
[0295] Furthermore, such as Figure 4 As shown, the computing module 3033 includes:
[0296] The sixth calculation submodule 30336 can be used to calculate the similarity between the product information and the standard information in all dimensions of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, and when the similarity exceeds the first threshold, determine that the matching result is the complete match.
[0297] The seventh calculation submodule 30337 can be used to calculate the similarity between the product information and the standard information in all dimensions of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, and when the similarity is between the first threshold and the second threshold, determine the matching result as the suspected match.
[0298] The eighth calculation submodule 30338 can be used to calculate the similarity between the product information and the standard information in all dimensions of the primary dimension if, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, and when the similarity is lower than the second threshold, determine that the matching result is a complete mismatch.
[0299] Furthermore, such as Figure 4 As shown, the matching unit 303 further includes:
[0300] The second determining module 3034 can be used to determine the matching result as a suspected match if, based on the regular expression, it is determined that the product information and the standard information are inconsistent only in the data corresponding to the secondary dimension.
[0301] Furthermore, such as Figure 4 As shown, the device further includes:
[0302] The second operation unit 310 can be used to send the product information to a second manual review when the matching result is determined to be a suspected match, and to perform a second target operation based on the feedback information of the second manual review, wherein the second target operation includes the first operation and the third operation;
[0303] The second execution unit 311 can be used to perform the information quality verification on the product information modified based on the second target operation when the second target operation is the first operation.
[0304] The second execution unit 311 can also be used to perform a matching operation between the product information modified based on the second target operation and the standard information when the information quality verification passes the information quality verification, and obtain the matching result. The matching operation can be used to match the data corresponding to each dimension according to regular expressions and similarity algorithms.
[0305] Furthermore, such as Figure 4 As shown, the device further includes:
[0306] The third operation unit 312 can be used to send the product information to a third human review when the matching result is determined to be a complete mismatch, and to perform a third target operation based on the feedback information of the third human review. The third target operation includes the first operation, the second operation, the third operation and the fourth operation. The fourth operation can be used to directly add the product information to the product information database.
[0307] The third execution unit 313 can be used to perform the information quality verification based on the product information modified by the third target operation when the third target operation is determined to be the first operation, and when the information quality verification is passed, to perform the matching operation based on the product information modified by the third target operation and the standard information to obtain the matching result.
[0308] The third execution unit 313 can also be used to, when the third target operation is determined to be the second operation, perform the information quality verification on the product information again using the information quality rules modified based on the third target operation, and when the information quality verification is passed, perform the matching operation on the product information modified based on the third target operation and the standard information to obtain the matching result;
[0309] The third execution unit 313 can also be used to directly add the product information to the product information database when the third target operation is determined to be the fourth operation.
[0310] Furthermore, such as Figure 4 As shown, the primary dimension includes product identifier and manufacturer identifier, and the secondary dimension includes product version; wherein, the product identifier includes Chinese product identifier and / or foreign language product identifier, and the manufacturer identifier includes Chinese manufacturer identifier and / or foreign language manufacturer identifier.
[0311] Furthermore, such as Figure 4 As shown, the determining unit 304 includes:
[0312] The first acquisition module 3041 can be used to acquire the dimension where there is a difference between the product information and the standard information, denoted as the difference dimension;
[0313] The determination module 3042 can be used to acquire data corresponding to the difference dimension, and determine the frequency of occurrence, change frequency, and confidence level of the data corresponding to the difference dimension in the same source information based on the data corresponding to the difference dimension; wherein, the same source information is information that represents the same product as the product information and the standard information; the confidence level can be used to characterize the accuracy of the information determined based on the source of the product information;
[0314] The calculation module 3043 can be used to calculate the degree of change based on the occurrence frequency, the change frequency, and the confidence level.
[0315] Furthermore, such as Figure 4 As shown, the determining unit 304 further includes:
[0316] The second acquisition module 3044 can be used to acquire the weights of the occurrence frequency, the change frequency, and the confidence level respectively; wherein the weights of the occurrence frequency and the confidence level are higher than the weight of the change frequency;
[0317] The computing module 3043 includes:
[0318] The first calculation submodule 30431 can be used to calculate the occurrence frequency score based on the occurrence frequency and the weight of the occurrence frequency;
[0319] The second calculation submodule 30432 can be used to calculate the change frequency score based on the change frequency and the weight of the change frequency;
[0320] The third calculation submodule 30433 can be used to calculate the confidence score based on the confidence level and the weight of the confidence level;
[0321] The determination submodule 30434 can be used to determine the degree of change by summing the occurrence frequency score, the change frequency score, and the execution score.
[0322] Furthermore, such as Figure 4 As shown, the first calculation submodule 30431 can also be used to calculate the frequency of occurrence by quotienting the number of times the data corresponding to the difference dimension appears in the source information and the number of times the difference dimension appears in the source information; and to calculate the frequency of occurrence by multiplying the frequency of occurrence with the weight of the frequency of occurrence.
[0323] Furthermore, such as Figure 4 As shown, the second calculation submodule 30432 can also be used to calculate the change frequency by quotienting the time for which the data corresponding to the difference dimension is updated in the source information and the time for which the difference dimension appears in the source information; and to calculate the change frequency by multiplying the change frequency with the weight of the change frequency.
[0324] This application provides a method and apparatus for storing product information. This method and apparatus can acquire product information and perform information quality verification on the product information. When the product information passes the information quality verification, it is matched with standard information to obtain a matching result. If the matching result is a complete match, the degree of product change is determined based on the product information, and the product information is added to the product information database based on the degree of change, thereby realizing the product information storage function. Compared with the prior art, since the information quality verification is used to determine whether the product information data in different dimensions conforms to information quality rules, and the information quality rules include rules that conform to Chinese standard requirements, this ensures that the product information is adapted to domestic Chinese product information during the storage process in the product information database. This solves the problem that existing product information databases, which rely on foreign technologies for storage, have poor adaptability and affect the accuracy of the storage results. Simultaneously, information quality verification is performed during storage, based on information quality rules. These rules ensure the "quality" of product information across different dimensions, guaranteeing verification when the product information itself contains errors or omissions. This avoids storing product information with errors or missing data in certain dimensions into the product information database, further improving the accuracy of information during storage. Furthermore, the standard information refers to product information pre-stored in the product information database according to its storage standards. The storage process involves not only storing the product information itself but also storing it based on its degree of change. This not only avoids wasting storage space by storing identical product information but also ensures that information is stored according to changes, guaranteeing the effectiveness of each storage action and avoiding invalid storage.
[0325] This application provides a storage medium that includes a stored program, wherein the program, when running, controls the device where the storage medium is located to execute the product information storage method described above.
[0326] Storage media may include non-permanent memory in the form of computer-readable media, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.
[0327] This application embodiment also provides a product information storage device, the device including a storage medium; and one or more processors, the storage medium being coupled to the processors, the processors being configured to execute program instructions stored in the storage medium; the program instructions, when executed, perform the product information storage method described above.
[0328] This application provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: acquiring product information; performing information quality verification on the product information, wherein the information quality verification determines whether the data of the product information in different dimensions conforms to information quality rules, the information quality rules including rules conforming to Chinese standard requirements; when the product information passes the information quality verification, matching the product information with standard information to obtain a matching result, wherein the standard information is product information pre-stored in a product information database according to the storage standards of the product information database; when the matching result is a complete match, determining the degree of change of the product based on the product information; and adding the product information to the product information database based on the degree of change.
[0329] Furthermore, before performing information quality verification on the product information, the method further includes:
[0330] The information quality rules are set based on the instruction information, wherein the information quality rules are used to limit the data filling specifications of the product information in each dimension.
[0331] Furthermore, after performing information quality verification on the product information, the method further includes:
[0332] When it is determined that the product information fails the information quality verification, the product information is sent to the first manual reviewer, and a first target operation is performed based on the feedback information from the first manual reviewer. The first target operation includes a first operation, a second operation, and a third operation. The first operation is used to modify the abnormal content in the product information, the second operation is used to modify the information quality rules, and the third operation is used to delete the product information.
[0333] When the first target operation is the first operation, the information quality verification is performed again on the modified product information;
[0334] When the first target operation is the second operation, the information quality verification is performed again on the product information using the modified information quality rules.
[0335] Furthermore, before performing information quality verification on the product information, the method further includes:
[0336] The product information is cleaned so that the format of the cleaned product information conforms to the data format of the product information database.
[0337] The process of performing information quality verification on the product information includes:
[0338] Perform the information quality verification on the cleaned product information.
[0339] Furthermore, the product information includes data corresponding to at least one dimension;
[0340] The step of matching the product information with the standard information to obtain the matching result includes:
[0341] The product information is matched with the standard information according to the data pairs corresponding to each dimension to obtain the matching result.
[0342] Furthermore, the step of matching the product information with the standard information according to the data pairs corresponding to each dimension to obtain the matching result includes:
[0343] The product information and the standard information are matched according to the data corresponding to each dimension using regular expressions and similarity algorithms to obtain the matching results.
[0344] Furthermore, the step of matching the product information with the standard information according to the data corresponding to each dimension, using regular expressions and similarity algorithms, to obtain the matching result includes:
[0345] Regular expressions are used to determine whether the product information and the standard information are completely consistent in the data corresponding to the common dimension.
[0346] If they are completely consistent, then the product information is determined to be a complete match with the standard information.
[0347] If they are not completely identical, a similarity algorithm is used to calculate the similarity between the product information and the standard information in the same dimension, and the matching result is determined based on the similarity and a preset threshold.
[0348] Furthermore, the common dimensions include primary dimensions and secondary dimensions, with the primary dimension being more important than the secondary dimension; the matching results also include complete mismatch and suspected match.
[0349] The step of calculating the similarity between the product information and the standard information in the same dimension using a similarity algorithm, and determining the matching result based on the similarity and a preset threshold, includes:
[0350] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and when the similarity exceeds the preset threshold, the matching result is determined to be the suspected match.
[0351] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and if the similarity does not exceed the preset threshold, then the matching result is determined to be a complete mismatch.
[0352] Furthermore, the preset threshold includes a first threshold and a second threshold, wherein the first threshold is greater than the second threshold;
[0353] The step of calculating the similarity between the product information and the standard information in the same dimension using a similarity algorithm, and determining the matching result based on the similarity and a preset threshold, includes:
[0354] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and it is also determined that the product information and the standard information have the same data in the secondary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and when the similarity exceeds the first threshold, the matching result is determined to be a complete match.
[0355] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and it is also determined that the product information and the standard information have the same data in the secondary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and when the similarity is between the first threshold and the second threshold, the matching result is determined to be the suspected match.
[0356] If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and that the product information and the standard information also have the same data in the secondary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated. If the similarity is lower than the second threshold, the matching result is determined to be a complete mismatch.
[0357] Furthermore, the step of calculating the similarity between the product information and the standard information in the same dimension using a similarity algorithm, and determining the matching result based on the similarity and a preset threshold, includes:
[0358] If, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, then the similarity between the product information and the standard information in all dimensions corresponding to the primary dimension is calculated, and when the similarity exceeds the first threshold, the matching result is determined to be a complete match.
[0359] If, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, then the similarity between the product information and the standard information in all dimensions corresponding to the primary dimension is calculated, and when the similarity is between the first threshold and the second threshold, the matching result is determined to be the suspected match.
[0360] If, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, then the similarity between the product information and the standard information in all dimensions corresponding to the primary dimension is calculated, and when the similarity is lower than the second threshold, the matching result is determined to be a complete mismatch.
[0361] Furthermore, after determining whether the product information and the standard information are completely consistent in the common dimension using regular expressions, the method further includes:
[0362] If, based on the regular expression, it is determined that the product information and the standard information are inconsistent only in the data corresponding to the secondary dimension, then the matching result is determined to be a suspected match.
[0363] Furthermore, after matching the product information with standard information to obtain a matching result when the product information passes the information quality verification, the method further includes:
[0364] When the matching result is determined to be a suspected match, the product information is sent to a second human reviewer, and a second target operation is performed based on the feedback information from the second human reviewer. The second target operation includes the first operation and the third operation.
[0365] When the second target operation is the first operation, the information quality verification is performed on the product information modified based on the second target operation.
[0366] When the product information modified based on the second target operation passes the information quality verification, a matching operation is performed between the product information modified based on the second target operation and the standard information to obtain the matching result. The matching operation is used to match the data corresponding to each dimension according to regular expressions and similarity algorithms.
[0367] Furthermore, after matching the product information with standard information to obtain a matching result when the product information passes the information quality verification, the method further includes:
[0368] When the matching result is determined to be a complete mismatch, the product information is sent to a third human reviewer, and a third target operation is performed based on the feedback information from the third human reviewer. The third target operation includes the first operation, the second operation, the third operation, and a fourth operation, wherein the fourth operation is used to directly add the product information to the product information database.
[0369] When the third target operation is determined to be the first operation, the information quality verification will be performed based on the product information modified by the third target operation. When the information quality verification is passed, the matching operation will be performed based on the product information modified by the third target operation and the standard information to obtain the matching result.
[0370] When the third target operation is determined to be the second operation, the information quality verification is performed again on the product information using the information quality rules modified based on the third target operation. When the information quality verification is passed, the matching operation is performed between the product information modified based on the third target operation and the standard information to obtain the matching result.
[0371] When the third target operation is determined to be the fourth operation, the product information is directly added to the product information database.
[0372] Furthermore, the primary dimension includes product identifier and manufacturer identifier, and the secondary dimension includes product version; wherein, the product identifier includes Chinese product identifier and / or foreign language product identifier, and the manufacturer identifier includes Chinese manufacturer identifier and / or foreign language manufacturer identifier.
[0373] Furthermore, when the matching result is a perfect match, determining the degree of product change based on the product information includes:
[0374] The dimensions in which the product information differs from the standard information are identified and denoted as the difference dimensions.
[0375] Obtain the data corresponding to the difference dimension, and determine the frequency of occurrence, change frequency, and confidence level of the data corresponding to the difference dimension in the same source information based on the data corresponding to the difference dimension; wherein, the same source information is information that represents the same product as the product information and the standard information; the confidence level is used to characterize the accuracy of the information determined based on the source of the product information;
[0376] The degree of change is calculated based on the occurrence frequency, the change frequency, and the confidence level.
[0377] Furthermore, before calculating the degree of change based on the occurrence frequency, the change frequency, and the confidence level, the method further includes:
[0378] The weights of the occurrence frequency, the change frequency, and the confidence level are obtained respectively; wherein the weights of the occurrence frequency and the confidence level are higher than the weight of the change frequency.
[0379] The calculation of the degree of change based on the occurrence frequency, the change frequency, and the confidence level includes:
[0380] The occurrence frequency score is calculated based on the occurrence frequency and the weight of the occurrence frequency;
[0381] The change frequency score is calculated based on the change frequency and its weight.
[0382] The confidence score is calculated based on the confidence level and its weight.
[0383] The sum of the occurrence frequency score, the change frequency score, and the execution score is determined as the degree of change.
[0384] Furthermore, the calculation of the frequency of occurrence score based on the frequency of occurrence and the weight of the frequency of occurrence includes:
[0385] The frequency of occurrence is obtained by quotienting the number of times the data corresponding to the difference dimension appears in the source information and the number of times the difference dimension appears in the source information.
[0386] The occurrence frequency score is obtained by multiplying the occurrence frequency by its weight.
[0387] Furthermore, the calculation of the change frequency score based on the change frequency and the weight of the change frequency includes:
[0388] The frequency of change is obtained by quotienting the time it takes for the data corresponding to the difference dimension to be updated in the source information and the time it takes for the difference dimension to appear in the source information.
[0389] The change frequency score is obtained by multiplying the change frequency by its weight.
[0390] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing program code with the following initialization steps: acquiring product information; performing information quality verification on the product information, wherein the information quality verification is used to determine whether the data of the product information in different dimensions conforms to information quality rules, the information quality rules including rules conforming to Chinese standard requirements; when the product information passes the information quality verification, matching the product information with standard information to obtain a matching result, wherein the standard information is product information pre-stored in a product information database according to the storage standards of the product information database; when the matching result is a complete match, determining the degree of change of the product based on the product information; and adding the product information to the product information database based on the degree of change.
[0391] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0392] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0393] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0394] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0395] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0396] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0397] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0398] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0399] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0400] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A method for storing product information, characterized in that, include: Obtain product information; The product information is subjected to information quality verification, which is used to determine whether the data of the product information in different dimensions conforms to information quality rules, and the information quality rules include rules that conform to Chinese specification requirements; When the product information passes the information quality verification, the product information is matched with standard information to obtain a matching result. The standard information is product information pre-stored in the product information database according to the storage standard of the product information database. When the matching result is a complete match, the degree of product change is determined based on the product information. The product information is added to the product information database based on the degree of change. When the matching result is a perfect match, determining the degree of product change based on the product information includes: The dimensions in which the product information differs from the standard information are identified and denoted as the difference dimensions. Obtain the data corresponding to the difference dimension, and determine the frequency of occurrence, change frequency, and confidence level of the data corresponding to the difference dimension in the same source information based on the data corresponding to the difference dimension; wherein, the same source information is information that represents the same product as the product information and the standard information; the confidence level is used to characterize the accuracy of the information determined based on the source of the product information; The degree of change is calculated based on the occurrence frequency, the change frequency, and the confidence level.
2. The method according to claim 1, characterized in that, Before performing information quality verification on the product information, the method further includes: The information quality rules are set based on the instruction information, wherein the information quality rules are used to limit the data filling specifications of the product information in each dimension.
3. The method according to claim 2, characterized in that, After performing information quality verification on the product information, the method further includes: When it is determined that the product information fails the information quality verification, the product information is sent to the first manual reviewer, and a first target operation is performed based on the feedback information from the first manual reviewer. The first target operation includes a first operation, a second operation, and a third operation. The first operation is used to modify the abnormal content in the product information, the second operation is used to modify the information quality rules, and the third operation is used to delete the product information. When the first target operation is the first operation, the information quality verification is performed again on the modified product information; When the first target operation is the second operation, the information quality verification is performed again on the product information using the modified information quality rules.
4. The method according to claim 3, characterized in that, Before performing information quality verification on the product information, the method further includes: The product information is cleaned so that the format of the cleaned product information conforms to the data format of the product information database. The process of performing information quality verification on the product information includes: Perform the information quality verification on the cleaned product information.
5. The method according to claim 1, characterized in that, The product information includes data corresponding to at least one dimension; The step of matching the product information with the standard information to obtain the matching result includes: The product information is matched with the standard information according to the data pairs corresponding to each dimension to obtain the matching result.
6. The method according to claim 5, characterized in that, The step of matching the product information with the standard information according to the data pairs corresponding to each dimension to obtain the matching result includes: The product information and the standard information are matched according to the data corresponding to each dimension using regular expressions and similarity algorithms to obtain the matching results.
7. The method according to claim 6, characterized in that, The step of matching the product information with the standard information according to the data corresponding to each dimension, using regular expressions and similarity algorithms, to obtain the matching results includes: Regular expressions are used to determine whether the product information and the standard information are completely consistent in the data corresponding to the common dimension. If they are completely identical, then the product information is determined to be a complete match with the standard information; If they are not completely identical, a similarity algorithm is used to calculate the similarity between the product information and the standard information in the same dimension, and the matching result is determined based on the similarity and a preset threshold.
8. The method according to claim 7, characterized in that, The common dimensions include primary dimensions and secondary dimensions, with the primary dimensions being more important than the secondary dimensions; the matching results also include complete mismatches and suspected matches; The step of calculating the similarity between the product information and the standard information in the same dimension using a similarity algorithm, and determining the matching result based on the similarity and a preset threshold, includes: If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and when the similarity exceeds the preset threshold, the matching result is determined to be the suspected match. If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and if the similarity does not exceed the preset threshold, then the matching result is determined to be a complete mismatch.
9. The method according to claim 8, characterized in that, The preset threshold includes a first threshold and a second threshold, wherein the first threshold is greater than the second threshold; The step of calculating the similarity between the product information and the standard information in the same dimension using a similarity algorithm, and determining the matching result based on the similarity and a preset threshold, includes: If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and it is also determined that the product information and the standard information have the same data in the secondary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and when the similarity exceeds the first threshold, the matching result is determined to be a complete match. If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and it is also determined that the product information and the standard information have the same data in the secondary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated, and when the similarity is between the first threshold and the second threshold, the matching result is determined to be the suspected match. If, based on the regular expression, it is determined that the product information and the standard information have the same data in only one dimension of the primary dimension, and that the product information and the standard information also have the same data in the secondary dimension, then the similarity between the product information and the standard information in the other dimension of the primary dimension is calculated. If the similarity is lower than the second threshold, the matching result is determined to be a complete mismatch.
10. The method according to claim 9, characterized in that, The step of calculating the similarity between the product information and the standard information in the same dimension using a similarity algorithm, and determining the matching result based on the similarity and a preset threshold, includes: If, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, then the similarity between the product information and the standard information in all dimensions corresponding to the primary dimension is calculated, and when the similarity exceeds the first threshold, the matching result is determined to be a complete match. If, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, then the similarity between the product information and the standard information in all dimensions corresponding to the primary dimension is calculated, and when the similarity is between the first threshold and the second threshold, the matching result is determined to be the suspected match. If, based on the regular expression, it is determined that the product information and the standard information are consistent only in the data corresponding to the secondary dimension, then the similarity between the product information and the standard information in all dimensions corresponding to the primary dimension is calculated, and when the similarity is lower than the second threshold, the matching result is determined to be a complete mismatch.
11. The method according to claim 8, characterized in that, After determining whether the product information and the standard information are completely consistent in the common dimension using regular expressions, the method further includes: If, based on the regular expression, it is determined that the product information and the standard information are inconsistent only in the data corresponding to the secondary dimension, then the matching result is determined to be a suspected match.
12. The method according to claim 11, characterized in that, After the product information passes the information quality verification and is matched with standard information to obtain a matching result, the method further includes: When the matching result is determined to be a suspected match, the product information is sent to a second human reviewer, and a second target operation is performed based on the feedback information from the second human reviewer. The second target operation includes a first operation and a third operation. When the second target operation is the first operation, the information quality verification is performed on the product information modified based on the second target operation. When the product information modified based on the second target operation passes the information quality verification, a matching operation is performed between the product information modified based on the second target operation and the standard information to obtain the matching result. The matching operation is used to match the data corresponding to each dimension according to regular expressions and similarity algorithms.
13. The method according to claim 12, characterized in that, After the product information passes the information quality verification and is matched with standard information to obtain a matching result, the method further includes: When the matching result is determined to be a complete mismatch, the product information is sent to a third human reviewer, and a third target operation is performed based on the feedback information from the third human reviewer. The third target operation includes the first operation, the second operation, the third operation, and the fourth operation, wherein the fourth operation is used to directly add the product information to the product information database. When the third target operation is determined to be the first operation, the information quality verification will be performed based on the product information modified by the third target operation. When the information quality verification is passed, the matching operation will be performed based on the product information modified by the third target operation and the standard information to obtain the matching result. When the third target operation is determined to be the second operation, the information quality verification is performed again on the product information using the information quality rules modified based on the third target operation. When the information quality verification is passed, the matching operation is performed between the product information modified based on the third target operation and the standard information to obtain the matching result. When the third target operation is determined to be the fourth operation, the product information is directly added to the product information database.
14. The method according to any one of claims 8-13, characterized in that, The primary dimension includes product identification and manufacturer identification, and the secondary dimension includes product version; wherein, the product identification includes Chinese product identification and / or foreign language product identification, and the manufacturer identification includes Chinese manufacturer identification and / or foreign language manufacturer identification.
15. The method according to claim 1, characterized in that, Before calculating the degree of change based on the occurrence frequency, the change frequency, and the confidence level, the method further includes: The weights of the occurrence frequency, the change frequency, and the confidence level are obtained respectively; wherein the weights of the occurrence frequency and the confidence level are higher than the weight of the change frequency. The calculation of the degree of change based on the occurrence frequency, the change frequency, and the confidence level includes: The occurrence frequency score is calculated based on the occurrence frequency and the weight of the occurrence frequency; The change frequency score is calculated based on the change frequency and its weight. The confidence score is calculated based on the confidence level and its weight. The sum of the occurrence frequency score, the change frequency score, and the confidence score is determined as the degree of change.
16. The method according to claim 15, characterized in that, The calculation of the frequency of occurrence score based on the frequency of occurrence and the weight of the frequency of occurrence includes: The frequency of occurrence is obtained by quotienting the number of times the data corresponding to the difference dimension appears in the source information and the number of times the difference dimension appears in the source information. The occurrence frequency score is obtained by multiplying the occurrence frequency by its weight.
17. The method according to claim 15, characterized in that, The calculation of the change frequency score based on the change frequency and its weight includes: The frequency of change is obtained by quotienting the time it takes for the data corresponding to the difference dimension to be updated in the source information and the time it takes for the difference dimension to appear in the source information. The change frequency score is obtained by multiplying the change frequency by its weight.
18. A product information storage device, characterized in that, include: The acquisition unit is used to acquire product information; The verification unit is used to perform information quality verification on the product information. The information quality verification is used to determine whether the data of the product information in different dimensions conforms to the information quality rules. The information quality rules include rules that conform to Chinese specification requirements. The matching unit is used to match the product information with standard information when the product information passes the information quality verification, and obtain a matching result. The standard information is product information pre-stored in the product information database according to the storage standard of the product information database. A determining unit is used to determine the degree of product change based on the product information when the matching result is a complete match. An adding unit is used to add the product information to the product information database based on the degree of change. The determining unit includes: The first acquisition module is used to acquire the dimensions that differ between the product information and the standard information, denoted as the difference dimension; The determination module is used to acquire data corresponding to the difference dimensions, and based on the data corresponding to the difference dimensions, determine the frequency of occurrence, frequency of change, and confidence level of the data corresponding to the difference dimensions in the same source information; wherein, the same source information is information that represents the same product as the product information and the standard information; the confidence level is used to characterize the accuracy of the information determined based on the source of the product information; The calculation module is used to calculate the degree of change based on the occurrence frequency, the change frequency, and the confidence level.
19. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, it controls the device where the storage medium is located to perform the method for storing product information as described in any one of claims 1-17.
20. A product information storage device, characterized in that, The apparatus includes a storage medium; and one or more processors, the storage medium being coupled to the processors, the processors being configured to execute program instructions stored in the storage medium; the program instructions, when executed, perform the method for storing product information according to any one of claims 1-17.