A method, device, equipment and storage medium for extracting common attributes of commodities

Abstract

The present invention discloses a method, device, equipment and storage medium for extracting common attributes of commodities. The method comprises: obtaining SKU text reflecting commodity information, performing data cleaning on the SKU text, removing specific information reflecting specific attributes of commodities in the SKU text, obtaining target text, bucketing multiple target texts based on the text length of the target text, putting multiple target texts with text lengths within a preset range into the same text bucket, calculating the similarity between target texts in each text bucket for each text bucket, and taking multiple target texts with similarities greater than a similarity threshold as SPU text reflecting common attributes of commodities, without the need for data labeling and model training, requiring fewer computing resources, reducing costs, and facilitating deployment to edge devices with low computing power.

Description

Technical Field

[0001] The present invention relates to commodity information processing technology, and in particular to a commodity universal attribute extraction method, device, equipment and storage medium. Background Art

[0002] SPU (Standard Product Unit) refers to a group of commodities with common attributes, representing the general specifications of commodities without considering the differences in details such as color and size. SPU is a way to classify and manage commodities, which reflects the basic type and core characteristics of commodities. SKU (Stock Keeping Unit) is a unique identifier assigned to each commodity in the inventory, usually composed of a combination of letters and numbers, providing specific information about the product. For example, for a T-shirt, different colors and sizes constitute different SKUs, but they share the same design, material and other attributes, and these shared attributes define the SPU of the T-shirt. Using SPU for analysis is of great significance for optimizing product management, accurate inventory adjustment, market and customer behavior analysis, and formulating competitive strategies.

[0003] Currently, most methods for extracting SPU universal attributes are to extract SPU universal attributes from SKU names through technologies such as machine learning and natural language processing (NLP). Existing methods require a large amount of labeled data, strong computing resources, and a lot of time to train the model, resulting in high costs for existing methods. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for extracting common attributes of commodities, so as to reduce the extraction cost.

[0005] In a first aspect, the present invention provides a method for extracting common attributes of commodities, comprising:

[0006] Get the SKU text that reflects the product information;

[0007] Performing data cleaning on the SKU text to remove specific information reflecting specific attributes of the product in the SKU text to obtain a target text;

[0008] Dividing the target texts into buckets based on the text lengths of the target texts, and placing the target texts whose text lengths are within a preset range into the same text bucket;

[0009] For each of the text buckets, calculating the similarity between the target texts in the text bucket;

[0010] The plurality of target texts whose similarity is greater than a similarity threshold are used as SPU texts reflecting the common attributes of commodities.

[0011] Optionally, data cleaning is performed on the SKU text to remove specific information reflecting specific attributes of the product in the SKU text to obtain a target text, including:

[0012] Convert uppercase letters in the SKU text to lowercase letters;

[0013] Regular matching is used to match the converted SKU text to obtain specific information that reflects the specific attributes of the product.

[0014] Delete specific information reflecting specific attributes of the product;

[0015] The brand name of the product is added to the text after the specific information is deleted to obtain the target text.

[0016] Optionally, bucketing the target texts based on the text lengths of the target texts, and placing the target texts whose text lengths are within a preset range into the same text bucket, includes:

[0017] Count the number of occurrences of each text length and obtain an array sequence of text length-number of occurrences;

[0018] Fitting the array sequence to obtain a relationship curve between the number of occurrences of the text length and the text length;

[0019] Calculate the text length corresponding to the maximum value of the relationship curve;

[0020] Based on the text length corresponding to the maximum value, the plurality of target texts whose text lengths are within a preset range are put into the same text bucket.

[0021] Optionally, placing a plurality of target texts having text lengths within a preset range into the same text bucket based on the text lengths corresponding to the maximum value includes:

[0022] Calculate the average text length corresponding to two adjacent maximum values;

[0023] The text length corresponding to the average value is used as a segmentation point, and multiple target texts between two adjacent segmentation points are put into the same text bucket.

[0024] Optionally, before calculating the similarity between the target texts in each text bucket, the method further includes:

[0025] Deduplication is performed on the plurality of target texts to delete duplicate target texts;

[0026] Segment the target text and determine the word vector corresponding to each word;

[0027] A weight value is assigned to the word based on the modulus of the word vector.

[0028] Optionally, assigning a weight value to the word based on the modulus of the word vector includes:

[0029] Calculating the modulus of the word vector;

[0030] Determine whether the word is a number;

[0031] If yes, then the product of the modulus of the word vector and the weight adjustment coefficient is calculated as the weight value of the word, and the weight adjustment coefficient is greater than 1;

[0032] If not, the modulus of the word vector is used as the weight value of the word.

[0033] Optionally, for each of the text buckets, calculating the similarity between the target texts in the text bucket includes:

[0034] For each of the text buckets, multiply each word vector in each of the target texts in the text bucket by a corresponding weight value to obtain a target vector;

[0035] The similarity between the target vectors is calculated.

[0036] In a second aspect, the present invention further provides a device for extracting common attributes of commodities, comprising:

[0037] SKU text acquisition module, used to obtain SKU text reflecting product information;

[0038] A data cleaning module is used to clean the SKU text, remove specific information reflecting specific attributes of the product in the SKU text, and obtain a target text;

[0039] A bucketing module, used for bucketing the plurality of target texts based on the text length of the target texts, and putting the plurality of target texts whose text lengths are within a preset range into the same text bucket;

[0040] A similarity calculation module, used for calculating the similarity between the target texts in each text bucket for each text bucket;

[0041] The SPU text determination module is used to take the plurality of target texts whose similarity is greater than a similarity threshold as SPU texts reflecting the common attributes of commodities.

[0042] In a third aspect, the present invention further provides an electronic device, comprising:

[0043] one or more processors;

[0044] A storage device for storing one or more programs;

[0045] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for extracting common attributes of commodities as provided in the first aspect of the present invention.

[0046] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for extracting common attributes of commodities as provided in the first aspect of the present invention.

[0047] The method for extracting common attributes of commodities provided by the present invention comprises: obtaining SKU text reflecting commodity information, performing data cleaning on the SKU text, removing specific information reflecting specific attributes of commodities in the SKU text, obtaining target text, bucketing multiple target texts based on the text length of the target text, putting multiple target texts with text lengths within a preset range into the same text bucket, calculating the similarity between the target texts in the text bucket for each text bucket, taking multiple target texts with similarities greater than a similarity threshold as SPU text reflecting common attributes of commodities, without the need for data labeling and model training, requiring fewer computing resources, reducing costs, and facilitating deployment to edge devices with low computing power.

[0048] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 A flowchart of a method for extracting common attributes of a commodity provided by an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of a relationship curve provided by an embodiment of the present invention;

[0052] Figure 3 A schematic diagram of the structure of a device for extracting common attributes of commodities provided by an embodiment of the present invention;

[0053] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0054] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0056] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0057] Figure 1 This is a flow chart of a method for extracting common attributes of a product provided by an embodiment of the present invention. This embodiment is applicable to the case of extracting common attribute information of a product from the SKU description text of the product. The method can be executed by a common attribute extraction device for a product provided by an embodiment of the present invention. The device can be implemented by software and / or hardware and is usually configured in an electronic device, such as Figure 1 As shown, the method for extracting common attributes of commodities includes the following steps:

[0058] S101. Obtain SKU text reflecting product information.

[0059] In the embodiment of the present invention, the SKU texts of a large number of commodities are obtained, wherein the SKU text of each commodity is a text describing the commodity information of the commodity. The commodity information may include specific information of the commodity, such as color, specification, version, efficacy, etc., and also include common attributes of similar commodities, such as material, shape, etc.

[0060] S102: Perform data cleaning on the SKU text to remove specific information in the SKU text that reflects specific attributes of the product, and obtain the target text.

[0061] In the embodiment of the present invention, the SKU text is cleaned to remove specific information reflecting the specific attributes of the product, such as color, specification, special symbols and other redundant information, to obtain a target text with a normalized text style. The target text retains the general attribute information of the product.

[0062] In some embodiments of the present invention, regular matching can be used to match specific information from the SKU text and delete it. Exemplarily, in order to better perform regular matching, the uppercase letters in the SKU text are first converted to lowercase letters, and then a pre-built regular expression is used to perform information regular matching on the converted SKU text to match specific information reflecting the specific attributes of the product, and then the specific information reflecting the specific attributes of the product is deleted.

[0063] In reality, different brands often have similar products, and these products may share common attributes. In order to facilitate subsequent distinction, in some embodiments of the present invention, during or after data cleaning of the SKU text to remove specific information in the SKU text that reflects the specific attributes of the product, the brand name of the product can also be added to the text after the specific information is deleted to obtain the target text.

[0064] S103 , bucketing the multiple target texts based on the text lengths of the target texts, and putting the multiple target texts whose text lengths are within a preset range into the same text bucket.

[0065] In some embodiments of the present invention, the text length of each target text is counted, and multiple target texts are bucketed based on the text length of the target text, and multiple target texts with text lengths within a preset range are placed in the same text bucket. Exemplarily, the text length can be represented by the number of characters in the target text.

[0066] In some embodiments of the present invention, the above step S103 includes the following sub-steps:

[0067] S1031. Count the number of occurrences of each text length to obtain an array sequence of text length-number of occurrences.

[0068] In an embodiment of the present invention, the number of occurrences of each text length is counted to obtain an array consisting of text length-number of occurrences, and arrays corresponding to multiple text lengths form a discrete array sequence.

[0069] S1032. Fit the array sequence to obtain a curve showing the relationship between the number of occurrences of the text length and the text length.

[0070] The discrete array sequence is fitted to obtain a curve of the relationship between the number of occurrences of the text length and the text length. Exemplarily, in some embodiments of the present invention, cubic spline interpolation can be used for fitting in sections to obtain a curve of the relationship between the number of occurrences of the text length and the text length.

[0071] S1033. Calculate the text length corresponding to the maximum value of the relationship curve.

[0072] After obtaining the relationship curve of the number of occurrences of the text length with respect to the text length, the relationship curve is differentiated to obtain the maximum value of the relationship curve, and the text length corresponding to the maximum value is determined.

[0073] Figure 2 A schematic diagram of a relationship curve provided by an embodiment of the present invention, such as Figure 2 As shown, the horizontal axis is the text length (Length) (Times), the vertical axis is the number of occurrences (Times), and the maximum points in the relationship curve are points a, b, and c, corresponding to text lengths of 5, 15, and 19, respectively.

[0074] S1034. Put multiple target texts whose text lengths are within a preset range into the same text bucket based on the text lengths corresponding to the maximum value.

[0075] In an embodiment of the present invention, multiple target texts with text lengths within a preset range are placed in the same text bucket based on the text lengths corresponding to the maximum values. Exemplarily, the horizontal axis interval of the relationship curve is divided into multiple sub-intervals based on the text lengths corresponding to the maximum values, and then the target texts corresponding to the text lengths in the same sub-interval are placed in the same text bucket.

[0076] For example, in some embodiments of the present invention, the average value of the text lengths corresponding to two adjacent maxima is calculated, the text length corresponding to the average value is used as the segmentation point, and multiple target texts between the two adjacent segmentation points are placed in the same text bucket. Figure 2 As shown, the average text length corresponding to the maximum points a and b is 10, and the average text length corresponding to the maximum points b and c is 17. Therefore, with 10 and 17 as the dividing points, the entire relationship curve is divided into three sub-intervals: 2-10, 11-17, and 18-23. Then, the target texts in the sub-interval of 2-10 are put into the same text bucket, the target texts in the sub-interval of 11-17 are put into the same text bucket, and the target texts in the sub-interval of 18-23 are put into the same text bucket. In this way, multiple target texts are bucketed.

[0077] S104: For each text bucket, calculate the similarity between target texts in the text bucket.

[0078] In the embodiment of the present invention, for each text bucket, the similarity between each target text in the text bucket is calculated. Exemplarily, the target text in the text bucket can be segmented in advance, and each word is mapped to a corresponding word vector, so as to vectorize the target text. Then, the cosine similarity of the vectors corresponding to any two target texts is calculated.

[0079] In some embodiments of the present invention, in order to improve the accuracy of similarity calculation and thus improve the accuracy of extracting common attributes, a corresponding weight value may be assigned to each word in advance according to the text features of the words in the target text.

[0080] Exemplarily, multiple target texts may be deduplicated in advance, and repeated target texts may be deleted, that is, the same common attributes may be deleted. The target texts may be segmented, and the word vectors corresponding to each word may be determined, and then weight values ​​may be assigned to the words based on the modulus of the word vectors.

[0081] Exemplarily, in some embodiments of the present invention, the modulus of the word vector is calculated, and it is determined whether the word is a number. If so, the product of the modulus of the word vector and the weight adjustment coefficient is calculated as the weight value of the word. The weight adjustment coefficient is greater than 1. In commodities, the year and version number usually have special meanings. The weight value of such words is increased here; if not, the modulus of the word vector is used as the weight value of the word. Specifically, the calculation formula of the word weight value is as follows:

[0082]

[0083] Among them, weight is the weight value, is the modulus of the word vector, and k is the weight adjustment coefficient.

[0084] When calculating the similarity between target texts in a text bucket, for each text bucket, multiply each word vector in each target text in the text bucket by the corresponding weight value to obtain a target vector, and then calculate the similarity between any two target vectors.

[0085] S105: Use multiple target texts whose similarities are greater than a similarity threshold as SPU texts reflecting common attributes of the products.

[0086] In the embodiment of the present invention, a similarity threshold is preset, and a plurality of target texts with similarities greater than the similarity threshold are used as SPU texts reflecting the common attributes of commodities, thereby extracting the common attributes of a category of commodities.

[0087] The method for extracting common attributes of goods provided by an embodiment of the present invention includes: obtaining SKU text reflecting product information, performing data cleaning on the SKU text, removing specific information reflecting specific attributes of the product in the SKU text, obtaining a target text, bucketing multiple target texts based on the text length of the target text, putting multiple target texts with text lengths within a preset range into the same text bucket, calculating the similarity between the target texts in the text bucket for each text bucket, and using multiple target texts with similarities greater than a similarity threshold as SPU texts reflecting the common attributes of the product. Data labeling and model training are not required, fewer computing resources are required, and costs are reduced. At the same time, it is convenient to deploy to edge devices with low computing power.

[0088] The embodiment of the present invention also provides a device for extracting common attributes of commodities. Figure 3 A schematic diagram of a device for extracting common attributes of commodities provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, the commodity general attribute extraction device includes:

[0089] SKU text acquisition module 201, used to acquire SKU text reflecting product information;

[0090] A data cleaning module 202 is used to clean the SKU text, remove specific information reflecting specific attributes of the product in the SKU text, and obtain a target text;

[0091] A bucketing module 203 is used to bucket the target texts based on the text length of the target texts, and put the target texts whose text lengths are within a preset range into the same text bucket;

[0092] A similarity calculation module 204 is used to calculate the similarity between the target texts in each text bucket for each text bucket;

[0093] The SPU text determination module 205 is configured to use the plurality of target texts whose similarity is greater than a similarity threshold as SPU texts reflecting the common attributes of commodities.

[0094] In some embodiments of the present invention, the data cleaning module 202 includes:

[0095] A conversion submodule, used for converting uppercase letters in the SKU text into lowercase letters;

[0096] The matching submodule is used to match the converted SKU text with regular matching to obtain specific information reflecting the specific attributes of the product;

[0097] The deletion submodule is used to delete specific information reflecting specific attributes of the product;

[0098] The brand adding submodule is used to add the brand name of the product to the text after deleting the specific information to obtain the target text.

[0099] In some embodiments of the present invention, the bucketing module 203 includes:

[0100] The statistical submodule is used to count the number of occurrences of each text length and obtain an array sequence of text length-number of occurrences;

[0101] A fitting submodule, used for fitting the array sequence to obtain a relationship curve between the number of occurrences of the text length and the text length;

[0102] An extreme value determination submodule, used for finding the text length corresponding to the maximum value of the relationship curve;

[0103] The bucketing submodule is used to put the multiple target texts whose text lengths are within a preset range into the same text bucket based on the text lengths corresponding to the maximum values.

[0104] In some embodiments of the present invention, the bucketing submodule includes:

[0105] An average value calculation unit, used to calculate the average value of the text lengths corresponding to two adjacent maximum values;

[0106] The segmentation unit is used to take the text length corresponding to the average value as a segmentation point, and put multiple target texts between two adjacent segmentation points into the same text bucket.

[0107] In some embodiments of the present invention, the commodity general attribute extraction device further includes:

[0108] A deduplication module, used for deduplicating a plurality of target texts and deleting duplicate target texts before calculating the similarity between the target texts in each text bucket for each text bucket;

[0109] A word segmentation module is used to segment the target text and determine the word vector corresponding to each word;

[0110] A weight calculation module is used to assign a weight value to the word based on the modulus of the word vector.

[0111] In some embodiments of the present invention, the weight calculation module includes:

[0112] A modulus calculation submodule, used to calculate the modulus of the word vector;

[0113] A judgment submodule, used to judge whether the word is a number;

[0114] a first weight determination submodule, configured to calculate, when the word is a number, a product of a modulus of the word vector and a weight adjustment coefficient as a weight value of the word, wherein the weight adjustment coefficient is greater than 1;

[0115] The first weight determination submodule is used to use the modulus of the word vector as the weight value of the word when the word is not a number.

[0116] In some embodiments of the present invention, the similarity calculation module 204 includes:

[0117] A target vector determination submodule, for each of the text buckets, multiplies each word vector in each of the target texts in the text bucket by a corresponding weight value to obtain a target vector;

[0118] The similarity calculation submodule is used to calculate the similarity between the target vectors.

[0119] The above-mentioned commodity general attribute extraction device can execute the commodity general attribute extraction method provided by the above-mentioned embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the commodity general attribute extraction method.

[0120] Figure 4 A schematic diagram of the structure of an electronic device provided for an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0121] like Figure 4 As shown, the electronic device includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0122] A number of components in the electronic device are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0123] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for extracting general attributes of commodities.

[0124] In some embodiments, the method for extracting common attributes of goods may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for extracting common attributes of goods described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for extracting common attributes of goods in any other appropriate manner (e.g., by means of firmware).

[0125] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0126] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0127] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0128] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0129] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0130] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0131] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the method for extracting common attributes of commodities provided in any embodiment of the present application.

[0132] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0133] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0134] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for extracting common attributes of commodities, characterized in that: include: Get the SKU text reflecting the product information; Performing data cleaning on the SKU text to remove specific information reflecting specific attributes of the product in the SKU text to obtain a target text; Dividing the target texts into buckets based on the text lengths of the target texts, and placing the target texts whose text lengths are within a preset range into the same text bucket; For each of the text buckets, calculating the similarity between the target texts in the text bucket; taking the plurality of target texts whose similarity is greater than a similarity threshold as SPU texts reflecting the common attributes of commodities; The step of performing data cleaning on the SKU text to remove specific information reflecting specific attributes of the product in the SKU text to obtain the target text includes: Convert uppercase letters in the SKU text to lowercase letters; Regular matching is used to match the converted SKU text to obtain specific information that reflects the specific attributes of the product. Delete specific information reflecting specific attributes of the product; The brand name of the product is added to the text after the specific information is deleted to obtain the target text.

2. The method for extracting common attributes of commodities according to claim 1, characterized in that: Bucketing the target texts based on their lengths, and placing the target texts whose lengths are within a preset range into the same text bucket includes: Count the number of occurrences of each text length and obtain an array sequence of text length-number of occurrences; Fitting the array sequence to obtain a relationship curve between the number of occurrences of the text length and the text length; Calculate the text length corresponding to the maximum value of the relationship curve; Based on the text length corresponding to the maximum value, the plurality of target texts whose text lengths are within a preset range are put into the same text bucket.

3. The method for extracting common attributes of commodities according to claim 2, characterized in that: Putting the target texts whose text lengths are within a preset range into the same text bucket based on the text lengths corresponding to the maximum value includes: Calculate the average text length corresponding to two adjacent maximum values; The text length corresponding to the average value is used as a segmentation point, and multiple target texts between two adjacent segmentation points are put into the same text bucket.

4. The method for extracting common attributes of commodities according to any one of claims 1 to 3, characterized in that: Before calculating the similarity between the target texts in each text bucket, the method further includes: Deduplication is performed on the plurality of target texts to delete duplicate target texts; Segment the target text and determine the word vector corresponding to each word; A weight value is assigned to the word based on the modulus of the word vector.

5. The method for extracting common attributes of commodities according to claim 4, characterized in that: Assigning a weight value to the word based on the modulus of the word vector includes: Calculate the modulus of the word vector; Determine whether the word is a number; If yes, then the product of the modulus of the word vector and the weight adjustment coefficient is calculated as the weight value of the word, and the weight adjustment coefficient is greater than 1; If not, the modulus of the word vector is used as the weight value of the word.

6. The method for extracting common attributes of commodities according to claim 4, characterized in that: For each of the text buckets, calculating the similarity between the target texts in the text buckets includes: For each of the text buckets, multiply each word vector in each of the target texts in the text bucket by a corresponding weight value to obtain a target vector; The similarity between the target vectors is calculated.

7. A device for extracting common attributes of commodities, characterized in that: include: SKU text acquisition module, used to obtain SKU text reflecting product information; A data cleaning module is used to clean the SKU text, remove specific information reflecting specific attributes of the product in the SKU text, and obtain a target text; A bucketing module, used for bucketing the plurality of target texts based on the text length of the target texts, and putting the plurality of target texts whose text lengths are within a preset range into the same text bucket; A similarity calculation module, used for calculating the similarity between the target texts in each text bucket for each text bucket; An SPU text determination module, used for taking the plurality of target texts whose similarity is greater than a similarity threshold as SPU texts reflecting the common attributes of commodities; The data cleaning module includes: A conversion submodule, used for converting uppercase letters in the SKU text into lowercase letters; The matching submodule is used to match the converted SKU text with regular matching to obtain specific information reflecting the specific attributes of the product; The deletion submodule is used to delete specific information reflecting specific attributes of the product; The brand adding submodule is used to add the brand name of the product to the text after deleting the specific information to obtain the target text.

8. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for extracting common attributes of commodities as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for extracting common attributes of commodities as described in any one of claims 1 to 6 is implemented.

Images