Problem classification method and device based on large model in marketing industry, medium and equipment

Abstract

The invention provides a problem classification method and device based on a large model in the marketing industry, a medium and equipment. The method comprises the following steps: collecting a user feedback document; segmenting the user feedback document into a plurality of entries, and forming the plurality of entries into an entry list; splicing each entry in the entry list and the prompt information into model input content, and inputting the model input content into the problem identification large model to obtain a model identification result; according to a model identification result, determining whether the item is a problem item, and if so, retaining the item; forming a problem entry list by the reserved problem entries; performing clustering processing on the problem entries in the problem entry list to obtain a plurality of problem entry groups; and sorting the problem item groups to obtain the sorted problem item groups, and forming an ordered array of problem classification according to the sorted problem item groups. According to the invention, efficient, accurate and automatic marketing problem identification and classification statistics are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of problem classification, and in particular to a problem classification method, device, medium and equipment based on a large model in the marketing industry. Background Art

[0002] In the traditional marketing industry, identifying and summarizing user questions and demands, as well as counting and classifying user questions, all need to be done manually, which is an important and tedious task. By counting the top questions of users and providing targeted feedback to users, it is very helpful to improve products and enhance user experience. Summary of the invention

[0003] In response to at least one of the above technical problems, an embodiment of the present invention provides a problem classification method, device, medium, and equipment based on a large model in the marketing industry.

[0004] According to the first aspect, the problem classification method based on a large model in the marketing industry provided by an embodiment of the present invention includes:

[0005] Collect user feedback documents in the marketing industry;

[0006] Dividing the user feedback document into a plurality of items, and forming an item list from the plurality of items;

[0007] Each item in the item list and the prompt information are spliced ​​into model input content, and the model input content is input into the problem recognition model to obtain a model recognition result; according to the model recognition result, it is determined whether the item is a problem item, and if so, the item is retained; otherwise, the item is discarded; and the retained problem items are formed into a problem item list;

[0008] Clustering each question item in the question item list to obtain multiple question item groups;

[0009] According to the order of the number of question items included in each question item group from large to small, each question item group is sorted to obtain arranged question item groups, and an ordered array of question classification is formed based on the sorted question item groups.

[0010] In one embodiment, the collecting of user feedback documents in the marketing industry includes: collecting user messages, letters and customer service conversations in the marketing industry, and forming the user feedback documents with the collected information.

[0011] In one embodiment, dividing the user feedback document into a plurality of items includes:

[0012] Obtain a separator list; wherein the separator list includes multiple separators;

[0013] Identify the separators in the user feedback document;

[0014] Separate the contents between each two discontinuous and nearest delimiters as an entry from the user feedback document to obtain a plurality of entries;

[0015] Entries with lengths less than a first length and greater than a second length are eliminated from the multiple entries obtained by segmentation; wherein the first length is less than the second length, and the second length is determined by an input content length threshold of the large question recognition model.

[0016] In one embodiment, the prompt information includes rules for problem identification by the problem identification large model and examples of problem identification by the problem identification large model; correspondingly, determining whether the entry is a problem entry based on the model identification result includes: if the model identification result output by the problem identification large model for an entry is yes, then the entry is a problem entry.

[0017] In one embodiment, the method further comprises: uploading the question item list to a vector knowledge base.

[0018] In one embodiment, clustering the question items in the question item list to obtain multiple question item groups includes:

[0019] Creating a first array; wherein the first array is used to store the final set of question items after arrangement;

[0020] Create a hash table;

[0021] Taking the subscript of each question item in the question item list as the key of the question item, querying whether the hash table contains the key of the question item; if so, skipping the question item; if not, retrieving the question item whose similarity with the question item is higher than a preset threshold in the vector knowledge base; clustering the retrieved question items into an initial question item group; creating a second array corresponding to the initial question item group; traversing each question item in the initial question item group, taking the subscript of the question item in the question item list as the key of the question item, querying whether the hash table contains the key of the question item; if so, skipping the question item; if not, placing the question item in the second array corresponding to the initial question item group, and placing the subscript of the question item in the question item list in the hash table; after traversing all the question items in the initial question item group, taking the second array corresponding to the initial question item group as the final question item group, and adding each final question item group as an element to the first array.

[0022] In one embodiment, the question item groups are sorted in descending order of the number of question items included in each question item group to obtain arranged question item groups, and an ordered array of question classification is formed based on the sorted question item groups, including:

[0023] Define the json format of the ordered array, wherein the json format involves multiple fields, including a total number of question items field, a total number of categories field, and a field of each final question item group, a sorting position field of the final question item group, and a question item number field of the final question item group;

[0024] Assign the number of question items in the question item list to the total number of question items field; assign the number of elements in the first array to the total number of categories field;

[0025] Determine the number of question items in each final question item group in the first array, and sort the final question item groups in descending order of the number of question items included in each final question item group; assign each final question item group to a corresponding field of the final question item group one by one according to the arrangement order, assign the arrangement sequence number of each final question item group to a corresponding sorting position field, and assign the number of question items in each final question item group to a corresponding field of the number of question items;

[0026] After the assignment is completed, an ordered array in JSON format is obtained.

[0027] According to the second aspect, the problem classification device based on a large model in the marketing industry provided by the embodiment of the present invention includes:

[0028] Document collection module, used to collect user feedback documents in the marketing industry;

[0029] A document segmentation module, used for segmenting the user feedback document into a plurality of items, and forming an item list from the plurality of items;

[0030] The problem identification module is used to combine each item in the item list and the prompt information into model input content, input the model input content into the problem identification model, and obtain a model identification result; according to the model identification result, determine whether the item is a problem item, if so, retain the item; otherwise, discard the item; and form a problem item list with the retained problem items;

[0031] A question clustering module, used for clustering each question item in the question item list to obtain a plurality of question item groups;

[0032] The classification and sorting module is used to sort each question item group in the order of the number of question items included in each question item group from large to small, so as to obtain arranged question item groups; and form an ordered array of question classification based on the sorted question item groups.

[0033] According to a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method provided in the first aspect.

[0034] According to a fourth aspect, an embodiment of the present invention provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method provided in the first aspect is implemented.

[0035] The embodiments of the present invention provide a method and apparatus, medium, and equipment for classifying problems in the marketing industry based on a big model, and a user feedback document in the marketing industry. The big model is used to identify user problems in the document, and then the identified user problems are classified, sorted by the number of problems in each category, and provided to relevant personnel in charge of marketing to help improve the most common problems and demands of users. This method of automatically classifying problems based on a big model can overcome the limitations of manual classification in the prior art and the shortcomings of rule-based classification methods, and achieve efficient, accurate, and automated marketing problem identification and classification statistics. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flowchart of a problem classification method based on a large model in the marketing industry in one embodiment of the present invention;

[0037] Figure 2 It is a structural block diagram of a problem classification device based on a large model in the marketing industry in one embodiment of the present invention. DETAILED DESCRIPTION

[0038] With the development of artificial intelligence technology, especially the rise of large model technology, its powerful text content understanding and generation capabilities provide an automated solution for the identification and classification of user questions, which is conducive to reducing labor costs and accurately grasping user demands.

[0039] To this end, in the first aspect, the embodiment of the present invention provides a problem classification method based on a large model in the marketing industry, see Figure 1 The method comprises the following steps S110 to S150:

[0040] S110. Collect user feedback documents in the marketing industry;

[0041] In one embodiment, collecting user feedback documents of the marketing industry in S110 may include: collecting user messages, letters and customer service conversations of the marketing industry, and forming the user feedback documents with the collected information.

[0042] It can be seen that the user feedback document is formed by obtaining user messages, letters, customer service conversations, etc. The user feedback document will contain some user problems and some content that is not related to the user problems.

[0043] S120, dividing the user feedback document into multiple entries, and forming an entry list from the multiple entries;

[0044] In one embodiment, dividing the user feedback document into multiple items in S120 may include:

[0045] 1. Obtain a separator list; wherein the separator list includes multiple separators;

[0046] The separators include, for example, a period, a question mark, and a line break in Chinese and English.

[0047] 2. Identify the separators in the user feedback document;

[0048] 3. Separate the contents between each two discontinuous and nearest delimiters as an entry from the user feedback document to obtain a plurality of entries;

[0049] 4. Eliminate entries whose length is less than the first length and greater than the second length from the multiple entries obtained by segmentation; wherein the first length is less than the second length, and the second length is determined by the input content length threshold of the problem recognition large model.

[0050] It can be seen that the user feedback document is segmented by the separators in the preset separator list to obtain multiple entries, and then the entries that are too short or too long are discarded.

[0051] Among them, the first length is, for example, 5.

[0052] Among them, the second length depends on the specific large model used. Taking the open source Baichuan2-13B-Chat as an example, its context length is limited to 4096 words.

[0053] S130, splicing each item in the item list and the prompt information into model input content, inputting the model input content into the problem recognition large model to obtain a model recognition result; determining whether the item is a problem item according to the model recognition result, if so, retaining the item; otherwise, discarding the item; and forming a problem item list with the retained problem items;

[0054] The prompt information may include rules for problem identification by the problem identification large model and examples of problem identification by the problem identification large model.

[0055] For example, the question recognition model in a prompt message uses the following rules for question recognition: Your task is to determine whether the following sentence is a question sentence. If it is a question sentence, please output "yes"; otherwise, output "no". Please also output "no" for imperative sentences. The example of question recognition model for question recognition includes four examples:

[0056] <Example 1>

[0057] Content: OK, received, thank you

[0058] Output: No

[0059] Reason: The content is a declarative sentence, not a question.

[0060] <Example 2>

[0061] Content: Can I reset my password?

[0062] Output: Yes

[0063] Reason: The content is a question sentence

[0064] <Example 3>

[0065] Content: Please help me, teacher.

[0066] Output: No

[0067] Reason: The content is an imperative sentence, not a question.

[0068] <Example 4>

[0069] Content: Picture: File: Video: Address: http: / / xxx.jpg.xlsx.mp4

[0070] Output: No

[0071] Reason: The content is a declarative sentence, not a question.

[0072] In S130, an entry and prompt information are spliced ​​into model input content, and the model input content is input into the question recognition model, and the question recognition model can output yes or no.

[0073] In one embodiment, determining whether the entry is a problem entry based on the model recognition result may include: if the model recognition result output by the problem recognition large model for an entry is yes, then the entry is a problem entry.

[0074] It can be seen that if the model recognition result output by the problem recognition model is yes, the entry can be considered to be a problem entry, and if the model recognition result output is no, the entry can be considered to be irrelevant to the problem. Therefore, it is possible to determine whether to retain or discard the entry based on the model recognition result of the problem recognition model. The problem entry list formed by the retained problem entries is the problem entry list after the problem recognition model has screened.

[0075] In one embodiment, the method may further include: uploading the list of question items to a vector knowledge base. Here, uploading the list of question items to the vector knowledge base facilitates subsequent clustering processing.

[0076] S140, clustering each question item in the question item list to obtain a plurality of question item groups;

[0077] For example, there are 30 question items in the question item list, and after clustering, 5 categories are obtained, that is, 5 question item groups. The question items in each question item group have high similarity and can be considered as a type of user question.

[0078] In one embodiment, clustering the question items in the question item list in S140 to obtain multiple question item groups may include:

[0079] 1. Create a first array; wherein the first array is used to store the final set of question items after arrangement;

[0080] A question item group is stored as an element in the first array.

[0081] 2. Create a hash table;

[0082] Among them, the hash table is used to determine whether a certain question entry has been classified.

[0083] 3. Take the subscript of each question item in the question item list as the key of the question item, and query whether the key of the question item exists in the hash table; if it exists, skip the question item; if not, search the vector knowledge base for question items whose similarity with the question item is higher than a preset threshold; cluster the retrieved question items into an initial question item group; create a second array corresponding to the initial question item group; traverse each question item in the initial question item group, take the subscript of the question item in the question item list as the key of the question item, and query whether the key of the question item exists in the hash table; if it exists, skip the question item; if not, place the question item in the second array corresponding to the initial question item group, and place the subscript of the question item in the question item list in the hash table; after traversing all the question items in the initial question item group, take the second array corresponding to the initial question item group as the final question item group, and add each final question item group as an element to the first array.

[0084] Among them, the subscript of each question item in the question item list is unique and can be used as the identifier of each question item, so the subscript is used as the key of the question item. If the subscript of a question item already exists in the hash table, it means that the question item has been classified and does not need to be classified again. If it is not in the hash table, the similarity between the question item and each question item in the vector knowledge base is calculated, and the question items with similarity higher than the preset threshold are selected to form an initial question item group, so as to obtain a preliminary classification. Then check whether each question item in this initial question item group has been classified before, that is, check whether the subscript of each question item in this initial question item group exists in the hash table. If not, the question item is placed in the second array corresponding to the initial question item group. The second array is used to store each question item after further eliminating the classified question items from the question item group. After traversing an initial question item group, the corresponding second array can also be called the final question item group. An initial question item group corresponds to a final question item group, and the subscripts of each question item in the final question item group will be placed in the hash table.

[0085] Through the above process, each initial question item group is processed as above, and the final question item group corresponding to each initial question item group can be obtained. Each final question item group is added as an element in the first array.

[0086] S150, sorting each question item group in descending order of the number of question items included in each question item group to obtain arranged question item groups, and forming an ordered array of question classifications based on the sorted question item groups.

[0087] In one embodiment, in S150, the question item groups are sorted in descending order of the number of question items included in each question item group to obtain arranged question item groups, and an ordered array of question classification is formed according to the sorted question item groups, which may include:

[0088] 1. Define the json format of the ordered array, wherein the json format involves multiple fields, including a total number of question items field, a total number of categories field, and a field of each final question item group, a sorting position field of the final question item group, and a question item number field of the final question item group;

[0089] The specific example is: {"total_questions":,"total_kinds":,"kinds":[{"questions":[],"num":1,"total":},{"questions":[],"num":2,"total":},{"questions":[],"num":3,"total":}]}. In it, total_questions represents the total number of question items; total_kinds represents the total number of categories; kinds represents each question category that has been classified and sorted from high to low according to the number of question items; each category includes the following fields: questions represents the field of the final question item group; num represents the sorting position field of the final question item group; total represents the number of question items in the final question item group.

[0090] 2. Assign the number of question items in the question item list to the total number of question items field; assign the number of elements in the first array to the total number of categories field;

[0091] 3. Determine the number of question items in each final question item group in the first array, and sort the final question item groups in descending order of the number of question items included in each final question item group; assign each final question item group to a corresponding field of the final question item group one by one according to the arrangement order, assign the arrangement sequence number of each final question item group to a corresponding sorting position field, and assign the number of question items in each final question item group to a corresponding field of the number of question items;

[0092] Among them, the sorting method can be bubble sort, quick sort, etc.

[0093] 4. After the assignment is completed, an ordered array in JSON format is obtained.

[0094] For example, the ordered array is: {"total_questions":6,"total_kinds":3,"kinds":[{"questions":["Question 1","Question 2","Question 3"],"num":1,"total":3},{"questions":["Question 4","Question 5"],"num":2,"total":2},{"questions":["Question 6"],"num":3,"total":1}]}.

[0095] The embodiments of the present invention have the following effects:

[0096] 1. For marketers, it reduces labor costs and improves the efficiency of identifying and classifying user issues and demands.

[0097] 2. For developers, they only need to provide documents containing user questions, and they can return a json list of identified and classified questions, which facilitates the development of the front-end interface.

[0098] 3. For enterprises, the method provided by the embodiment of the present invention can be used regularly to provide feedback and make improvements on the top questions most frequently asked by users, which is conducive to improving the competitiveness of products.

[0099] It can be seen that the embodiment of the present invention provides a problem identification and classification solution based on a large model in the marketing industry. By organizing the collected data such as user messages, letters, customer service conversations, etc. into documents, the large model is used to identify the user problems in the documents, and then the identified user problems are matched for similarity, so as to be classified, sorted by the number of each type of problems, and provided to the relevant personnel responsible for marketing to help improve the most user problems and demands. This method of automatically realizing problem classification based on a large model can overcome the limitations of manual classification in the prior art and the shortcomings of rule-based classification methods, and realize efficient, accurate and automated marketing problem identification and classification statistics.

[0100] In a second aspect, an embodiment of the present invention provides a problem classification device based on a large model in the marketing industry, see Figure 2 , the device 100 comprises:

[0101] The document collection module 110 is used to collect user feedback documents in the marketing industry;

[0102] A document segmentation module 120, configured to segment the user feedback document into a plurality of items, and form an item list from the plurality of items;

[0103] The problem identification module 130 is used to combine each item in the item list and the prompt information into model input content, input the model input content into the problem identification model, and obtain a model identification result; according to the model identification result, determine whether the item is a problem item, if so, retain the item; otherwise, discard the item; and form a problem item list with the retained problem items;

[0104] A question clustering module 140, configured to cluster each question item in the question item list to obtain a plurality of question item groups;

[0105] The classification and sorting module 150 is used to sort each question item group in descending order of the number of question items included in each question item group to obtain arranged question item groups; and form an ordered array of question classifications based on the sorted question item groups.

[0106] In one embodiment, the document collection module is specifically used to collect user messages, letters and customer service conversations in the marketing industry, and form the user feedback document with the collected information.

[0107] In one embodiment, the document segmentation module is specifically used to: obtain a separator list; wherein the separator list includes multiple separators; identify the separators existing in the user feedback document; separate the content between each two discontinuous and closest separators as an entry from the user feedback document to obtain multiple entries; and remove entries whose length is less than a first length and greater than a second length from the multiple entries obtained by segmentation; wherein the first length is less than the second length, and the second length is determined by the input content length threshold of the problem recognition large model.

[0108] In one embodiment, the prompt information includes rules for problem identification by the problem identification large model and examples of problem identification by the problem identification large model; correspondingly, the problem identification module determines whether the entry is a problem entry based on the model identification result, including: if the model identification result output by the problem identification large model for an entry is yes, then the entry is a problem entry.

[0109] In one embodiment, the device may further include:

[0110] The item uploading module is used to upload the question item list to the vector knowledge base.

[0111] In one embodiment, the question clustering module is specifically used to: create a first array; wherein the first array is used to store the final question item group after arrangement; create a hash table; use the subscript of each question item in the question item list as the key of the question item, and query whether the key of the question item exists in the hash table; if it exists, skip the question item; if it does not exist, search the vector knowledge base for question items whose similarity with the question item is higher than a preset threshold; cluster the retrieved question items into an initial question item group; create a second array corresponding to the initial question item group; traverse each question item in the initial question item group, use the subscript of the question item in the question item list as the key of the question item, and query whether the key of the question item exists in the hash table; if it exists, skip the question item; if it does not exist, place the question item in the second array corresponding to the initial question item group, and place the subscript of the question item in the question item list in the hash table; after traversing all the question items in the initial question item group, use the second array corresponding to the initial question item group as the final question item group, and add each final question item group as an element to the first array.

[0112] In one embodiment, the classification and sorting module is specifically used to: define the json format of the ordered array, which involves multiple fields, including a total number of question items field, a total number of classifications field, and a field of each final question item group, as well as a sorting position field of the final question item group and a question item number field of the final question item group; assign the number of question items in the question item list to the total number of question items field; assign the number of elements in the first array to the total number of classifications field; determine the number of question items in each final question item group in the first array, and sort each final question item group in order from large to small according to the number of question items included in each final question item group; assign each final question item group to the field of the corresponding final question item group one by one according to the arrangement order, assign the arrangement sequence number of each final question item group to the corresponding sorting position field, and assign the number of question items in each final question item group to the corresponding question item number field; after the assignment is completed, an ordered array in json format is obtained.

[0113] It is understandable that the explanation, specific implementation, beneficial effects, examples, etc. of the relevant contents in the device provided in the embodiment of the present invention can be found in the corresponding parts of the method provided in the first aspect, and will not be repeated here.

[0114] In a third aspect, an embodiment of the present invention provides a computer-readable medium, on which computer instructions are stored. When the computer instructions are executed by a processor, the processor executes the method provided in the first aspect.

[0115] Specifically, a system or device equipped with a storage medium can be provided, on which software program code that implements the functions of any of the above-mentioned embodiments is stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program code stored in the storage medium.

[0116] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0117] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0118] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0119] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0120] It is understandable that the explanation, specific implementation methods, beneficial effects, examples, etc. of the relevant contents in the computer-readable medium provided in the embodiment of the present invention can be found in the corresponding parts of the method provided in the first aspect, and will not be repeated here.

[0121] In a fourth aspect, an embodiment of the present specification provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method in any embodiment of the specification.

[0122] It is understandable that the explanation, specific implementation, beneficial effects, examples, etc. of the relevant contents in the computing device provided in the embodiment of the present invention can be found in the corresponding parts of the method provided in the first aspect, and will not be repeated here.

[0123] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0124] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present invention can be implemented by hardware, software, widgets, or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0125] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A problem classification method based on a large model in the marketing industry, characterized in that: include: Collect user feedback documents in the marketing industry; Dividing the user feedback document into a plurality of items, and forming an item list from the plurality of items; Each item in the item list and the prompt information are spliced ​​into model input content, and the model input content is input into the problem recognition model to obtain a model recognition result; according to the model recognition result, it is determined whether the item is a problem item, and if so, the item is retained; otherwise, the item is discarded; and the retained problem items are formed into a problem item list; Clustering each question item in the question item list to obtain multiple question item groups; According to the order of the number of question items included in each question item group from large to small, each question item group is sorted to obtain arranged question item groups, and an ordered array of question classification is formed based on the sorted question item groups.

2. The method according to claim 1, characterized in that The collecting of user feedback documents of the marketing industry includes: collecting user messages, letters and customer service conversations of the marketing industry, and forming the user feedback documents with the collected information.

3. The method according to claim 1, characterized in that The step of dividing the user feedback document into a plurality of items includes: Obtain a separator list; wherein the separator list includes multiple separators; Identify the separators in the user feedback document; Separate the contents between each two discontinuous and nearest delimiters as an entry from the user feedback document to obtain a plurality of entries; Entries with lengths less than a first length and greater than a second length are eliminated from the multiple entries obtained by segmentation; wherein the first length is less than the second length, and the second length is determined by an input content length threshold of the large question recognition model.

4. The method according to claim 1, characterized in that: The prompt information includes rules for problem identification by the problem identification large model and examples of problem identification by the problem identification large model; Correspondingly, determining whether the entry is a problem entry based on the model recognition result includes: if the model recognition result output by the problem recognition large model for an entry is yes, then the entry is a problem entry.

5. The method according to claim 1, characterized in that The method further comprises: The question item list is uploaded to the vector knowledge base.

6. The method according to claim 1, characterized in that The clustering of each question item in the question item list to obtain a plurality of question item groups includes: Creating a first array; wherein the first array is used to store the final set of question items after arrangement; Create a hash table; Taking the subscript of each question item in the question item list as the key of the question item, querying whether the hash table contains the key of the question item; if so, skipping the question item; if not, retrieving the question item whose similarity with the question item is higher than a preset threshold in the vector knowledge base; clustering the retrieved question items into an initial question item group; creating a second array corresponding to the initial question item group; traversing each question item in the initial question item group, taking the subscript of the question item in the question item list as the key of the question item, querying whether the hash table contains the key of the question item; if so, skipping the question item; if not, placing the question item in the second array corresponding to the initial question item group, and placing the subscript of the question item in the question item list in the hash table; after traversing all the question items in the initial question item group, taking the second array corresponding to the initial question item group as the final question item group, and adding each final question item group as an element to the first array.

7. The method according to claim 6, characterized in that The question item groups are sorted in descending order of the number of question items included in each question item group to obtain arranged question item groups, and an ordered array of question classification is formed based on the sorted question item groups, including: Define the json format of the ordered array, wherein the json format involves multiple fields, including a total number of question items field, a total number of categories field, and a field of each final question item group, a sorting position field of the final question item group, and a question item number field of the final question item group; Assign the number of question items in the question item list to the total number of question items field; assign the number of elements in the first array to the total number of categories field; Determine the number of question items in each final question item group in the first array, and sort the final question item groups in descending order of the number of question items included in each final question item group; assign each final question item group to a corresponding field of the final question item group one by one according to the arrangement order, assign the arrangement sequence number of each final question item group to a corresponding sorting position field, and assign the number of question items in each final question item group to a corresponding field of the number of question items; After the assignment is completed, an ordered array in JSON format is obtained.

8. A problem classification device based on a large model in the marketing industry, characterized in that: include: Document collection module, used to collect user feedback documents in the marketing industry; A document segmentation module, used for segmenting the user feedback document into a plurality of items, and forming an item list from the plurality of items; The problem identification module is used to combine each item in the item list and the prompt information into model input content, input the model input content into the problem identification model, and obtain a model identification result; according to the model identification result, determine whether the item is a problem item, if so, retain the item; otherwise, discard the item; and form a problem item list with the retained problem items; A question clustering module, used for clustering each question item in the question item list to obtain a plurality of question item groups; The classification and sorting module is used to sort each question item group in the order of the number of question items included in each question item group from large to small, so as to obtain arranged question item groups; and form an ordered array of question classification based on the sorted question item groups.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

10. A computing device, characterized in that The method comprises a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 7 is implemented.

Images