Abnormal commodity identification method and system based on instruction questions and answers

Abstract

The invention relates to an abnormal commodity recognition method and system based on instruction questions and answers, and the method comprises the steps: obtaining a query instruction inputted by a user, and inputting the query instruction based on a natural language of the user; analyzing the query instruction through a natural language processing module to generate an analysis result; determining the type of the query task according to the analysis result, and creating a corresponding query request; extracting feature information from the multi-modal data source according to the query request, performing anomaly analysis on the feature information, and generating an anomaly recognition report; and feeding back the exception recognition report to the display end, and if the exception recognition result shows that an exception exists, extracting visual data of a corresponding exception situation to the display end according to the exception recognition report. The method has the effect of improving the user experience.

Description

Technical Field

[0001] The present application relates to the field of retail security and artificial intelligence interaction technology, and in particular to a method and system for identifying abnormal commodities based on command question and answer. Background Art

[0002] With the rapid development of e-commerce and logistics industries, the number of abnormal situations in the circulation of goods has increased day by day, which has caused great troubles to merchants and consumers. Traditional methods of identifying abnormal goods usually rely on manual monitoring or a single data source, and cannot effectively identify all potential abnormal situations in real time. For example, in the process of goods leaving the warehouse, transportation, listing and sales, there may be problems such as product quality, logistics anomalies, inventory differences, etc.

[0003] The above-mentioned existing technical solutions have the following defects: most of the existing abnormal product identification methods rely on preset query conditions or data fields, and users need to perform more complex operations during the query process, lacking flexible interaction methods, so there is room for improvement. Summary of the invention

[0004] In order to improve user experience, the present application provides a method and system for identifying abnormal products based on command question and answer.

[0005] The above-mentioned invention objective of the present application is achieved through the following technical solutions: A method for identifying abnormal commodities based on command question and answer, the method comprising: obtaining a query instruction input by a user, wherein the query instruction is based on a natural language input of the user; Parsing the query instruction through a natural language processing module to generate a parsing result; According to the analysis result, determine the type of query task and create a corresponding query request; Extracting feature information from a multimodal data source according to the query request, performing anomaly analysis on the feature information, and generating an anomaly identification report; The abnormality identification report is fed back to the display end. If the abnormality identification result shows that an abnormality exists, the visualization data of the corresponding abnormal situation is extracted and sent to the display end according to the abnormality identification report.

[0006] By adopting the above technical solution, by capturing the user's natural language input, the system can flexibly receive user instructions, avoiding the fixed format and complex operations in the traditional query method, thereby improving the user experience; by processing the natural language input by the user, the system can accurately extract the user's query intention and specific needs, thereby providing precise guidance for subsequent task processing, which not only improves the query efficiency, but also enables the system to understand complex and changeable user expressions, and expands the system's applicable scenarios; based on the natural language parsing results, the system can automatically identify the specific task type of the user's query and generate corresponding query requests, which not only saves the user's operation time, but also ensures It ensures the accuracy and precision of the query and avoids the tedious steps of users manually setting query parameters; according to the query request, the system extracts feature information related to the query from multimodal data sources such as video, scanned code, structured data, etc., which provides more comprehensive and multi-angle data support; after identifying the abnormal situation, the system will automatically feedback the abnormal identification report to the user and extract relevant visual data from the report so that the user can intuitively understand the specific context of the abnormality. This feedback method improves the efficiency and accuracy of exception handling. At the same time, through visual data, it helps users understand the causes and impacts of the abnormality more quickly and clearly, enhancing the operability and user-friendliness of the system.

[0007] In a preferred example, the present application may be further configured as follows: parsing the query instruction by a natural language processing module to generate a parsing result includes: Performing language analysis on the query instruction using a pre-trained language model to generate the type of the query instruction; According to the type of the query instruction, a corresponding parsing result is generated, and the parsing result includes the specific content and target of the query task.

[0008] By adopting the above technical solution, by using the trained language model, the system can quickly and accurately analyze the natural language queries input by users. This method enables the system to understand complex language structures and process different grammars and semantics, so as to adapt to various user input expressions, which not only improves processing efficiency but also ensures the accuracy of analysis results. Through language analysis of query instructions, the system can automatically identify the type of instruction. The system can adopt different processing methods according to different task types, optimize query efficiency, and provide clear guidance for subsequent operations. After identifying the query type, the system will generate matching parsing results according to the specific requirements of each type, ensuring the accuracy and pertinence of the query, so that subsequent processing can complete the task more pertinently and efficiently.

[0009] In a preferred example, the present application may be further configured as follows: determining the type of the query task according to the parsing result and creating a corresponding query request includes: Identify the query type according to the analysis result, and determine the type of the query task, the type of the query task including but not limited to abnormal product query, abnormal order query and abnormal type query; According to the type of the query task, a query request corresponding to the query task is created, and the query request includes a query data source, a query field, and a query condition.

[0010] By adopting the above technical solution and analyzing the parsing results, the system can accurately identify the specific task types of query instructions, such as abnormal product queries, abnormal order queries, etc., ensuring that the system can correctly classify query tasks according to user needs, thereby adopting a processing method suitable for this type of task, and can effectively distinguish different types of query requests, thereby improving the pertinence and accuracy of system responses; by clearly listing a variety of query task types, it is ensured that the system can perform diversified processing when dealing with different types of queries, meet a wide range of application needs, and avoid limiting system functions due to overly narrow task types; based on the type of query task, the system will automatically create corresponding query requests, ensuring efficient generation of query requests, so that subsequent data retrieval can be executed accurately and quickly, and according to the specific requirements of the query task, the system will accurately allocate the required resources to avoid invalid queries or information redundancy.

[0011] In a preferred example, the present application may be further configured as follows: extracting feature information from a multimodal data source according to the query request includes: Extracting a commodity operation video clip related to the query instruction from a video data source to generate a commodity operation video clip; Extracting product code scanning records related to the query instruction from the code scanning data source, and generating product code scanning records; Extracting commodity trajectory information related to the query instruction from a structured data source to generate commodity trajectory information; The commodity operation video clip, the commodity code scanning record and the commodity trajectory information are integrated to obtain the feature information.

[0012] By adopting the above technical solution, by extracting the operation video clips related to the query instruction from the video data source, the system can obtain the real-time or historical operation process related to the product, ensuring that the user can directly observe the operation of the product from a visual perspective, providing key dynamic information for further abnormal identification, and thus providing support for analyzing whether the product has abnormal behavior; by extracting the scanning record of the product from the scanning data source, the system can obtain the identity information of the product, which helps to track the circulation information of the product, ensure the traceability of the movement of the product in the supply chain, and then support the identification and positioning of abnormal products; by extracting the trajectory information of the product from the structured data source, the system can obtain the circulation path data of the product at different stages, which can help the system restore the actual circulation process of the product, thereby providing a comprehensive view of the product life cycle, which helps to analyze the root cause of the abnormal situation; by integrating the video clips, scanning records and trajectory information into a unified feature information, the system can comprehensively analyze the behavior and status of the product from multiple dimensions, ensuring the comprehensiveness of abnormal identification, and improving the accuracy and timeliness of the system's identification of abnormalities through the combination of cross-source data, thereby more accurately identifying potential problems of the product.

[0013] In a preferred example, the present application may be further configured as follows: performing abnormal analysis on the feature information and generating an abnormal identification report includes: Perform weight calculation on each feature point in the feature information to generate a priority of each feature point in the feature information; based on the priority of each feature point in the feature information, use a preset anomaly detection algorithm to analyze the feature information to generate an anomaly detection result, and generate the anomaly identification report based on the anomaly detection result, wherein the anomaly identification report includes but is not limited to the type of abnormal product, the type of anomaly and the cause of the anomaly.

[0014] By adopting the above technical solution, by calculating the weight of each feature point in the feature information, the system can identify and quantify the importance of each feature point in anomaly identification, helping the system to automatically focus on those feature points that may have an important impact on anomaly identification, thereby improving the efficiency and accuracy of anomaly analysis; by combining the priority of feature points with the preset anomaly detection algorithm, the system can sort the feature points according to their importance during the analysis process, give priority to those key features, ensure accurate detection of possible abnormal situations, and identify the most influential abnormal signals in multidimensional data; based on the anomaly detection results, the system generates a detailed anomaly identification report, the report content includes the category of abnormal goods, the specific type of anomaly and its possible causes. Through the generation of the report, the system can clearly feedback the specific details of the abnormal situation, help users quickly understand the cause of the anomaly, and provide accurate data support for subsequent processing or decision-making.

[0015] In a preferred example, the present application may be further configured as follows: if the abnormality recognition result shows that an abnormality exists, extracting the visualization data of the corresponding abnormal situation to the display terminal according to the abnormality recognition report includes: Extract abnormal information from the abnormal identification report to obtain information about abnormal commodities; Based on the information of the abnormal product, visualization data of the abnormal situation corresponding to the abnormal product is extracted, and the visualization data of the abnormal situation is transmitted to the display end, wherein the visualization data of the abnormal situation includes real-time video clips, code scanning records, product ID and trajectory data related to the abnormal product.

[0016] By adopting the above technical solution, by extracting detailed information related to the anomaly from the anomaly identification report, the system can identify and extract the product data involved in the anomaly, and can accurately locate which products have anomalies, ensuring that product information with abnormal characteristics is screened out from a large amount of data, providing a basis for subsequent situational analysis and problem troubleshooting; based on the key information extracted from the abnormal products, the system can further extract visualization content of abnormal situations related to the abnormal products from multiple data sources. Through this visualization display, users can intuitively see the detailed situations and behaviors related to the abnormal products, helping users to quickly identify and locate problems.

[0017] In a preferred example, the present application can be further configured as follows: the abnormal commodity identification method based on instruction question and answer also includes: Obtaining background information of the user query, and providing supplementary explanation to the anomaly identification report based on the background information and in combination with real-time environmental data; After the abnormal identification report is generated, corresponding user guidance suggestions are generated according to the abnormal type of the identification report, and the user guidance suggestions include but are not limited to operation steps, problem troubleshooting suggestions and abnormal product handling solutions; The user guidance suggestions are converted into natural language text through the natural language processing module, the user guidance suggestions are displayed on the display end, and user interaction options are provided.

[0018] By adopting the above technical solution, by obtaining the background information of the user's query and the real-time environmental data, the system can understand the user's problems and situations more accurately, so that the anomaly identification report does not only stay at the level of raw data analysis, but is dynamically supplemented and adjusted in combination with the context, providing more targeted and realistic explanations, thereby improving the applicability of the report and the efficiency of problem solving; according to the anomaly type in the generated anomaly identification report, the system can automatically generate user guidance suggestions for the anomaly to help users take correct actions quickly, thereby shortening the time to solve the problem and improving the response efficiency of the system; through the natural language processing module, the guidance suggestions are converted into natural language text that is easy for users to understand. The system can present complex technical suggestions to users in a simple and clear manner. At the same time, the system also provides interactive options, allowing users to choose to continue the operation or get more help as needed, thereby improving the user experience and ensuring that users can more conveniently follow up on the guidance suggestions and make corresponding operations.

[0019] The second object of the invention is achieved by the following technical solutions: A system for identifying abnormal commodities based on command question and answer, the system comprising: a query instruction acquisition module, used to acquire a query instruction input by a user, wherein the query instruction is based on a natural language input of the user; A parsing module, used to parse the query instruction through a natural language processing module to generate a parsing result; A request creation module is used to determine the type of query task according to the analysis result and create a corresponding query request; an anomaly identification module is used to extract feature information from a multimodal data source according to the query request, perform an anomaly analysis on the feature information, and generate an anomaly identification report; The display module is used to feed back the abnormality identification report to the display end. If the abnormality identification result shows that an abnormality exists, the visualization data of the corresponding abnormal situation is extracted and sent to the display end according to the abnormality identification report.

[0020] By adopting the above technical solution, by capturing the user's natural language input, the system can flexibly receive user instructions, avoiding the fixed format and complex operations in the traditional query method, thereby improving the user experience; by processing the natural language input by the user, the system can accurately extract the user's query intention and specific needs, thereby providing precise guidance for subsequent task processing, which not only improves the query efficiency, but also enables the system to understand complex and changeable user expressions, and expands the system's applicable scenarios; based on the natural language parsing results, the system can automatically identify the specific task type of the user's query and generate corresponding query requests, which not only saves the user's operation time, but also ensures It ensures the accuracy and precision of the query and avoids the tedious steps of users manually setting query parameters; according to the query request, the system extracts feature information related to the query from multimodal data sources such as video, scanned code, structured data, etc., which provides more comprehensive and multi-angle data support; after identifying the abnormal situation, the system will automatically feedback the abnormal identification report to the user and extract relevant visual data from the report so that the user can intuitively understand the specific context of the abnormality. This feedback method improves the efficiency and accuracy of exception handling. At the same time, through visual data, it helps users understand the causes and impacts of the abnormality more quickly and clearly, enhancing the operability and user-friendliness of the system.

[0021] In summary, the present application includes at least one of the following beneficial technical effects: 1. By capturing the user's natural language input, the system can flexibly receive user instructions, avoiding the fixed format and complex operations in traditional query methods, thereby improving the user experience; by processing the natural language input by the user, the system can accurately extract the user's query intention and specific needs, thereby providing precise guidance for subsequent task processing, which not only improves the query efficiency, but also enables the system to understand complex and changeable user expressions, expanding the system's applicable scenarios; 2. Based on the natural language analysis results, the system can automatically identify the specific task type of the user's query and generate the corresponding query request, which not only saves the user's operation time, but also ensures the query The accuracy and precision of the system is improved, avoiding the tedious steps of users manually setting query parameters; according to the query request, the system extracts feature information related to the query from multimodal data sources such as video, scanned code, structured data, etc., which provides more comprehensive and multi-angle data support; after identifying the abnormal situation, the system will automatically feedback the abnormal identification report to the user, and extract relevant visual data from the report so that the user can intuitively understand the specific context of the abnormality. This feedback method improves the efficiency and accuracy of exception handling. At the same time, through visual data, it helps users understand the causes and impacts of the abnormality more quickly and clearly, enhancing the operability and user-friendliness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flow chart of a method for identifying abnormal commodities based on command question and answer in one embodiment of the present application; Figure 2 This is a flowchart for implementing step S20 in a method for identifying abnormal commodities based on command question and answer in one embodiment of the present application; Figure 3 This is a flowchart for implementing step S30 in a method for identifying abnormal commodities based on command question and answer in one embodiment of the present application; Figure 4 This is a flowchart for implementing step S40 in a method for identifying abnormal commodities based on command question and answer in one embodiment of the present application; Figure 5 This is a flowchart for implementing step S40 in a method for identifying abnormal commodities based on command question and answer in one embodiment of the present application; Figure 6 This is a flowchart for implementing step S50 in a method for identifying abnormal commodities based on command question and answer in one embodiment of the present application; Figure 7 This is a flowchart of an implementation of a method for identifying abnormal commodities based on command question and answer in one embodiment of the present application; Figure 8 It is a principle block diagram of an abnormal commodity identification system based on command question and answer in one embodiment of the present application. DETAILED DESCRIPTION

[0023] The present application is further described in detail below in conjunction with the accompanying drawings.

[0024] In one embodiment, if Figure 1 As shown, the present application discloses a method for identifying abnormal commodities based on command question and answer, which specifically includes the following steps: S10: Obtaining a query instruction input by a user, where the query instruction is based on a natural language input by the user.

[0025] Specifically, the user's natural language input is first received. The user may express the query demand for product anomalies in a free form, such as "Please check the anomaly of product A", "Product B has a problem, can you check it for me?" or "Check whether there are any missed products in this order?" This natural language input can be received through keyboard input or voice recognition. For voice input, voice recognition technology converts voice into text, and then processes the input as a query instruction, and determines the subject and scope of the query by automatically identifying the user's intention.

[0026] S20: Parse the query instruction through the natural language processing module to generate a parsing result.

[0027] Specifically, after receiving the user's query instruction, the system analyzes and processes the input natural language through the natural language processing module. This processing process includes steps such as word segmentation, part-of-speech tagging, named entity recognition, and dependency syntax analysis. The purpose is to extract key information from the sentence, such as product name, question type, query requirements, etc. Through these processes, the generated parsing results will clearly describe the specific tasks and goals of the query. For example, the parsing results may indicate that the user is querying for abnormal conditions of the product, and product A is the query object. The goal of this process is to accurately understand the user's intention so that the next steps can be executed accurately.

[0028] S30: Determine the type of the query task according to the parsing result, and create a corresponding query request.

[0029] Specifically, the parsing result will determine the specific type of task based on the target information of the query task, such as the product name or the type of exception. For example, if the parsing result shows that the user is querying "the exception of product A", then the query task type is determined to be a product exception query. If the query is for order-related exceptions, the task type is an order exception query. Next, create a query request corresponding to the query task type. The query request will include the specific query data source, the fields to be queried, and the query conditions for filtering data. For example, in a product exception query task, the query request may include fields such as product ID, operation records, and code scanning information, and filter relevant data based on conditions such as time range and product category.

[0030] S40: extracting feature information from the multimodal data source according to the query request, performing anomaly analysis on the feature information, and generating an anomaly identification report.

[0031] Specifically, according to the query request, the system will extract feature information related to the query task from the multimodal data source. For example, when the query request is for product anomalies, the operation video clips related to the product can be extracted from the video data source. These videos record the operation steps of the product and reflect whether there are any anomalies in the transportation, inventory or sales process of the product; at the same time, the product scanning record is extracted from the code scanning data source, recording the scanning operation information of each product and whether the product is circulated as expected; and then the trajectory information of the product is extracted from the structured data source to reflect the movement of the product between different nodes, forming a complete set of feature information. After the anomaly analysis, these feature information will be processed by the preset algorithm to generate an anomaly identification report.

[0032] S50: Feedback the abnormality identification report to the display end. If the abnormality identification result shows that an abnormality exists, extract the visualization data of the corresponding abnormal situation to the display end according to the abnormality identification report.

[0033] Specifically, when an abnormal identification report is generated, the report will be fed back to the display end and displayed to the user through a graphical interface or report. If the abnormal identification result shows that the product is indeed abnormal, such as abnormal scanning records of the product, non-compliant operation steps, etc., the system will extract and display the visual data related to the abnormality based on the identification report. This visual data includes real-time operation video clips of abnormal products, showing the abnormal situations that may occur during the operation of the product; scanning records, which show the scanning history of the product to help users check whether there are scanning errors or missing records; product ID and trajectory data, visualize the circulation process of the product, so that users can see the location and status of the product in the entire process, helping users to understand abnormal situations more intuitively.

[0034] By adopting the above technical solution, by capturing the user's natural language input, the system can flexibly receive user instructions, avoiding the fixed format and complex operations in the traditional query method, thereby improving the user experience; by processing the natural language input by the user, the system can accurately extract the user's query intention and specific needs, thereby providing precise guidance for subsequent task processing, which not only improves the query efficiency, but also enables the system to understand complex and changeable user expressions, and expands the system's applicable scenarios; based on the natural language parsing results, the system can automatically identify the specific task type of the user's query and generate corresponding query requests, which not only saves the user's operation time, but also ensures It ensures the accuracy and precision of the query and avoids the tedious steps of users manually setting query parameters; according to the query request, the system extracts feature information related to the query from multimodal data sources such as video, scanned code, structured data, etc., which provides more comprehensive and multi-angle data support; after identifying the abnormal situation, the system will automatically feedback the abnormal identification report to the user and extract relevant visual data from the report so that the user can intuitively understand the specific context of the abnormality. This feedback method improves the efficiency and accuracy of exception handling. At the same time, through visual data, it helps users understand the causes and impacts of the abnormality more quickly and clearly, enhancing the operability and user-friendliness of the system.

[0035] In one embodiment, if Figure 2 As shown, in step S20, the query instruction is parsed by the natural language processing module to generate a parsing result, which specifically includes: S21: Perform language analysis on the query instruction using a pre-trained language model to generate the type of the query instruction.

[0036] Specifically, the query command input by the user is first analyzed by a pre-trained language model. The language model is trained by a large-scale corpus and the keyword information in the natural language is used to train the language model built based on the decision tree algorithm so that the language model can recognize and understand the structure and semantics of the natural language. When the user enters a query command, such as "check the inventory status of product A" or "the abnormal record of product B", these commands contain specific goals and content. The language model will identify the keywords and grammatical structures through the decision tree algorithm, and then determine the query intent type. For example, "the inventory status of product A" may be classified as an inventory query type, while "the abnormal record of product B" may be identified as an abnormal query type. In this process, the decision tree algorithm evaluates the splitting effect of each keyword and its grammatical structure by calculating the information gain or Gini index, thereby determining the optimal classification feature. For example, assuming that the best splitting feature is selected by calculating entropy, the decision tree algorithm uses the following information gain formula to evaluate the classification of the query command: A represents the features extracted from the query instruction (such as "product A" or "inventory status"), and D is the instruction set containing all queries. v It is a subset when feature A takes a certain value. The decision tree compares the information gain and selects the feature that is most helpful for classification, thereby determining which task type the query instruction belongs to. At this time, the language model not only recognizes the syntax of the query, but also can understand the semantics of the query through its internal decision tree algorithm and classify it into different task types, such as product query, order query or abnormal type query.

[0037] S22: Generate a corresponding analysis result according to the type of the query instruction, where the analysis result includes the specific content and target of the query task.

[0038] Specifically, after determining the type of query instruction, the next step is to generate the corresponding parsing results according to the query type. The parsing results will clarify the specific details of the query task based on the content of the query instruction. For example, if the query instruction is "check the inventory status of product A", the parsing result will clarify the specific content of the task as "query the inventory status of product A", and the goal is to obtain the inventory information of product A; if the query instruction is "check the abnormal records of product B", the parsing result will clarify the task as "query the abnormal information of product B", and the goal is to obtain the relevant abnormal records of product B and their detailed descriptions. The parsing process includes extracting keywords in the query instruction, such as product ID, query type, query target, etc., and converting this information into a standardized query format.

[0039] In one embodiment, if Figure 3 As shown, in step S30, the type of the query task is determined according to the parsing result, and a corresponding query request is created, which specifically includes: S31: Identify the query type according to the analysis result and determine the type of the query task. The types of the query task include but are not limited to product abnormality query, order abnormality query and abnormal type query.

[0040] Specifically, it is necessary to first analyze the parsing results, extract relevant information features from them, and then determine the specific target of the query. For example, if a consumer encounters a product display error during settlement, the parsing result may indicate that the information of the product in the database does not match the actual situation. At this time, it can be determined that the query task is for product abnormalities. For another example, if the parsing result shows that there is an abnormal state in the consumer's order, such as unsuccessful payment or insufficient product inventory, the system will recognize that this is a demand for order abnormality query. The determination of the query task type not only depends on the simple keyword matching in the parsing results, but also needs to consider the relevance of the data context. For example, if the parsing result points to data information directly related to the consumer's order, and the information is incomplete or inconsistent, it is identified as an order abnormality query. If the parsing result points to a problem with the product barcode, price, name and other information, it is determined to be a product abnormality query. By matching the keywords, fields and values ​​in the parsed information, the query task type is determined, and which type of abnormal information needs to be queried or processed is identified.

[0041] S32: Creating a query request corresponding to the query task according to the type of the query task, where the query request includes a query data source, a query field, and a query condition.

[0042] Specifically, the target data source of the query, the fields required for the query, and the corresponding query conditions are clarified according to the task type. Taking the query of product exceptions as an example, the query request needs to point to the product information data source, and the query fields include product name, price, inventory, etc. The query condition may be to compare the product ID with the product ID at the time of current settlement, or to query the status of a specific product barcode in the system. If the task is an order exception query, the query request will point to the data source of the order information. The query fields may include order ID, product ID, consumer ID, etc., and the query conditions may include whether the order status is "payment successful" or "inadequate inventory". Through the precise matching of these fields and conditions, it can ensure that the query request can get accurate and effective feedback, helping to identify the root cause of the exception.

[0043] In one embodiment, if Figure 4 As shown, in step S40, feature information is extracted from the multimodal data source according to the query request, specifically including: S41: extracting commodity operation video segments related to the query instruction from the video data source, and generating commodity operation video segments.

[0044] Specifically, when the query instruction points to a specific product, it is first necessary to filter out the segments related to the product from the product operation videos stored in the video data source. Through video content analysis technologies, such as image recognition and voice recognition, the system can identify the product information in the video and match it with the product in the query instruction. For example, the product ID, product name or other specific identifier in the video segment can be compared with the product information in the query instruction, so as to extract the operation video segment related to the query instruction, ensuring that the video content is consistent with the product queried by the consumer.

[0045] S42: extracting product scanning records related to the query instruction from the scanning data source, and generating product scanning records.

[0046] Specifically, during the consumer settlement process, the scanning operation will generate scanning records related to the product, which will be stored in the scanning data source. When the query instruction involves the scanning information of the product, the scanning record database is first used to retrieve the scanning records related to the product. The scanning record usually includes information such as the product's barcode, scanning time, scanning location, and scanning operator. For example, after a consumer scans the barcode of a product, the system will extract the scanning record of the product to confirm the time and location of the scanning and whether it meets the relevant query requirements.

[0047] S43: extracting commodity trajectory information related to the query instruction from the structured data source, and generating commodity trajectory information.

[0048] Specifically, the product trajectory information includes structured data such as the inventory flow, location change, and sales status of the product, which are usually stored in a structured database. When a query instruction requests to obtain the trajectory information of a product, all logistics information and sales records related to the product can be retrieved by querying the product ID, product name, or other identifiers. For example, if the query instruction requires to view the outbound record of a certain product, the relevant entries of the product can be queried through the inbound and outbound tables in the database to generate the trajectory information of the product and help analyze the circulation history of the product.

[0049] S44: Integrate the product operation video clip, the product scanning record and the product trajectory information to obtain feature information.

[0050] Specifically, product operation video clips, code scanning records, and product trajectory information are integrated to form complete product feature information, helping to fully display all aspects of the product. For example, product operation video clips can show consumers how to use the product, code scanning records can provide the product's transaction history, and trajectory information can reveal the product's inventory flow and sales. By integrating these three types of information, a full range of product feature information can be generated, allowing consumers to obtain multi-dimensional data support about the product when settling.

[0051] In one embodiment, if Figure 5 As shown, in step S40, the feature information is analyzed for abnormality and an abnormality identification report is generated, which specifically includes: S45: Calculate the weight of each feature point in the feature information to generate the priority of each feature point in the feature information.

[0052] Specifically, each feature point in the feature information, including the product operation video clip, code scanning record and trajectory information, contains information of different importance. In this process, it is first necessary to calculate the weight of each feature point to determine its importance and priority in the anomaly detection process, and quantify the role of different feature points in the transaction process of the product. For example, for a product operation video clip, the weight may be assigned based on the completeness and relevance of the video content. A complete operation video clip will be given a higher weight. At this time, the weight calculation formula can be expressed as: Among them, W video is the weight of the video clip, C video R is the completeness score of the video content. video Score the relevance of the video content, T video is the duration of the video clip. This formula reflects the high priority of complete and relevant operation video clips in anomaly detection. Higher completeness and relevance scores will increase the weight. The weight of the code scanning record may be calculated based on the frequency and time interval of the code scanning. Frequent code scanning records that are closer to the settlement time may be given a higher weight because it can reflect the circulation frequency and possible quality problems of the goods. Finally, the calculated weight will be used to generate the priority of each feature point, which will affect the subsequent anomaly detection analysis.

[0053] S46: Based on the priority of each feature point of the feature information, a preset anomaly detection algorithm is used to analyze the feature information, an anomaly detection result is generated, and an anomaly identification report is generated based on the anomaly detection result. The anomaly identification report includes but is not limited to the type of abnormal product, the type of anomaly and the cause of the anomaly.

[0054] Specifically, based on the priority of each feature point, the importance of the feature point will affect the decision of the detection algorithm during the anomaly detection analysis process. At this stage, the preset anomaly detection algorithm will analyze the weight of each feature point, use the weighted average method for calculation and analysis, and analyze each feature point separately through the weighted average method to check whether there are obvious anomalies, such as the product in the operation video does not match the actual product, the product barcode error in the scan record, and abnormal inventory fluctuations in the product trajectory. Then, based on the priority of the feature point, the algorithm will comprehensively evaluate the relationship between each feature point. If the anomaly of a feature point is highly correlated with the anomaly of other feature points, it may indicate a more serious anomaly. Finally, the analysis results will be summarized into anomaly detection results, indicating which feature points are abnormal, as well as the types of goods, anomaly types and causes of these anomalies, and anomaly identification reports will be generated based on the anomaly detection results.

[0055] In one embodiment, if Figure 6 As shown, in step S50, that is, according to the abnormality identification report, the visualization data of the corresponding abnormal situation is extracted to the display end, which specifically includes: S51: Extract abnormal information from the abnormal identification report to obtain information about abnormal products.

[0056] Specifically, when the anomaly identification report is generated, the key information related to the anomaly needs to be extracted from the report first. The anomaly identification report usually includes the anomaly analysis results of different commodities. Therefore, by performing structured analysis on the data in the report, the identifier of each abnormal commodity (such as commodity ID) and the specific circumstances related to the anomaly (such as price error, inconsistent inventory, etc.) are obtained. For example, if the report marks "commodity ID: 12345" as an anomaly, the system will extract the commodity ID and obtain the detailed information of the commodity for further analysis and processing.

[0057] S52: Extract visualization data of abnormal situations corresponding to abnormal products based on the information of abnormal products, and transmit the visualization data of abnormal situations to a display terminal. The visualization data of abnormal situations include real-time video clips, code scanning records, product IDs, and trajectory data related to abnormal products.

[0058] Specifically, when there are scanning record errors or inventory anomalies in the products scanned by consumers, visual data of abnormal situations corresponding to the abnormal products will be extracted based on the information of the abnormal products. For example, operation video clips will be extracted from the video database based on the product ID to show the use or operation process of the product; scanning records can extract relevant barcode scanning records from the database to show the scanning history of the product; product ID and trajectory data contain information such as inventory changes and transaction records of the product, providing the circulation history of the product. All of this data will be integrated into a set of visual reports that graphically display the status of abnormal products. For example, charts will be used to show inventory fluctuations of the product, and videos will be used to show whether the product meets the expected usage method. In one embodiment, if Figure 7 As shown, the abnormal commodity identification method based on instruction question and answer also includes: S60: Obtain background information of the user's query, and provide additional explanations for the anomaly identification report based on the background information and in combination with real-time environmental data.

[0059] Specifically, first, the background information of the user's query is obtained from the transaction platform, user account or sensor data through the interface, which may include but is not limited to the user's transaction history, product purchase record, user environment (such as equipment and network status at checkout, etc.), and any relevant information provided by the user when querying. When combined with real-time environmental data, the real-time data may include the store environment where the consumer is located, such as the actual inventory status of the product, the normal operation of the settlement equipment, network delays or ambient temperature in the store, etc., to identify whether the abnormal phenomenon in the report is related to the external environment, and supplement or correct the abnormal diagnostic results. For example, if the identification of abnormal products is related to network delays, then the supplementation of real-time environmental data can help confirm that the abnormality is not due to quality problems of the product itself but to misidentification caused by equipment problems.

[0060] S70: After the abnormality identification report is generated, corresponding user guidance suggestions are generated according to the abnormality type of the identification report. The user guidance suggestions include but are not limited to operation steps, problem troubleshooting suggestions and abnormal product handling solutions.

[0061] Specifically, based on the generation results of the exception identification report, it is first necessary to identify the exception types marked in the report, such as out-of-stock products, abnormal prices, scanning errors, etc., and then generate guidance suggestions for users based on these exception types. Taking "out-of-stock products" as an example, the generated user guidance suggestions may include operation steps, such as "try to rescan the product barcode", problem troubleshooting suggestions, such as "please check whether the inventory information has been synchronized to the settlement device", and solutions for abnormal products. Finally, corresponding user guidance suggestions are generated based on the exception type in the identification report.

[0062] S80: Convert the user guidance suggestions into natural language text through a natural language processing module, display the user guidance suggestions on a display terminal, and provide user interaction options.

[0063] Specifically, after the user guidance suggestions are generated, they will be converted into concise and easy-to-understand natural language text through the natural language processing module. In this process, each part of the guidance suggestions is analyzed and converted to ensure that they are clearly expressed and in line with the user's understanding habits. When the suggestions are displayed, the user will be able to see clear operation steps. In addition, user interaction options need to be provided, such as "Confirm step completion", "Skip this step" or "Contact customer service", so that users can make feedback based on their own circumstances. The interactive options can be fed back to the subsequent processing flow based on the user's choice to further adjust or optimize the user experience. For example, if the user chooses "Contact customer service", the system may guide the user to the online customer service page or provide a customer service phone number so that the problem can be resolved as soon as possible.

[0064] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0065] In one embodiment, a system for identifying abnormal commodities based on command question and answer is provided, and the system for identifying abnormal commodities based on command question and answer corresponds one to one with the method for identifying abnormal commodities based on command question and answer in the above embodiment. Figure 8 As shown, the abnormal commodity identification system based on command question and answer includes a query instruction acquisition module, a parsing module, a request creation module, an abnormal identification module and a display module. The detailed description of each functional module is as follows: A query instruction acquisition module is used to acquire a query instruction input by a user, where the query instruction is based on a natural language input by the user; A parsing module, used to parse the query instruction through the natural language processing module and generate a parsing result; Create a request module to determine the type of query task based on the parsing results and create a corresponding query request; The anomaly identification module is used to extract feature information from the multimodal data source according to the query request, perform anomaly analysis on the feature information, and generate an anomaly identification report; The display module is used to feed back the abnormality identification report to the display end. If the abnormality identification result shows that an abnormality exists, the visualization data of the corresponding abnormal situation is extracted and displayed to the display end according to the abnormality identification report.

[0066] Optionally, parsing modules include: A language analysis submodule is used to perform language analysis on the query instruction using a pre-trained language model to generate the type of the query instruction; The type parsing submodule is used to generate corresponding parsing results according to the type of query instruction. The parsing results include the specific content and objectives of the query task.

[0067] Optionally, create a request module including: The type determination submodule is used to identify the query type according to the parsing result and determine the type of the query task. The types of query tasks include but are not limited to abnormal product query, abnormal order query and abnormal type query; The request determination submodule is used to create a query request corresponding to the query task according to the type of the query task, and the query request includes a query data source, a query field, and a query condition.

[0068] Optionally, the anomaly identification module includes: The video extraction submodule is used to extract the commodity operation video clips related to the query instruction from the video data source and generate the commodity operation video clips; The scanning record extraction submodule is used to extract the product scanning records related to the query instruction from the scanning data source and generate the product scanning records; The extraction track submodule is used to extract the commodity track information related to the query instruction from the structured data source and generate the commodity track information; The integration submodule is used to integrate the product operation video clips, product scanning records and product trajectory information to obtain feature information; The weight calculation submodule is used to calculate the weight of each feature point in the feature information and generate the priority of each feature point in the feature information; The anomaly analysis submodule is used to analyze the feature information based on the priority of each feature point of the feature information and use a preset anomaly detection algorithm to generate anomaly detection results, and generate an anomaly identification report based on the anomaly detection results. The anomaly identification report includes but is not limited to the type of abnormal product, the type of anomaly and the cause of the anomaly.

[0069] Optionally, the display module includes: The information extraction submodule is used to extract abnormal information from the abnormal identification report to obtain the information of abnormal commodities; The visualization submodule is used to extract the visualization data of the abnormal situation corresponding to the abnormal product based on the information of the abnormal product, and transmit the visualization data of the abnormal situation to the display end. The visualization data of the abnormal situation includes real-time video clips, code scanning records, product ID and trajectory data related to the abnormal product.

[0070] Optionally, the abnormal commodity identification system based on instruction question and answer further includes: The query module is used to obtain the background information of the user's query and provide supplementary explanations for the anomaly identification report based on the background information and combined with real-time environmental data; A suggestion generation module is used to generate corresponding user guidance suggestions according to the abnormality type of the identification report after the abnormality identification report is generated. The user guidance suggestions include but are not limited to operation steps, problem troubleshooting suggestions and abnormal product handling solutions; The interactive module is used to convert the user guidance suggestions into natural language text through the natural language processing module, display the user guidance suggestions on the display end, and provide user interaction options.

[0071] For the specific definition of an abnormal commodity identification system based on command question and answer, please refer to the definition of an abnormal commodity identification method based on command question and answer in the above text, which will not be repeated here. Each module in the above-mentioned abnormal commodity identification system based on command question and answer can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0072] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

[0073] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for identifying abnormal commodities based on command question and answer, characterized in that: The abnormal commodity identification method based on command question and answer includes: Acquire a query instruction input by a user, where the query instruction is based on a natural language input by the user; Parsing the query instruction through a natural language processing module to generate a parsing result; According to the analysis result, determine the type of query task and create a corresponding query request; Extracting feature information from a multimodal data source according to the query request, performing anomaly analysis on the feature information, and generating an anomaly identification report; The abnormality identification report is fed back to the display end. If the abnormality identification result shows that an abnormality exists, the visualization data of the corresponding abnormal situation is extracted and sent to the display end according to the abnormality identification report.

2. The method for identifying abnormal commodities based on command question and answer according to claim 1, characterized in that: The step of parsing the query instruction by a natural language processing module to generate a parsing result includes: Performing language analysis on the query instruction using a pre-trained language model to generate the type of the query instruction; According to the type of the query instruction, a corresponding parsing result is generated, and the parsing result includes the specific content and target of the query task.

3. The method for identifying abnormal commodities based on command question and answer according to claim 1, characterized in that: Determining the type of the query task according to the parsing result and creating a corresponding query request includes: Identify the query type according to the analysis result, and determine the type of the query task, the type of the query task including but not limited to abnormal product query, abnormal order query and abnormal type query; According to the type of the query task, a query request corresponding to the query task is created, and the query request includes a query data source, a query field, and a query condition.

4. The method for identifying abnormal commodities based on command question and answer according to claim 1, characterized in that: The extracting feature information from the multimodal data source according to the query request includes: Extracting a commodity operation video clip related to the query instruction from a video data source to generate a commodity operation video clip; Extracting product code scanning records related to the query instruction from the code scanning data source, and generating product code scanning records; Extracting commodity trajectory information related to the query instruction from a structured data source to generate commodity trajectory information; The commodity operation video clip, the commodity code scanning record and the commodity trajectory information are integrated to obtain the feature information.

5. The method for identifying abnormal commodities based on command question and answer according to claim 1, characterized in that: The performing abnormality analysis on the feature information and generating an abnormality identification report comprises: Calculating the weight of each feature point in the feature information to generate a priority of each feature point in the feature information; Based on the priority of each feature point of the feature information, the feature information is analyzed using a preset anomaly detection algorithm to generate an anomaly detection result, and based on the anomaly detection result, the anomaly identification report is generated. The anomaly identification report includes but is not limited to the type of abnormal product, the type of anomaly and the cause of the anomaly.

6. The method for identifying abnormal commodities based on command question and answer according to claim 1, characterized in that: If the abnormality identification result shows that an abnormality exists, extracting corresponding visual data of the abnormal situation to the display terminal according to the abnormality identification report includes: Extract abnormal information from the abnormal identification report to obtain information about abnormal commodities; Based on the information of the abnormal product, visualization data of the abnormal situation corresponding to the abnormal product is extracted, and the visualization data of the abnormal situation is transmitted to the display end, wherein the visualization data of the abnormal situation includes real-time video clips, code scanning records, product ID and trajectory data related to the abnormal product.

7. The method for identifying abnormal commodities based on command question and answer according to claim 1, characterized in that: The abnormal commodity identification method based on instruction question and answer also includes: Obtaining background information of the user query, and providing supplementary explanation to the anomaly identification report based on the background information and in combination with real-time environmental data; After the abnormal identification report is generated, corresponding user guidance suggestions are generated according to the abnormal type of the identification report, and the user guidance suggestions include but are not limited to operation steps, problem troubleshooting suggestions and abnormal product handling solutions; The user guidance suggestions are converted into natural language text through the natural language processing module, the user guidance suggestions are displayed on the display end, and user interaction options are provided.

8. An abnormal commodity identification system based on command question and answer, characterized in that: The abnormal commodity identification system based on command question and answer includes: A query instruction acquisition module, used to acquire a query instruction input by a user, wherein the query instruction is based on a natural language input by the user; A parsing module, used to parse the query instruction through a natural language processing module to generate a parsing result; A request creation module is used to determine the type of query task according to the analysis result and create a corresponding query request; An anomaly identification module, used to extract feature information from the multimodal data source according to the query request, perform an anomaly analysis on the feature information, and generate an anomaly identification report; The display module is used to feed back the abnormality identification report to the display end. If the abnormality identification result shows that an abnormality exists, the visualization data of the corresponding abnormal situation is extracted and sent to the display end according to the abnormality identification report.

Images