Scene recognition method, device, equipment and storage medium

Abstract

This application discloses a scene recognition method, apparatus, device, and storage medium, relating to the field of data processing technology. The method includes: extracting scene recognition query information according to different matching depths to obtain scene recognition sub-query information corresponding to each matching depth; matching each scene recognition sub-query information with business scenario description information corresponding to each matching depth in a preset structured scene knowledge base to obtain candidate business scenario description information corresponding to each matching depth; and determining the scene recognition result corresponding to the multimodal query information based on the candidate business scenario description information. This application can match query information corresponding to each matching depth with business scenario description information of different matching depths, and determine the scene recognition result based on the obtained candidate business scenario description information, thereby solving the technical problem that existing scene recognition methods rely on single keyword matching, which is prone to misjudgment of scenes.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to scene recognition methods, apparatus, devices and storage media. Background Technology

[0002] With the rapid development of artificial intelligence technology, intelligent customer service robots have been widely used in various industries such as finance, telecommunications, and e-commerce. They can automatically answer and handle user questions through natural language processing technology. However, in complex business scenarios (such as big data development and operation platforms), existing intelligent customer service robots still have significant technical shortcomings in scene recognition.

[0003] Currently, mainstream intelligent customer service systems primarily employ rule-based keyword matching methods for scenario identification. Specifically, the system can pre-configure associated keyword sets for each business scenario. When it receives user input text, it can identify whether it contains preset keywords through string matching or regular expression matching, thereby categorizing the user's question into the corresponding scenario. However, keyword matching typically only focuses on the correspondence between literal symbols, and its coverage of textual variations (such as the same concept being expressed in multiple different ways) is limited, which can easily lead to misjudgment of scenarios. Summary of the Invention

[0004] The main purpose of this application is to provide a scene recognition method, apparatus, device and storage medium, which aims to solve the technical problem that existing scene recognition relies on single keyword matching and is prone to misjudgment of scenes.

[0005] To achieve the above objectives, this application proposes a scene recognition method, which includes: Generate scene recognition query information based on multimodal query information input by the user; The scene recognition query information is extracted according to different matching depths to obtain scene recognition sub-query information corresponding to each matching depth. By matching the business scenario description information corresponding to each matching depth in the preset structured scenario knowledge base with the sub-query information of each scenario identification, candidate business scenario description information corresponding to each matching depth is obtained. The scene recognition result corresponding to the multimodal query information is determined based on the description information of each candidate business scenario.

[0006] In one embodiment, the scene identification sub-query information includes the name of the scene to be queried, the keywords of the scene to be queried, and the descriptive text of the scene to be queried; the business scene description information corresponding to each matching depth includes the business scene name, the business scene keywords, and the business scene descriptive text; The step of matching each scenario identification sub-query information with the business scenario description information corresponding to each matching depth in a preset structured scenario knowledge base to obtain candidate business scenario description information corresponding to each matching depth includes: The query scenario name is matched with the business scenario name in the preset structured scenario knowledge base at the first level to determine the candidate business scenario name that matches the query scenario name; The query scenario keywords are matched with the business scenario keywords in a second-level match to determine candidate business scenario keywords that match the query scenario keywords. The third-level matching is performed between the query scenario description text and the business scenario description text to determine the candidate business scenario description text that matches the query scenario description text. The step of determining the scene recognition result corresponding to the multimodal query information based on the description information of each candidate business scenario includes: The scene recognition result corresponding to the multimodal query information is determined based on the candidate business scene name, the candidate business scene keywords, and the candidate business scene description text.

[0007] In one embodiment, the step of determining the scene recognition result corresponding to the multimodal query information based on the candidate business scene name, the candidate business scene keywords, and the candidate business scene description text includes: Determine the candidate business scenarios represented by the candidate business scenario names; Determine whether the candidate business scenario keywords and the candidate business scenario description text belong to the candidate business scenario; If it belongs to the candidate business scenario, then the scene recognition result corresponding to the multimodal query information is generated based on the candidate business scenario.

[0008] In one embodiment, after the step of determining whether the candidate business scenario keywords and the candidate business scenario description text belong to the candidate business scenario, the method further includes: If it does not belong to the category, then check whether the candidate business scenario has a sub-business scenario; If it exists, then determine whether there is a target sub-business scenario in the sub-business scenario that matches the candidate business scenario keywords and the candidate business scenario description text; If it exists, then generate the scene recognition result corresponding to the multimodal query information based on the target sub-business scenario.

[0009] In one embodiment, after the step of matching each scenario identification sub-query information with the business scenario description information corresponding to each matching depth in the preset structured scenario knowledge base, the method further includes: In the event of a failed match, obtain the sequence of user actions performed on the business platform within a preset time window; The module identifier and operation timestamp corresponding to the business function module operated by the user are determined based on the operation behavior sequence. The user's business operation behavior data is obtained based on the module identifier and the operation timestamp; Based on the business operation behavior data, the scene recognition query information is downgraded and matched to obtain the scene recognition result corresponding to the multimodal query information.

[0010] In one embodiment, the step of generating scene recognition query information based on multimodal query information input by the user includes: Extract image query information and text query information from multimodal query information input by the user; Visual content parsing is performed on the image query information to extract text elements and user tag information from the image query information; The text query information, the text elements, and the user tag information are fused to obtain fused query information; The accuracy of the fused query information is verified, and scene recognition query information is generated based on the verification results.

[0011] In one embodiment, the step of verifying the accuracy of the fused query information and generating scene recognition query information based on the verification result includes: Semantic analysis is performed on the fused query information to identify abnormal query information in the fused query information, wherein the abnormal query information is information with ambiguous or incorrect expression. Based on preset interference word filtering rules, detect whether the abnormal query information belongs to scene recognition interference information; If it does not belong to the category, the abnormal query information is corrected, and scene recognition query information is generated based on the corrected abnormal query information.

[0012] Furthermore, to achieve the above objectives, this application also proposes a scene recognition device, the device comprising: The information generation module is used to generate scene recognition query information based on multimodal query information input by the user; The information extraction module is used to extract information from the scene recognition query information according to different matching depths, and obtain scene recognition sub-query information corresponding to each matching depth; The information matching module is used to match the sub-query information of each scenario identification using the business scenario description information corresponding to each matching depth in the preset structured scenario knowledge base, so as to obtain the candidate business scenario description information corresponding to each matching depth. The scene recognition module is used to determine the scene recognition result corresponding to the multimodal query information based on the description information of each candidate business scene.

[0013] In addition, to achieve the above objectives, this application also proposes a scene recognition device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the scene recognition method as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the scene recognition method described above.

[0015] This application provides a scene recognition method. It discloses generating scene recognition query information based on multimodal query information input by the user; extracting information from the scene recognition query information according to different matching depths to obtain scene recognition sub-queries corresponding to each matching depth; matching each scene recognition sub-queries with business scenario description information corresponding to each matching depth in a pre-set structured scene knowledge base to obtain candidate business scenario description information corresponding to each matching depth; and determining the scene recognition result corresponding to the multimodal query information based on each candidate business scenario description information. Because this application can match query information corresponding to each matching depth with business scenario description information of different matching depths in a structured scene knowledge base to obtain candidate business scenario description information corresponding to each matching depth, and determine the scene recognition result based on these candidate business scenario descriptions, it solves the technical problem of existing scene recognition methods that rely on single keyword matching and are prone to misjudgment of scenes. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating an embodiment of the scene recognition method of this application. Figure 2 This is a structural example diagram of the structured scene knowledge base in the scene recognition method of this application; Figure 3 This is a schematic diagram of the intelligent customer service process in the scenario recognition method of this application; Figure 4 This is a schematic diagram of the query information matching process in the scene recognition method of this application; Figure 5 This is a flowchart illustrating Embodiment 2 of the scene recognition method of this application; Figure 6 This is a flowchart illustrating Embodiment 3 of the scene recognition method of this application; Figure 7 This is a flowchart illustrating the overall scene recognition process in the scene recognition method of this application. Figure 8 This is a schematic diagram of the module structure of the scene recognition device according to an embodiment of this application; Figure 9 This is a schematic diagram of the device structure of the hardware operating environment involved in the scene recognition method in this application embodiment.

[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The main solution of this application embodiment is as follows: generating scene recognition query information based on multimodal query information input by the user; extracting information from the scene recognition query information according to different matching depths to obtain scene recognition sub-query information corresponding to each matching depth; matching each scene recognition sub-query information with the business scenario description information corresponding to each matching depth in the preset structured scene knowledge base to obtain candidate business scenario description information corresponding to each matching depth; and determining the scene recognition result corresponding to the multimodal query information based on each candidate business scenario description information.

[0023] Existing intelligent customer service systems mainly use rule-based keyword matching methods for scene recognition. However, since keyword matching usually only focuses on the correspondence of literal symbols, its coverage of text expression variations (such as the same concept being expressed in multiple different ways) is limited, which can easily lead to misjudgment of the scene.

[0024] This application provides a solution that can match query information corresponding to each matching depth with business scenario description information of different matching depths in a structured scenario knowledge base to obtain candidate business scenario description information corresponding to each matching depth, and determine the scenario recognition result based on these candidate business scenario description information. This solves the technical problem that existing scenario recognition relies on single keyword matching and is prone to misjudgment of scenarios.

[0025] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone; or an electronic device or scene recognition device capable of performing the above functions; or a device that includes both a scene recognition device and a scene recognition system. The following description uses a scene recognition system (hereinafter referred to as the system) as an example to illustrate this embodiment and the subsequent embodiments.

[0026] Based on this, embodiments of this application provide a scene recognition method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the scene recognition method of this application.

[0027] In this embodiment, the scene recognition method includes steps S10 to S40: Step S10: Generate scene recognition query information based on the multimodal query information input by the user.

[0028] It is understood that the aforementioned multimodal query information can be information input by the user instructing the intelligent customer service robot to perform data queries. In this embodiment, the multimodal query information may include, but is not limited to, text information and image information. The text information may be a natural language description input by the user; the image information may be screenshots, error images, etc., uploaded by the user.

[0029] It is also understood that the aforementioned scene recognition query information can be standardized input information used for scene matching. In practical applications, after receiving image and text information input by the user, the system can use a multimodal large model to extract effective information from the image (such as user tags and error messages), and fuse the effective information in the image with the text information to generate a unified query text. Furthermore, the system can also perform accuracy verification on the fused text, correcting possible typos, semantic ambiguities, etc., while retaining key scene keywords to obtain the final scene recognition query information. This embodiment, by standardizing the scene recognition query information, can ensure the accuracy and completeness of the input information, thereby providing a high-quality foundation for subsequent scene matching.

[0030] Step S20: Extract information from the scene recognition query information according to different matching depths to obtain scene recognition sub-query information corresponding to each matching depth.

[0031] It should be noted that the aforementioned matching depth can be different matching levels or dimensions of scene matching. In this embodiment, the matching depth may include, but is not limited to, scene name matching, keyword matching, and descriptive text matching. In the actual scene matching process, the system can perform scene matching sequentially according to the matching depth from fuzzy to precise to improve the accuracy of scene recognition.

[0032] It should also be noted that the above-mentioned scene identification subquery information can be query fragments or features extracted from the original scene identification query information that are related to each matching depth. For example, for scene name matching depth, the extracted scene identification subquery information can be the scene name, such as "application development"; for keyword matching depth, the extracted scene identification subquery information can be key terms, such as "insufficient permissions" or "table cannot be created"; for descriptive text matching depth, the extracted scene identification subquery information can be a longer semantic description, such as "encountered a permission error when creating a table in the application development environment".

[0033] In practical applications, the system can predefine three matching depths: Depth 1: scene name matching; Depth 2: keyword matching; Depth 3: description text matching. The system can then parse the scene identification query information and extract possible scene names, keywords, and description text from it to obtain the scene identification sub-query information corresponding to each matching depth.

[0034] Step S30: Match the sub-query information of each scenario identification using the business scenario description information corresponding to each matching depth in the preset structured scenario knowledge base, and obtain the candidate business scenario description information corresponding to each matching depth.

[0035] It should be noted that the aforementioned pre-built structured scenario knowledge base can be a pre-constructed structured database used to store business scenarios and their related attribute information. It can use JSON (JavaScript Object Notation) format for data storage to ensure clear data hierarchy, ease of machine parsing, and efficient retrieval. (See reference...) Figure 2 , Figure 2 This is a structural example diagram of the structured scene knowledge base in the scene recognition method of this application. (See diagram for example.) Figure 2As shown, in the pre-defined structured scenario knowledge base, each scenario can be defined as a data entity, which can contain business scenario description information corresponding to different matching depths, such as scenario name, scenario description, and several keywords. Furthermore, each business scenario can also have one or more sub-business scenarios, and each sub-business scenario can also have a corresponding sub-scenario name and sub-scenario description.

[0036] It should also be noted that the above-mentioned business scenario description information can be feature description data corresponding to different matching levels (i.e. matching depths) in a pre-defined structured scenario knowledge base. For example, for scenario name matching depth, the corresponding business scenario description information can be the scenario name in the knowledge base; for keyword matching depth, the corresponding business scenario description information can be the key terms in the knowledge base; and for description text matching depth, the corresponding business scenario description information can be the semantic description in the knowledge base.

[0037] It should be noted that the aforementioned candidate business scenario description information can be complete structured information of one or more scenarios that may match the user's intent, selected from the business scenario description information. In this embodiment, the candidate business scenario description information may include all attribute information of the hit scenario, such as scenario name, description, keywords, etc., and this embodiment does not impose any restrictions on this.

[0038] In practical applications, the system can match scene identification sub-queries sequentially from shallow to deep according to matching depth. For example, if the matching depth in this embodiment is scene name, keywords, and description text from shallow to deep, then scene name matching can be performed first. When performing scene name matching, the system can match the scene identification sub-queries at the scene name depth (i.e., the scene name extracted from the scene identification query information) with the "scene name" field in the preset structured scene knowledge base. If the match is successful, all information corresponding to the matched business scene name (including but not limited to scene name, description, keywords, and responsible person information) can be recorded. If the match fails, the system can continue to the next matching depth, i.e., keyword matching. When performing keyword matching, the system can match the scene identification sub-queries corresponding to the keyword depth (i.e., the keywords extracted from the scene identification query information) with the "keyword" field in the preset structured scene knowledge base. If the match is successful, the matched business scene keywords can be recorded. If the match fails, the system can continue to the next matching depth, i.e., description text. When performing descriptive text matching, the system can match the scene identification sub-query information corresponding to this depth of descriptive text (i.e., the descriptive text extracted from the scene identification query information) with the "description" field in the preset structured scene knowledge base. If the match is successful, the matched descriptive text can be recorded. Finally, the system can determine the matched scene name, keywords, and descriptive text as the candidate business scene description information corresponding to each matching depth.

[0039] Step S40: Determine the scene recognition result corresponding to the multimodal query information based on the description information of each candidate business scenario.

[0040] It should be understood that the above-mentioned scene identification results can be a set of information representing the business scenario to which the user input (such as a question, query, etc.) belongs and its related information. In this embodiment, the scene identification results may include scene identification information and other information associated with the business scenario, such as the scenario's confidence score, answers to common questions in the scenario, etc. This embodiment does not impose any limitations on this.

[0041] In practical applications, the system can collect all candidate business scenario description information from three matching depths (including scenario name matching, keyword matching, and description matching). Since the same scenario may be matched multiple times at different matching depths (e.g., scenario A matches both keyword matching and description matching), it is necessary to merge and deduplicate all candidate business scenario description information. After merging and deduplication, the system compares the keywords in the results with the scenario name and description to determine if the keywords and description belong to the business scenario represented by the scenario name. If they do, the system identifies the business scenario as the one entered by the user and outputs the business scenario and its related description information as the corresponding scenario recognition result. Furthermore, the system can configure the weights corresponding to each matching depth based on the accuracy and reliability of different matching depths (e.g., scenario name matching > keyword matching > description matching), and use a weighted fusion algorithm to comprehensively evaluate the candidate results from different matching depths, calculate the comprehensive confidence score for each candidate scenario, and then select the scenario with the highest comprehensive confidence score as the final recognition result.

[0042] In the specific implementation, refer to Figure 3 , Figure 3 This is a schematic diagram of the intelligent customer service processing flow in the scenario recognition method of this application. For example... Figure 3 As shown, after receiving a user's question, the system first attempts to answer it directly using the question-and-answer module in the intelligent customer service system, based on natural language processing technology and a general knowledge base. Simultaneously, the scenario support module also receives user questions and calls the scenario recognition unit to perform in-depth analysis, identifying the specific business scenario or sub-scenario to which the question belongs. If the business scenario is successfully identified, the system outputs a scenario recognition result containing scenario information, responsible person information, and solution templates. Subsequently, the scenario support module can retrieve scenario-specific solutions, operation guides, or frequently asked questions based on the identified scenario, providing more precise assistance to the user. Furthermore, when it is determined that human intervention is required, the system can quickly locate the specific person in charge (such as maintenance personnel of a specific data development platform, technical experts for a particular business scenario, etc.) who can solve the problem based on the responsible person information contained in the scenario recognition result output by the scenario support module. Based on the user question, context information, and scenario recognition result, the system generates a work order or directly pushes it to the corresponding responsible person via instant messaging tools, achieving efficient human support and improving problem-solving efficiency.

[0043] Furthermore, the scene identification sub-query information includes the name of the scene to be queried, the keywords of the scene to be queried, and the description text of the scene to be queried; the business scene description information corresponding to each matching depth includes the business scene name, the business scene keywords, and the business scene description text; It should be noted that the aforementioned query scenario name can be the name of the business scenario or sub-scenario to which the user's problem belongs, as used in the scenario identification query information. For example, for the user problem "encountering permission issues during application development," "application development" is the query scenario name, which can be used as input for the first-level matching (scenario name matching).

[0044] It should be understood that the keywords for the query scenario mentioned above can be keywords in the scenario identification query information used to represent the core demands of the user's problem. For example, for the user problem "Table creation failed, insufficient permissions prompted", "table creation failed" and "insufficient permissions" are the keywords for the query scenario, which can be used as input for the second-level matching (keyword matching).

[0045] Understandably, the above-mentioned scenario description text can be natural language text used to comprehensively describe the user's problem in the scenario recognition query information, and it can be used as input for the third-level matching (description text matching).

[0046] Accordingly, the aforementioned business scenario names can be standard name fields in a pre-defined structured scenario knowledge base used to identify various business scenarios or sub-scenarios, such as "application development," "data export," and "task scheduling" defined in the knowledge base. The aforementioned business scenario keywords can be key term fields in the pre-defined structured scenario knowledge base that are strongly related to various business scenarios or sub-scenarios. For example, the business scenario keywords corresponding to the "application development" scenario in the knowledge base could include "table creation," "partitioning," "permissions," and "import failure." The aforementioned business scenario description text can be natural language description fields in the pre-defined structured scenario knowledge base used to detail the functions, scope, and common problems of each business scenario or sub-scenarios.

[0047] Step S30 includes: Step S31: Perform a first-level match between the name of the scenario to be queried and the name of the business scenario in the preset structured scenario knowledge base to determine the candidate business scenario name that matches the name of the scenario to be queried.

[0048] It should be noted that the aforementioned candidate business scenario names can be business scenario names in a preset structured scenario knowledge base that match the name of the scenario to be queried. In this embodiment, the candidate business scenario names can be scenario names in the preset structured scenario knowledge base that are the same as, have similar semantics, or have similar expressions to the name of the scenario to be queried. For example, for the scenario name to be queried, "application development," the system can search all business scenario name fields in the knowledge base to find records that are completely consistent with "application development," and can also search for semantically similar expressions such as "software development."

[0049] In practical applications, the system can use precise string matching or a fuzzy matching algorithm based on edit distance (LevenshteinDistance) to perform a first-level match between the query scenario name and the business scenario names in a pre-defined structured scenario knowledge base. This is to accommodate potential spelling errors or non-standard expressions in user input. For example, if the user inputs "application development" as the query scenario name, the system can retrieve data entities in the knowledge base whose "scenario name" field is "application development". If the match is successful, the matched scenario name can be identified as a candidate business scenario name, and all information of the candidate business scenario name (including but not limited to scenario name, description, keywords, and responsible person information) can be recorded. If the match fails, the current matching result can be marked as "no result," and the system can proceed to the next matching depth.

[0050] Step S32: Perform a second-level match between the query scenario keyword and the business scenario keyword to determine candidate business scenario keywords that match the query scenario keyword.

[0051] It should be noted that the aforementioned candidate business scenario keywords can be keywords in a preset structured scenario knowledge base that match the keyword of the scenario to be queried. In this embodiment, the system can calculate the similarity between the keyword of the scenario to be queried and all business scenario keywords in the preset structured scenario knowledge base, and determine the keywords in the knowledge base whose similarity to the keyword of the scenario to be queried exceeds a set threshold as candidate business scenario keywords.

[0052] In practical applications, if the first-level matching fails to obtain a valid business scenario name, it can proceed to the second-level matching. During the second-level matching, the system can match the query scenario keywords corresponding to the scenario keyword depth with all "keyword" fields in the preset structured scenario knowledge base, calculate the semantic similarity between these keywords and the query scenario keywords, and identify keywords with semantic similarity exceeding a set threshold as candidate business scenario keywords.

[0053] Step S33: Perform a third-level match between the query scenario description text and the business scenario description text to determine candidate business scenario description texts that match the query scenario description text.

[0054] It should be noted that the aforementioned candidate business scenario description text can be description text in a preset structured scenario knowledge base that matches the description text of the scenario to be queried. In this embodiment, the system can calculate the similarity between the description text of the scenario to be queried and all business scenario description texts in the knowledge base, and determine the description texts in the knowledge base whose similarity to the description text of the scenario to be queried exceeds a set threshold as candidate business scenario description texts.

[0055] In practical applications, if the second-level matching fails to obtain valid business scenario keywords, it can proceed to the third-level matching. During the third-level matching, the system matches the query scenario description text corresponding to the scenario description text depth with all "description" fields in a pre-defined structured scenario knowledge base. In this embodiment, the system can utilize a pre-trained large language model to convert the query scenario description text and the business scenario description text in the knowledge base into high-dimensional semantic vectors, and determine their semantic similarity by calculating the cosine similarity between the two vectors. Then, description texts with a similarity higher than a set threshold can be identified as candidate business scenario description texts.

[0056] Step S40 includes: Step S41: Determine the scene recognition result corresponding to the multimodal query information based on the candidate business scene name, the candidate business scene keywords, and the candidate business scene description text.

[0057] In this embodiment, after completing the first, second and third level matching, the system can obtain candidate matching results at all levels, including candidate business scenario names, candidate business scenario keywords and candidate business scenario description text. Then, the system can comprehensively determine the final scenario recognition result based on this information.

[0058] Further, step S41 includes: Step S411: Determine the candidate business scenario represented by the candidate business scenario name.

[0059] It is understood that the aforementioned candidate business scenarios can be the business scenarios represented by the candidate business scenario name. In this embodiment, the candidate business scenario needs to be matched with at least one depth, which points to a scenario data object with complete attribute definitions and relationships in the preset structured scenario knowledge base. Therefore, after determining the candidate business scenario name, the system can directly query its associated business scenarios in the preset structured scenario knowledge base to determine the candidate business scenario.

[0060] Step S412: Determine whether the candidate business scenario keywords and the candidate business scenario description text belong to the candidate business scenario.

[0061] It should be understood that after determining the candidate business scenario represented by the candidate business scenario name, the system can extract all business scenario keywords and business scenario description text associated with the candidate business scenario from the preset structured scenario knowledge base, and match the candidate business scenario keywords with all business scenario keywords associated with the candidate business scenario, and at the same time match the candidate business scenario description text with all business scenario description texts associated with the candidate business scenario, in order to determine whether the candidate business scenario keywords and candidate business scenario description text belong to the candidate business scenario.

[0062] Step S413: If it belongs to the candidate business scenario, generate the scene recognition result corresponding to the multimodal query information based on the candidate business scenario.

[0063] Understandably, if there are keywords in the business scenario keywords associated with the candidate business scenario that match the candidate business scenario keywords, and there are description texts in the business scenario description text associated with the candidate business scenario that match the candidate business scenario description text, then it is determined that the candidate business scenario keywords and the candidate business scenario description text belong to the candidate business scenario. At this time, the system can use the candidate business scenario as the business scenario to which the user input belongs, and generate the final scene recognition result based on the candidate business scenario and the corresponding scene description information.

[0064] Furthermore, after step S412, the method further includes: if it does not belong to the candidate business scenario, then detecting whether there is a sub-business scenario in the candidate business scenario; if it exists, then determining whether there is a target sub-business scenario in the sub-business scenario that matches the candidate business scenario keyword and the candidate business scenario description text; if it exists, then generating a scene recognition result corresponding to the multimodal query information based on the target sub-business scenario.

[0065] It should be understood that the aforementioned sub-business scenarios can be lower-level scenario entities belonging to candidate business scenarios in a pre-defined structured scenario knowledge base. In practical applications, such as... Figure 2 As shown, in the JSON data structure, sub-business scenarios can be defined and stored through the fields of "whether to include sub-scenarios" and "sub-scenarios". Each sub-business scenario can also contain attribute information such as "sub-scenarios name", "description" and "keywords".

[0066] It should also be understood that the aforementioned target sub-business scenario can be the specific sub-scenario entity that best matches all sub-business scenarios of the candidate business scenario with the user input (such as candidate business scenario keywords and candidate business scenario description text). In this embodiment, when the candidate business scenario keywords and candidate business scenario description text do not belong to the candidate business scenario, the system can detect whether the candidate business scenario contains sub-business scenarios in the preset structured scenario knowledge base. Specifically, the system can detect the "whether it contains sub-scenarios" field in the JSON data structure. If the value of this field is "yes", it means that the candidate business scenario has sub-business scenarios; if the value of this field is "no", it means that the scenario is the smallest unit of granularity and does not contain any sub-business scenarios. When the candidate business scenario has sub-business scenarios, the system can match the candidate business scenario keywords and candidate business scenario description text with the sub-business scenario keywords and sub-business scenario description text of each sub-business scenario, respectively. If a successfully matched sub-business scenario exists, the sub-business scenario can be identified as the target sub-business scenario, and the target sub-business scenario can be identified as the business scenario to which the user input belongs. Then, the final scenario recognition result is generated based on the target sub-business scenario and the corresponding sub-scenario description information.

[0067] In the specific implementation, refer to Figure 4 , Figure 4 This is a schematic diagram illustrating the query information matching process in the scene recognition method of this application. For example... Figure 4As shown, after receiving user input, the system performs multimodal fusion and accuracy verification to generate standardized scene recognition query information. This information can include three levels of content: the name of the scene to be queried, the keywords of the scene to be queried, and the descriptive text of the scene to be queried, for subsequent multi-level matching. Then, the system can begin the first-level matching (scene name matching). At this stage, the name of the scene to be queried is matched with the business scene names in the preset structured scene knowledge base. If the match is successful, the matching result is recorded, and the matched candidate scene names are passed to the matching result verification step. If the match fails, the system proceeds to scene keyword matching. During scene keyword matching, the system matches the keywords of the scene to be queried with the business scene keywords in the preset structured scene knowledge base. If the match is successful, the matching result (which may include multiple candidate scenes) is recorded, and the matched candidate scene keywords are passed to the matching result verification step. If none of the scenarios reach the threshold, the system proceeds to scene description matching. During scene description matching, the system matches the descriptive text of the scene to be queried with the business scene description text in the preset structured scene knowledge base. If the match is successful, the matching result is recorded. Finally, the system can compare the keywords and scene / sub-scene names in the matching results with the descriptions in the rules. If it finds that the keywords and description text in the matching results do not belong to the scene represented by the scene name, the scene name matching is performed again; if they do belong, the final matching result is output.

[0068] This embodiment provides a scene recognition method. The method discloses generating scene recognition query information based on multimodal query information input by the user; extracting information from the scene recognition query information according to different matching depths to obtain scene recognition sub-query information corresponding to each matching depth; matching each scene recognition sub-query information with business scenario description information corresponding to each matching depth in a preset structured scene knowledge base to obtain candidate business scenario description information corresponding to each matching depth; and determining the scene recognition result corresponding to the multimodal query information based on each candidate business scenario description information. Because this embodiment can match the query information corresponding to each matching depth with business scenario description information of different matching depths in a structured scene knowledge base to obtain candidate business scenario description information corresponding to each matching depth, and determine the scene recognition result based on these candidate business scenario description information, it solves the technical problem of existing scene recognition methods that rely on single keyword matching and are prone to misjudgment of scenes.

[0069] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 5 , Figure 5 This is a flowchart illustrating the second embodiment of the scene recognition method of this application.

[0070] In this embodiment, after step S30, the method further includes steps S31 to S34: Step S31: In the event of a matching failure, obtain the sequence of user actions performed on the business platform within a preset time window.

[0071] It should be noted that the aforementioned preset time window can be a time range used to trace user operation behavior, such as "today", "past 1 hour", "past 30 minutes", etc., and this embodiment does not limit it. In this embodiment, the preset time window can be flexibly configured according to business needs. For example, for operation and maintenance operations with high real-time requirements, a shorter time window (such as the past 15 minutes) can be set; for development-related issues, a longer time window (such as today) can be set.

[0072] Understandably, the aforementioned business platform can be the target system for users to perform data development, operation and maintenance deployment, and other operations. In practical applications, user actions on the business platform can be fully recorded by the system, forming user journey data, i.e., user action behavior data.

[0073] It is also understood that the aforementioned sequence of operational behaviors can be a collection of records of a series of operations performed by a user on a business platform in chronological order within a preset time window, which can reflect the user's activity trajectory and behavioral patterns within a specific time period. In this embodiment, the sequence of operational behaviors may include information such as user identifier, operation time, operation type, operation object, and module to which it belongs.

[0074] In practical applications, if all matching depths fail to obtain valid candidate business scenario description information (i.e., no results are found for scenario name matching, keyword matching, and description matching), the scenario identification matching can be deemed a failure, and a downgraded matching strategy can be triggered. During downgraded matching, the system first queries the business platform's operation log system based on the current user's unique identifier (such as user ID) to obtain all the user's operation records within a preset time window, and then integrates these records into a sequence of the user's operational behaviors.

[0075] Step S32: Determine the module identifier and operation timestamp corresponding to the business function module operated by the user based on the operation behavior sequence.

[0076] It should be noted that a business platform can contain multiple business function modules, such as a "task scheduling module" and a "permission management module". These business function modules can be independent components of the business platform divided according to their functions. Each module can be responsible for a specific business function. For example, the "task scheduling module" is responsible for the configuration and monitoring of scheduled tasks.

[0077] It should be understood that the module identifier mentioned above can be a code or name used to uniquely identify each business function module in the business platform. The operation timestamp mentioned above can be the specific time point when a user performs an operation, which is usually recorded in a standard time format.

[0078] In practical applications, after obtaining the user's operation sequence, the system can parse and statistically analyze the sequence to extract key information related to business function modules. Specifically, the system can traverse each operation record in the operation sequence to extract the module identifier corresponding to the business function module from each record, and record the operation timestamp corresponding to each operation for subsequent time series analysis and recent operation determination.

[0079] Step S33: Obtain the user's business operation behavior data based on the module identifier and the operation timestamp.

[0080] It should be noted that the aforementioned business operation behavior data can be a structured information set used to characterize user operation behavior, such as a list of business function modules accessed by the user within a preset time window, the access frequency of each module, the time of the most recent access, and other statistical information. This information can serve as the basis for subsequent inference of degradation scenarios.

[0081] Step S34: Based on the business operation behavior data, perform downgraded matching on the scene recognition query information to obtain the scene recognition result corresponding to the multimodal query information.

[0082] In practical applications, since the problems encountered by users are often directly related to the operations they have just performed, after obtaining business operation behavior data, the system can prioritize selecting the business function module that the user last operated within a preset time window as the degradation scenario, determine the degradation scenario as the scenario to which the user input belongs, and generate the scenario recognition result corresponding to the multimodal query information based on the scenario and its description information.

[0083] This embodiment discloses a method for obtaining a sequence of user actions performed on the business platform within a preset time window when matching fails; determining the module identifier and operation timestamp corresponding to the user's operated business function module based on the action sequence; obtaining the user's business operation behavior data based on the module identifier and operation timestamp; and performing downgraded matching on scene recognition query information based on the business operation behavior data to obtain scene recognition results corresponding to multimodal query information. Since this embodiment can perform downgraded matching on scene recognition query information based on the user's business operation behavior data on the business platform even when scene description information matching fails, it ensures that the system can output scene recognition results under any circumstances, achieving full scene coverage and improving user experience.

[0084] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 6 , Figure 6 This is a flowchart illustrating the scene recognition method of this application in Embodiment 3.

[0085] In this embodiment, step S10 further includes steps S11 to S14: Step S11: Extract image query information and text query information from the multimodal query information input by the user.

[0086] It is understood that the aforementioned image query information can be an image component of the multimodal query information input by the user, such as screenshots uploaded by the user, images of error pop-ups, screenshots of the operation interface, etc. This embodiment does not impose any limitations on this. In this embodiment, the image query information can include rich content such as text, icons, interface elements, and user-added markers to intuitively reflect the problem status encountered by the user.

[0087] It is also understood that the aforementioned text query information can be a text component of the multimodal query information entered by the user, that is, a natural language description entered by the user.

[0088] Step S12: Perform visual content parsing on the image query information to extract text elements and user tag information from the image query information.

[0089] It should be understood that the aforementioned text elements can be text content identified and extracted from image query information, such as error message text extracted from user-uploaded error screenshots, interface button text, pop-up titles, etc. The aforementioned user tagging information can be tagging information manually added by the user in the image query information, which can include, but is not limited to, visual tags such as red circles, arrows, underlines, highlights, and selection boxes.

[0090] Step S13: Merge the text query information, the text element, and the user tag information to obtain fused query information.

[0091] It should be noted that the aforementioned fused query information can be a comprehensive query information generated by fusing text query information with text elements extracted from images and user-labeled information. It can integrate all valid content from the user's multimodal input, including the question description directly entered by the user, as well as the semantic description transformed from text content and label information in the image, thereby providing a comprehensive and rich information foundation for subsequent scene recognition.

[0092] Step S14: Perform accuracy verification on the fused query information and generate scenario recognition query information based on the verification result.

[0093] In practical applications, after the system receives the multi-modal query information input by the user, it can perform optical character recognition (OCR) processing on the images in the multi-modal query information, identify all text regions in the images, extract the text content therein, and obtain text elements. At the same time, the system can identify and extract marker elements such as red circles, arrows, underlines, highlights, etc. in the images through object detection and image segmentation technologies to obtain user marker information. Then, the system can fuse the text query information, text elements, and user marker information to obtain fused query information, and perform accuracy verification on the fused query information to ensure the accuracy and usability of the information. Finally, the verified fused query information is output as scenario recognition query information for subsequent scenario recognition based on the scenario recognition query information.

[0094] Further, step S14 includes: performing semantic analysis on the fused query information to determine abnormal query information in the fused query information, where the abnormal query information is information with ambiguous or incorrect expressions; detecting whether the abnormal query information belongs to scenario recognition interference information based on a preset interference word filtering rule; if not, correcting the abnormal query information and generating scenario recognition query information based on the corrected abnormal query information.

[0095] It should be noted that the above abnormal query information can be an information segment with expression problems in the fused query information, such as typos (e.g., "create table" miswritten as "create table name"), grammar errors (e.g., disordered word order, incomplete components), semantic ambiguity (e.g., the same word can be interpreted as multiple meanings in different contexts), logical contradictions (e.g., the descriptions before and after conflict with each other), etc. This embodiment does not limit this.

[0096] It should be noted that the aforementioned preset interference word filtering rules can be a pre-defined set of rules used to determine whether abnormal query information belongs to scene recognition interference information. In this embodiment, the preset interference word filtering rules can be constructed based on in-depth analysis of business scenarios and user expression habits, which can include multiple dimensions of judgment conditions, such as a stop word list, a colloquial expression mapping table, and an interference pattern library. The stop word list can contain interjections and filler words that have no practical meaning for scene recognition. The stop word list can identify stop words in the text so that stop words in the query information can be removed subsequently. The colloquial expression mapping table stores the mapping relationship between colloquial expressions and standardized terms, allowing the system to map colloquial expressions in the query information to standardized terms based on the colloquial expression mapping table. The interference pattern library defines common interference expression patterns, such as repeated input and irrelevant symbol stacking. The interference pattern library can identify information in the query information that may interfere with scene recognition.

[0097] It should also be noted that the aforementioned scene recognition interference information may be information in abnormal query information that is determined to be of no value to scene recognition or may have a negative impact, such as interjections, catchphrases, and irrelevant social expressions in user input. This embodiment does not impose any restrictions on this.

[0098] It is understood that the above-mentioned corrected abnormal query information can be obtained by semantically correcting abnormal query information that does not belong to scene recognition interference information. In this embodiment, the correction process may include operations such as typo correction, grammatical normalization, ambiguity elimination, and terminology standardization.

[0099] In practical applications, the system first performs deep semantic analysis on the fused query information (such as spell checking, grammar analysis, and ambiguity detection) to identify abnormal information fragments. It then calls preset interference word filtering rules to detect each identified abnormal query. If an abnormal information fragment matches any rule in the stop word list, scene-independent vocabulary, or interference pattern library, it is determined that the fragment belongs to scene recognition interference information. If the abnormal information fragment does not match any interference rule, it is determined that the abnormal query information does not belong to scene recognition interference information, and correction processing is required. After performing correction processing on the abnormal query information that does not belong to scene recognition interference information, the system can generate corrected abnormal query information. Subsequently, the system can fuse the unidentified parts of the fused query information with the corrected abnormal query information to generate the final scene recognition query information.

[0100] In the specific implementation, refer to Figure 7 , Figure 7 This is a flowchart illustrating the overall scene recognition process in the scene recognition method of this application. Figure 7As shown, in the system backend, administrators can maintain information on all functional modules of the business platform through the platform module information configuration module, construct structured scene data, and store this data in a preset structured scene knowledge base. After receiving the natural language question description entered by the user through a dialog box and the image file uploaded by the user, the system can perform accuracy verification on the user-input text and image information, generating standardized scene recognition query information. Then, the system can perform multi-level matching between the scene recognition query information and the structured scene data in the preset structured scene knowledge base, including scene name matching, keyword matching, and description text matching. The system also verifies the consistency of the candidate results obtained at each level of matching. If the verification passes, the scene recognition result is output; if the verification fails, the matching process is re-run.

[0101] This embodiment discloses the extraction of image query information and text query information from multimodal query information input by the user; visual content parsing of the image query information to extract text elements and user tag information; fusion of text query information, text elements, and user tag information to obtain fused query information; accuracy verification of the fused query information; and generation of scene recognition query information based on the verification results. This embodiment, by fusing text query information and text elements and user tag information from image query information, and verifying the accuracy of the fused query information, enables the generation of scene recognition query information with high completeness and high accuracy, thereby improving the accuracy of subsequent scene recognition.

[0102] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the scene recognition method of this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0103] This application also provides a scene recognition device; please refer to... Figure 8 The scene recognition device includes: Information generation module 10 is used to generate scene recognition query information based on multimodal query information input by the user; Information extraction module 20 is used to extract information from the scene recognition query information according to different matching depths, and obtain scene recognition sub-query information corresponding to each matching depth; Information matching module 30 is used to match the sub-query information of each scenario identification using the business scenario description information corresponding to each matching depth in the preset structured scenario knowledge base, so as to obtain the candidate business scenario description information corresponding to each matching depth. The scene recognition module 40 is used to determine the scene recognition result corresponding to the multimodal query information based on the description information of each candidate business scene.

[0104] The scene recognition device provided in this application, employing the scene recognition method in the above embodiments, can solve the technical problem that existing scene recognition relies on single keyword matching, which easily leads to misjudgment of scenes. Compared with the prior art, the beneficial effects of the scene recognition device provided in this application are the same as those of the scene recognition method provided in the above embodiments, and other technical features in the scene recognition device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0105] This application provides a scene recognition device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the scene recognition method in the above embodiment 1.

[0106] The following is for reference. Figure 9 The diagram illustrates a structural schematic of a scene recognition device suitable for implementing embodiments of this application. The scene recognition device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 9 The scene recognition device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0107] like Figure 9As shown, the scene recognition device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the scene recognition device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the scene recognition device to communicate wirelessly or wiredly with other devices to exchange data. While the figures show scene recognition devices with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0108] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0109] The scene recognition device provided in this application, employing the scene recognition method in the above embodiments, can solve the technical problem of scene recognition. Compared with the prior art, the beneficial effects of the scene recognition device provided in this application are the same as those of the scene recognition method provided in the above embodiments, and other technical features in this scene recognition device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0110] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0111] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0112] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the scene recognition method in the above embodiments.

[0113] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination thereof.

[0114] The aforementioned computer-readable storage medium may be included in the scene recognition device; or it may exist independently and not be assembled into the scene recognition device.

[0115] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the scene recognition device, the scene recognition device: generates scene recognition query information based on multimodal query information input by the user; extracts information from the scene recognition query information according to different matching depths to obtain scene recognition sub-query information corresponding to each matching depth; matches each scene recognition sub-query information with business scenario description information corresponding to each matching depth in a preset structured scene knowledge base to obtain candidate business scenario description information corresponding to each matching depth; and determines the scene recognition result corresponding to the multimodal query information based on each candidate business scenario description information.

[0116] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0118] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0119] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described scene recognition method. This solves the technical problem that existing scene recognition methods rely on single keyword matching, which easily leads to misjudgments of scenes. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the scene recognition method provided in the above embodiments, and will not be repeated here.

[0120] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the scene recognition method described above.

[0121] The computer program product provided in this application can solve the technical problem that existing scene recognition methods rely on single keyword matching, which is prone to misjudgment of scenes. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the scene recognition methods provided in the above embodiments, and will not be repeated here.

[0122] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A scene recognition method, characterized in that, The method includes: Generate scene recognition query information based on multimodal query information input by the user; The scene recognition query information is extracted according to different matching depths to obtain scene recognition sub-query information corresponding to each matching depth. By matching the business scenario description information corresponding to each matching depth in the preset structured scenario knowledge base with the sub-query information of each scenario identification, candidate business scenario description information corresponding to each matching depth is obtained. The scene recognition result corresponding to the multimodal query information is determined based on the description information of each candidate business scenario.

2. The method as described in claim 1, characterized in that, The scene identification sub-query information includes the name of the scene to be queried, the keywords of the scene to be queried, and the description text of the scene to be queried; the business scene description information corresponding to each matching depth includes the business scene name, the business scene keywords, and the business scene description text; The step of matching each scenario identification sub-query information with the business scenario description information corresponding to each matching depth in a preset structured scenario knowledge base to obtain candidate business scenario description information corresponding to each matching depth includes: The query scenario name is matched with the business scenario name in the preset structured scenario knowledge base at the first level to determine the candidate business scenario name that matches the query scenario name; The query scenario keywords are matched with the business scenario keywords in a second-level match to determine candidate business scenario keywords that match the query scenario keywords. The third-level matching is performed between the query scenario description text and the business scenario description text to determine the candidate business scenario description text that matches the query scenario description text. The step of determining the scene recognition result corresponding to the multimodal query information based on the description information of each candidate business scenario includes: The scene recognition result corresponding to the multimodal query information is determined based on the candidate business scene name, the candidate business scene keywords, and the candidate business scene description text.

3. The method as described in claim 2, characterized in that, The step of determining the scene recognition result corresponding to the multimodal query information based on the candidate business scene name, the candidate business scene keywords, and the candidate business scene description text includes: Determine the candidate business scenarios represented by the candidate business scenario names; Determine whether the candidate business scenario keywords and the candidate business scenario description text belong to the candidate business scenario; If it belongs to the candidate business scenario, then the scene recognition result corresponding to the multimodal query information is generated based on the candidate business scenario.

4. The method as described in claim 3, characterized in that, After the step of determining whether the candidate business scenario keywords and the candidate business scenario description text belong to the candidate business scenario, the method further includes: If it does not belong to the category, then check whether the candidate business scenario has a sub-business scenario; If it exists, then determine whether there is a target sub-business scenario in the sub-business scenario that matches the candidate business scenario keywords and the candidate business scenario description text; If it exists, then generate the scene recognition result corresponding to the multimodal query information based on the target sub-business scenario.

5. The method according to any one of claims 1 to 4, characterized in that, After the step of matching the sub-query information of each scenario identification using the business scenario description information corresponding to each matching depth in the preset structured scenario knowledge base, the method further includes: In the event of a failed match, obtain the sequence of user actions performed on the business platform within a preset time window; The module identifier and operation timestamp corresponding to the business function module operated by the user are determined based on the operation behavior sequence. The user's business operation behavior data is obtained based on the module identifier and the operation timestamp; Based on the business operation behavior data, the scene recognition query information is downgraded and matched to obtain the scene recognition result corresponding to the multimodal query information.

6. The method according to any one of claims 1 to 4, characterized in that, The step of generating scene recognition query information based on multimodal query information input by the user includes: Extract image query information and text query information from multimodal query information input by the user; Visual content parsing is performed on the image query information to extract text elements and user tag information from the image query information; The text query information, the text elements, and the user tag information are fused to obtain fused query information; The accuracy of the fused query information is verified, and scene recognition query information is generated based on the verification results.

7. The method as described in claim 6, characterized in that, The step of verifying the accuracy of the fused query information and generating scene recognition query information based on the verification result includes: Semantic analysis is performed on the fused query information to identify abnormal query information in the fused query information, wherein the abnormal query information is information with ambiguous or incorrect expression. Based on preset interference word filtering rules, detect whether the abnormal query information belongs to scene recognition interference information; If it does not belong to the category, the abnormal query information is corrected, and scene recognition query information is generated based on the corrected abnormal query information.

8. A scene recognition device, characterized in that, The device includes: The information generation module is used to generate scene recognition query information based on multimodal query information input by the user; The information extraction module is used to extract information from the scene recognition query information according to different matching depths, and obtain scene recognition sub-query information corresponding to each matching depth; The information matching module is used to match the sub-query information of each scenario identification using the business scenario description information corresponding to each matching depth in the preset structured scenario knowledge base, so as to obtain the candidate business scenario description information corresponding to each matching depth. The scene recognition module is used to determine the scene recognition result corresponding to the multimodal query information based on the description information of each candidate business scene.

9. A scene recognition device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the scene recognition method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the scene recognition method as described in any one of claims 1 to 7.

Images