Scene cognition method and device

Abstract

The embodiment of the invention discloses a scene cognition method and device, and the method comprises the steps: obtaining multi-source and multi-mode data samples of services in different scenes, and extracting semantic segmented words in the samples; fusing the semantic segmented words to obtain a feature vector set; encoding semantic segmented words in the semantic segmented word set to obtain a standard semantic segmented word classification tree code; on the basis of the standard semantic word segmentation classification tree code, determining a code mapped by multi-source and multi-modal data of a service in a to-be-recognized target scene; and inputting the mapped code into a model of the configured code calculation rule, and mapping to obtain a final code value so as to determine a cognitive result of the set scene based on the final code value. The code-driven scene data comprehensive analysis is realized, so that people can better obtain more insights from massive and complex multi-source and multi-mode scene data, and various scene services are enabled.

Description

Technical Field

[0001] The present invention relates to the field of intelligent recognition technology, and in particular to a scene recognition method and device. Background Art

[0002] With the rapid development of intelligent technology, especially the rapid application of technologies represented by generative AI, the deepening integration of multi-source and multi-modal data and intelligence has been promoted, bringing new changes and challenges to data infrastructure. However, due to the wide variety of data, wide sources, and diverse and inconsistent protocols, there are problems such as data clutter, low quality, chaotic data architecture, and inconsistent storage. Currently, only the human brain can accurately understand the content of such data, but it has long been overwhelmed by the flood of big data. Most or at least 80% of the data cannot be processed or understood by the current IT system. How to simplify the increasingly complex system architecture, improve data processing efficiency, reduce development and operation costs, and promote data openness, sharing, and innovative applications has become a core issue of concern. Summary of the invention

[0003] The main purpose of the present invention is to provide a scene recognition method and device to solve the technical problems existing in the related technology.

[0004] In order to achieve the above-mentioned purpose, according to a first aspect of the present invention, a scene recognition method is provided, including obtaining multi-source multi-modal data samples of services in different scenarios, and extracting semantic word segmentations in the samples; fusing the semantic word segmentations to obtain a feature vector set; encoding the semantic word segmentations in the semantic word segmentation set to obtain a standard semantic word segmentation classification tree code; based on the standard semantic word segmentation classification tree code, determining the code mapped to the multi-source multi-modal data of the service in the target scenario to be recognized; inputting the mapped code into a model of the configured coding calculation rule, mapping to obtain a final code value to determine the cognition result of the set scene based on the final code value.

[0005] Optionally, fusing the semantic word segmentations to obtain a feature vector set includes: fusing the semantic word segmentations based on a probability density estimation method; wherein, classifiers with the same structure are trained based on different semantic word segmentations; and the probability density output by the classifier is used as the fused feature vector.

[0006] Optionally, encoding the semantic participles in the semantic participle set includes: obtaining encoding rules configured by a user based on the granularity of the semantic participles, wherein the encoding rules are determined based on different types of code segments, and different types of code segments are composed of one or more encoding nodes; encoding the semantic participles based on the encoding rules.

[0007] Optionally, the different types of code segments include classification code segments, attribute code segments, pipeline code segments, and / or symbol code segments, wherein the classification code segments and attribute code segments are associated with the classification information and attribute information in the classification library and attribute library, respectively; the classification library and attribute library store the classification information and attribute information of physical objects in the scene, and there is an association relationship between the classification library and the attribute library, so that different classifications are associated with different attribute information.

[0008] Optionally, based on the standard semantic word segmentation classification tree encoding, determining the encoding mapped to the multi-source multi-modal data of the service in the target scenario to be recognized includes: performing keyword probability matching based on a hidden Markov model to obtain the mapped encoding.

[0009] Optionally, the mapped code is input into a model of the configured coding calculation rules, and the mapping obtains a final code value to determine the cognitive result of the set scene based on the final code value, including: abstracting variable factors based on the mapped code into variables; inputting the variables into the model of the configured coding calculation rules for processing, wherein the value constraints and relational expressions of the variables are used to calculate the face changing to obtain the final code value; and determining the scene cognition result based on the final code value.

[0010] According to a second aspect of the present invention, a scene recognition device is provided, comprising: a preprocessing unit, used to obtain multi-source multi-modal data samples of services in different scenarios, and extract semantic participles in the samples; fuse the semantic participles to obtain a feature vector set; encode the semantic participles in the semantic participle set to obtain a standard semantic participle classification tree code; a code mapping unit, used to determine the code mapped to the multi-source multi-modal data of the service in the target scenario to be recognized based on the standard semantic participle classification tree code; a scene recognition result determination unit, used to input the mapped code into a model of the configured coding calculation rule, and map the final code value to determine the recognition result of the set scene based on the final code value.

[0011] According to a third aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute any one of the methods described in the first aspect.

[0012] According to a fourth aspect of the present invention, there is provided an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the method described in any one implementation of the first aspect.

[0013] The scene recognition method and device of this embodiment include obtaining multi-source multi-modal data samples of services in different scenarios, and extracting semantic segmentation in the samples; fusing the semantic segmentation to obtain a feature vector set; encoding the semantic segmentation in the semantic segmentation set to obtain a standard semantic segmentation classification tree code; based on the standard semantic segmentation classification tree code, determining the code mapped by the multi-source multi-modal data of the services in the target scenario to be recognized; inputting the mapped code into the model of the configured coding calculation rule, mapping to obtain the final code value to determine the cognitive result of the set scenario based on the final code value. The multi-source multi-modal data is converted and mapped into a scene segmentation matrix semantic cognitive computing system, thereby realizing multi-source (multi-modal) data fusion cognitive computing, getting rid of the large sample and large computing power model required for traditional AI empowerment, as well as the "black box" and the problem of being unable to truly integrate in machine learning, helping traditional industries to achieve digital transformation and upgrading, and quickly implement various small-scene intelligent applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 is a flow chart of a scene recognition method according to an embodiment of the present invention;

[0016] Figure 2 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so as to describe the embodiments of the present invention described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0019] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0020] According to an embodiment of the present invention, a scene recognition method and device are provided, such as Figure 1 As shown, the method includes the following steps 101 to 104:

[0021] Step 101: Acquire multi-source and multi-modal data samples of services in different scenarios, and extract semantic word segmentations from the samples; fuse the semantic word segmentations to obtain a feature vector set.

[0022] In this step, the semantic representation of the multi-source multi-modal data samples of the business under the set scenario conditions, i.e., the semantic word set, is obtained. The multi-source multi-modal scenario data samples include: structured scenario data, semi-structured scenario data, and unstructured scenario data. Here, food safety is taken as an example. In today's information age, the sources of food safety data are relatively wide, and all food and safety-related data that people can access in daily life are included, mainly including: (1) the results of various food safety testing devices; (2) food quality test data from RFID sensors; (3) enterprises and regulatory authorities; (4) mobile Internet, social media, etc. Food safety data covers a variety of forms such as images, videos, signals, charts, text, etc., and the amount of data accumulates over time and becomes larger and larger.

[0023] Here, the method for extracting semantic word segmentation is as follows: first, sort out the data in the existing food safety information system mentioned above to understand the data classification logic, coding rules and main problems in the existing information system; then, sort out the food and its food attribute fields, namely the semantic word segmentation description, including the semantic word segmentation of various fields such as food classification code, time attribute, spatial attribute, operator, product information, sampling information, and test results, and then compile a semantic word segmentation dictionary for specific food classification coding data to standardize the coding of the same field (semantic word segmentation) between systems.

[0024] Before encoding the semantic segmentation of the scene data, the semantic segmentations of various services and different target task data corresponding to the same scene may be merged.

[0025] For example, before encoding the semantic segmentation of the text data in the fever case set, it also includes: merging the semantic segmentation of the fever routine detection data and the fever accompanying symptom information corresponding to the same fever disease. The fever routine detection data includes low fever 37.2℃-38℃, moderate fever 38.1℃-39℃, high fever 39.1℃-41℃, and hyperpyrexia above 41℃; the accompanying symptom information includes but is not limited to cough, runny nose, chills, bleeding, hepatomegaly, splenomegaly, rash, diarrhea, abdominal distension, conjunctival congestion, joint pain, etc. The main symptoms of fever after merging include: 1) fever with chills, which is generally seen in lobar pneumonia, sepsis, acute cholecystitis, acute pyelonephritis, etc.; 2) fever with conjunctival congestion, which is generally seen in measles, epidemic hemorrhagic fever, typhus, etc.; 3) fever with herpes simplex labialis, which often occurs in acute febrile diseases, such as lobar pneumonia, epidemic meningitis, vivax malaria, etc.; 4) fever with lymphadenopathy, which is generally seen in infectious mononucleosis, lymphadenopathy, filariasis, leukemia, etc.; 5) fever with hepatosplenomegaly, which is generally seen in infectious mononucleosis, viral hepatitis, brucellosis, etc.; for another example, fever type with pain location: 1) high fever, side chest pain radiating to the shoulder, chills, cough, rust-colored sputum, which may be lobar pneumonia; 2) low fever, sharp chest pain, dry cough, pleural effusion, sputum, hemoptysis , night sweats, fatigue, and dyspnea may be tuberculous pleurisy; 3) low fever or moderately high fever, swollen lymph nodes, diarrhea, abdominal pain, fatigue, night sweats, lack of appetite, ascites, and abdominal masses may indicate tuberculous peritonitis; 4) high fever, sore throat, swollen anterior cervical lymph nodes (located in front of the sternocleidomastoid muscle), swollen mandibular angle lymph nodes, significantly swollen tonsils, chills, loss of appetite, difficulty eating and speaking, and sore throat radiating to the ears, It may indicate acute suppurative tonsillitis, etc. For example, fever type with pain location and main symptom characteristics: 1) Fever, sore throat, swollen lymph nodes, accompanied by tenderness, sneezing, nasal congestion, runny nose, cough, sputum, dry throat, itchy throat, indicating upper respiratory tract infection, i.e. cold; 2) Acute onset, high fever, sore throat, chills, accompanied by dyspnea, headache, loss of appetite, thirst, vomiting, indicating acute edematous pharyngitis, etc. All of the above can be used as information on fever-associated symptoms.

[0026] As an optional implementation method of this embodiment, fusing the semantic word segmentations to obtain a feature vector set includes: fusing the semantic word segmentations based on a probability density estimation method; wherein, classifiers with the same structure are trained based on different semantic word segmentations; and the probability density output by the classifier is used as the fused feature vector.

[0027] Step 102: Encode the semantic segmentation words in the semantic segmentation set to obtain a standard semantic segmentation classification tree code.

[0028] In this optional implementation, a standard scene data semantic segmentation classification tree is established with scene data fusion and sharing as the guide. Here, the probability density estimation method is mainly used to perform semantic segmentation fusion on the preprocessed scene multi-source multi-modal data and normalize it to form a multi-source multi-modal data feature set, i.e., a semantic segmentation set, under the set scene. Then, the multi-source multi-modal data is classified, clustered, and integrated according to the feature information, i.e., semantic segmentation, to generate a feature vector, including: training a set of classifiers with training samples of each feature, and using the probability density value output by the classifier as the feature vector after feature fusion.

[0029] Exemplarily, when the features to be fused are: A = [a1, a2, a3, a4.....an] and B = [b1, b2, b3, b4....bm], the feature fusion process includes: using the A feature of the feature training sample to train n classifiers f: f1, f2....fn; calculating xi = fi(ai), i = 1.2.3....n. Using the B feature of the feature training sample to train m classifiers g: g1, g2....gn, calculating yj = gj(bj), j = 1.2.3....m. Classifier f and classifier g are the same model, for example, they can be Bayes or SVM, and the outputs of classifier f and classifier g are the probabilities of a certain category, instead of setting the output of the classifier as the category, and the obtained probability density value is used as the feature vector after feature fusion: C = [x1.x2.x3.....xn.y1.y2.y3......ym].

[0030] Here, take food safety as an example. Considering that the current diverse food safety classification standards are difficult to unify in the short term, the present invention constructs a food safety semantic segmentation classification tree based on the principle of simple structure, and strives to find a food classification method that meets food safety governance, nutrition and health management, and has wide applicability and strong extensibility on the basis of respecting the food safety classification of all parties - establishing a standard food safety semantic segmentation classification tree. The classification tree content includes: semantic segmentations related to food safety, such as environment, nutrition, and health. Guided by food, the semantic segmentation classification coding of food safety data mainly includes: food semantic segmentation classification code, foodborne disease semantic segmentation classification code, biological pollutant semantic segmentation classification code (bacteria, parasites, viruses, toxins), chemical pollutant semantic segmentation classification code, pesticide semantic segmentation classification code, natural toxin semantic segmentation classification code, metal semantic segmentation classification code, etc.

[0031] As an optional implementation method of the present embodiment, encoding the semantic participles in the semantic participle set includes: obtaining encoding rules configured by the user based on the granularity of the semantic participles, wherein the encoding rules are determined based on different types of code segments, and different types of code segments are composed of one or more encoding nodes; encoding the semantic participles based on the encoding rules.

[0032] In this optional implementation, the granularity of the scene data semantic segmentation is determined and encoding and decoding rules are established. That is, semantic segmentation encoding is performed on the data in the scene data set. Different encodings represent different scene services or target task types, etc.

[0033] Data granularity is the smallest unit usually used to represent the composition of a data set, which involves the level and level of detail of classification coding. The more classification levels there are, the smaller the granularity, which may cause the amount of data processing in the later stage to grow exponentially. Therefore, it is particularly important to determine the appropriate granularity of semantic word segmentation for scene data in combination with actual analysis needs and operational convenience. At the same time, each type of information in the code segment, namely the semantic word segmentation, is divided into two attributes for storage, namely "character" and "meaning", representing the characters and related meanings that users can enter; dynamic rule trees are used to express rule logic knowledge, and rule templates are used to express common coding methods, so as to separate the logic content of the coding rules from the code segments, and achieve flexible coding and visual design effects.

[0034] Here, we take food safety as an example. By analyzing the classification principles of various food classification systems at home and abroad, we know that the following principles are basically followed when classifying food: (1) The main function of each classification system is to serve the formulation / revision of limits for pesticide residues, food additives, etc.; (2) Each classification system classifies foods with similar characteristics and the same risks into one category; (3) Each classification system divides foods into plant-derived foods and animal-derived foods; (4) Foods of the same type have the same physical and chemical characteristics or similar agricultural cultivation characteristics; (5) The same type of food includes a narrow range of food subcategories.

[0035] After the data granularity is determined, the semantic segmentation coding rules are established. In terms of encoding and decoding, the EU food classification and coding system FoodEx2 has strong applicability and can be used as an important reference. The system adopts a consistent coding system, and all food-related categories, terms and attribute semantic segmentations are encoded with serial numbers, which are automatically generated and assigned by FoodEx2. Here, there are 5 characters in total, the first character is A, the middle 3 characters are Latin letters and Arabic numerals, and the last character is a Latin letter. The code system is A[A-Z0-9]{3}[AZ]. Theoretically, the system can encode 1213056 entries. In terms of decoding, the food classification name corresponding to the food classification code can be directly searched on the classification tree, such as the code "A026V" represents "fish meat".

[0036] As an optional implementation method of this embodiment, the different types of code segments include classification code segments, attribute code segments, pipeline code segments, and / or symbol code segments, wherein the classification code segments and attribute code segments are associated with the classification information and attribute information in the classification library and attribute library respectively; the classification library and attribute library store the classification information and attribute information of physical objects in the scene, and there is an association relationship between the classification library and the attribute library, so that different classifications are associated with different attribute information.

[0037] In this optional implementation, the type library and the attribute library store the classification and extended attribute information of things in the scene, and are mainly used to define and record various classification methods and extended attributes in the scene. There is an association relationship between the type library and the attribute library to ensure that different types are associated with different attribute information. Therefore, when a scene is created in the future, it can have different extended attributes because of its different types. At the same time, because of the association relationship between the scene and the attribute library, the extended attributes of each scene can be defined separately, thereby facilitating the management of the scene.

[0038] It is precisely because of the correspondence between code segments and type and attribute libraries that automatic encoding of scenes becomes possible. For example, when creating a certain classification code segment of the encoding rule, first define the basic information of the code segment (such as name, description information, code segment length, etc.). Since it is a classified code segment, when further defining the code segment information, it is necessary to select a corresponding classification method in the type library. Then the code segment creates a coding node according to the subtype under the selected classification method (the coding node and the code segment are a many-to-one relationship). It is necessary to establish a one-to-one correspondence between the coding node and the subtype, and the user assigns the corresponding code value. For example, according to the classification of food safety characteristics, it creates several coding nodes based on the subtypes under "Classification by Food Safety Characteristics", including food semantic segmentation classification code, time attribute semantic segmentation, space attribute semantic segmentation, operator semantic segmentation, product information semantic segmentation, sampling information semantic segmentation, and test result semantic segmentation, and the user defines the code values ​​for them respectively, and maintains the corresponding relationship between the subtype and the coding node. In the future, when a certain part code is generated using the coding rule containing this code segment, if it belongs to "self-made parts" in "Classification by Food Safety Characteristics", then since the code value in the coding node corresponding to the subtype "self-made parts" is "1", the code generated by the food classification code, time attribute, space attribute, operator, product information, sampling information, test results, etc. through this code segment is also "1". After selecting the coding rule, the entire code generation process is automatically completed by the code generator in the background. For the establishment of attribute code segments, the process is similar to that of classification code segments, except that the information corresponding to attribute code segments sometimes may not need to be pre-defined code values. When generating codes in the future, the attribute value of the object will appear as a code value under certain constraints. For example, a certain size is used as the code value.

[0039] Step 103: Based on the standard semantic word segmentation classification tree encoding, determine the encoding mapped to the multi-source multi-modal data of the business in the target scenario to be recognized.

[0040] As an optional implementation method of this embodiment, based on the standard semantic word segmentation classification tree encoding, determining the encoding mapped to the multi-source multimodal data of the business in the target scenario to be recognized includes: performing keyword probability matching based on the hidden Markov model to obtain the mapped encoding.

[0041] In this step, the target scene related data system is mapped with the standard scene semantic segmentation classification tree coding. In terms of system data mapping, the main problem to be solved is the mapping, bridging and fusion of the original historical data of the scene with the classification tree of the semantic segmentation of the standard scene data. At present, it is still difficult to complete the data mapping work between systems through fully automated methods such as keyword search. In order to ensure the reliability of data mapping, when other fields carry out this work, the manual verification mapping method is preferred to solve historical problems. In order to solve the problem of large workload and long time consumption of manual mapping, the present invention adopts the "flexible coding docking system" method for mapping, which jumps out of the simple framework of matching keyword strings with the keywords of the scene name and the standard name in the original scene system, and is upgraded to the flexible coding docking system of the "hidden Markov model", which performs keyword probability matching, gives the probability arrangement of each keyword string match, realizes automated intelligent mapping, and finally supplemented by manual verification to improve the accuracy and work efficiency of mapping matching.

[0042] At the same time, the rule type class is used to manage the coding rule type. There is a one-to-many aggregation relationship between coding rules, code segments, and coding nodes. That is, a coding rule can be composed of one or more code segments, and a code segment is composed of one or more coding nodes. The four types of code segments mentioned above are created in the code segment class.

[0043] The code segment and coding node classes establish underlying associations with the type and attribute classes. The master class is used to detect and recover the code. The Identity class serves as an association class and code generator. After a specific scenario is associated with a coding rule, the coding rule can be used to generate code for the scenario.

[0044] Step 104: Input the mapped code into the model of the configured code calculation rule, and map to obtain a final code value to determine the cognitive result of the set scenario based on the final code value.

[0045] In this step, the variable factors that determine the encoding results of the encoded object (scene) (such as the scene classification of the encoded object, etc.) are abstracted into variables as the input information of the encoding, and the restrictions and dependencies between the variable factors are abstracted into the value constraints and relational expressions of the variables. According to these constraints and relational expressions, various values ​​are taken, verified and calculated for the variable values, and then the final code value is calculated based on the values ​​of each variable according to the encoding calculation rules, and finally the scene recognition result is obtained based on the final code value.

[0046] By interpreting and calculating the coding calculation rules, code values ​​of different coding structures and lengths are obtained to achieve scene recognition results. The coding calculation rules are functionally like a program language function body that uses input values ​​to calculate output values, so they are also called coding calculation functions. After the application parses the variables in the variable assignment table, it passes the variable name and variable value together to the coding calculation function as its input for processing by the coding calculation function. The coding calculation function returns different strings that express the coding structure according to different coding objects. The length and structure of the coding represented by the function return value can be configured according to the coding rules. Therefore, a coding calculation function can use process control to return multiple coding structures.

[0047] As an optional implementation method of this embodiment, the mapped code is input into the model of the configured coding calculation rules, and the mapping obtains the final code value to determine the cognitive result of the set scene based on the final code value, including: abstracting the variable factors of the mapped code into variables; inputting the variables into the model of the configured coding calculation rules for processing, wherein the value constraints and relational expressions of the variables are used to calculate the face changing to obtain the final code value; and determining the scene cognition result based on the final code value.

[0048] The processing flow of the coding cognitive computing system for the above various dependencies and functions is as follows: under the specified rule ID, the coding generation module reads the variables defined by the rule into the data structure of the program, and filters and selects appropriate variables according to the dependency of each variable and the initial value of the variable, and loads them into the variable assignment table, and then the user assigns values ​​to the variables.

[0049] Each time a variable is assigned a value, the program checks the dependencies of other variables on it, and determines the source of the values ​​of other variables based on the dependencies, as well as whether the variable needs to be assigned and how to assign the value. Then the coding calculation function is called to calculate the final coding value. The present invention analyzes that conventional coding tools are generally only suitable for relatively regular flexible coding systems. The coding cognitive calculation can generate codes of arbitrary length and structure; the dependencies in the coding rules are automatically reflected in the dynamic changes of the interface, providing navigation for the user's assignment process; different variables and script configurations are used to deal with different coding rules, thereby achieving flexible coding. This coding system can be integrated into Kaimu products as a built-in component, and used in general and special coding management.

[0050] This embodiment uses multi-source (multimodal) data in a specific scenario as the original data, extracts business semantic segmentation and corresponding knowledge in a specific scenario as the basic data for constructing the knowledge and business library in the specific scenario, and then adopts a consistent coding system to encode the categories, terms and attribute semantic segmentation related to the scene with serial numbers, establishes coding and decoding rules, and performs automatic intelligent mapping. Finally, by interpreting and calculating the coding calculation rules, code values ​​of different coding structures and lengths are obtained to achieve scene recognition results. That is, by aggregating and integrating cross-departmental, cross-level, and cross-regional scene data, analyzing them according to the three dimensions of time, space, and attributes, and combining different attributes with time and space dimensions with the help of coding, the intrinsic connection between scenes and the trend of change over time and the law of spatial distribution are completed, thereby realizing coding-driven comprehensive analysis of scene data, so that people can better gain more insights from massive and complex multi-source and multi-modal scene data, and empower various scene businesses.

[0051] The present invention

[0052] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0053] According to an embodiment of the present invention, a scene recognition device is also provided, including a preprocessing unit, which is used to obtain multi-source multi-modal data samples of services in different scenarios, and extract semantic participles in the samples; fuse the semantic participles to obtain a feature vector set; encode the semantic participles in the semantic participle set to obtain a standard semantic participle classification tree code; a code mapping unit, which is used to determine the code mapped to the multi-source multi-modal data of the service in the target scenario to be recognized based on the standard semantic participle classification tree code; and a scene recognition result determination unit, which is used to input the mapped code into a model of the configured coding calculation rule, and map the final code value to determine the recognition result of the set scene based on the final code value.

[0054] As an optional implementation method of this embodiment, fusing the semantic word segmentations to obtain a feature vector set includes: fusing the semantic word segmentations based on a probability density estimation method; wherein, classifiers with the same structure are trained based on different semantic word segmentations; and the probability density output by the classifier is used as the fused feature vector.

[0055] As an optional implementation method of this embodiment, encoding the semantic participles in the semantic participle set includes: obtaining encoding rules configured by a user based on the granularity of the semantic participles, wherein the encoding rules are determined based on different types of code segments, and different types of code segments are composed of one or more encoding nodes; encoding the semantic participles based on the encoding rules.

[0056] As an optional implementation method of this embodiment, the different types of code segments include classification code segments, attribute code segments, pipeline code segments, and / or symbol code segments, wherein the classification code segments and attribute code segments are associated with the classification information and attribute information in the classification library and attribute library respectively; the classification library and attribute library store the classification information and attribute information of physical objects in the scene, and there is an association relationship between the classification library and the attribute library, so that different classifications are associated with different attribute information.

[0057] As an optional implementation method of this embodiment, based on the standard semantic word segmentation classification tree encoding, determining the encoding mapped to the multi-source multimodal data of the business in the target scenario to be recognized includes: performing keyword probability matching based on the hidden Markov model to obtain the mapped encoding.

[0058] As an optional implementation method of this embodiment, the mapped code is input into the model of the configured coding calculation rules, and the mapping obtains the final code value to determine the cognitive result of the set scene based on the final code value, including: abstracting the variable factors of the mapped code into variables; inputting the variables into the model of the configured coding calculation rules for processing, wherein the value constraints and relational expressions of the variables are used to calculate the face changing to obtain the final code value; and determining the scene cognition result based on the final code value.

[0059] According to an embodiment of the present invention, the present invention also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the method described in any of the above embodiments when executing.

[0060] According to an embodiment of the present invention, the present invention further provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the method described in any of the above embodiments when executed.

[0061] According to an embodiment of the present invention, the present invention further provides a computer program product, which can implement the method described in any of the above embodiments when executed by a processor.

[0062] Figure 2A schematic block diagram of an example electronic device 300 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices.

[0063] like Figure 2 As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0064] A number of components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0065] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above, such as the object matching method. For example, in some embodiments, the object matching method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the method described above may be performed.

[0066] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0067] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0068] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Claims

1. A scene recognition method, characterized in that: include: Acquire multi-source and multi-modal data samples of services in different scenarios, and extract semantic word segmentations from the samples; fuse the semantic word segmentations to obtain a feature vector set; Encoding the semantic participles in the semantic participle set to obtain a standard semantic participle classification tree encoding; Based on the standard semantic word segmentation classification tree code, determine the code mapped by the multi-source multi-modal data of the business in the target scenario to be recognized; The mapped code is input into the model of the configured code calculation rule, and the final code value is obtained by mapping to determine the cognitive result of the set scene based on the final code value.

2. The scene recognition method according to claim 1, characterized in that: The feature vector set obtained by fusing the semantic word segmentation includes: The semantic segmentation is integrated based on the probability density estimation method; Among them, classifiers with the same structure are trained based on different semantic segmentations; the probability density output by the classifier is used as the fused feature vector.

3. The scene recognition method according to claim 1, characterized in that: Encoding the semantic segmentation words in the semantic segmentation set includes: Obtaining a coding rule configured by a user based on the granularity of semantic word segmentation, wherein the coding rule is determined based on different types of code segments, and different types of code segments are composed of one or more coding nodes; The semantic word segmentation is encoded based on the encoding rule.

4. The scene recognition method according to claim 3, characterized in that: The different types of code segments include classification code segments, attribute code segments, pipeline code segments, and / or symbol code segments. Among them, the classification code segment and the attribute code segment are associated with the classification information and the attribute information in the classification library and the attribute library respectively; the classification library and the attribute library store the classification information and the attribute information of the physical objects in the scene, and there is an association relationship between the classification library and the attribute library, so that different classifications are associated with different attribute information.

5. The scene recognition method according to claim 4, characterized in that: Based on the standard semantic word segmentation classification tree encoding, determining the encoding mapped to the multi-source multi-modal data of the business in the target scenario to be recognized includes: The keyword probability matching is performed based on the hidden Markov model to obtain the mapped code.

6. The scene recognition method according to claim 5, characterized in that: Inputting the mapped code into the model of the configured code calculation rule, mapping to obtain a final code value to determine the cognitive result of the set scene based on the final code value includes: The variable factors of the mapping-based encoding are abstracted into variables; The variables are input into the model of the configured coding calculation rules for processing, wherein the value constraints and relational expressions of the variables are used to calculate the face change to obtain the final code value; A scene recognition result is determined based on the final code value.

7. A scene recognition device, characterized in that: include: A preprocessing unit, used to obtain multi-source and multi-modal data samples of services in different scenarios, and extract semantic word segmentations from the samples; and fuse the semantic word segmentations to obtain a feature vector set; Encoding the semantic participles in the semantic participle set to obtain a standard semantic participle classification tree encoding; A coding mapping unit, used to determine the coding mapped to the multi-source multi-modal data of the service in the target scenario to be recognized based on the standard semantic word segmentation classification tree coding; The scene recognition result determination unit is used to input the mapped code into the model of the configured code calculation rule, map to obtain the final code value to determine the recognition result of the set scene based on the final code value.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the method described in any one of claims 1-6.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 6.

Images