Standard operating procedure system establishment method and device

CN116340744BActive Publication Date: 2026-08-07深圳鼎匠科技有限公司
View PDF 4 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
深圳鼎匠科技有限公司
Filing Date
2023-03-06
Publication Date
2026-08-07

AI Technical Summary

Technical Problem

[0003]但是,SOP系统建立过程中由于训练数据来源复杂,标准化程度不高,通常涉及文字、图像、语音和视频等多模态的数据,由于技术人员经验和关注点存在差异,导致通过多模态训练数据建立SOP系统的通用性不好

Benefits of technology

[0092]本发明实施例中,通过确定训练数据中各子数据的数据模态,进而确定相应的数据转换方法,获得对应的、文字形式的目标子数据,提取目标子数据中关键词的语义,获得对应于各所述目标子数据的SOP子树,进而通过所述SOP子树建立SOP系统,提升了多模态训练数据建立SOP系统的通用性和标准化水平。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116340744B_ABST
    Figure CN116340744B_ABST
Patent Text Reader

Abstract

The application discloses a standard operation procedure system establishment method and device, and the method comprises the following steps: acquiring training data; determining the data mode of each sub-data in the training data, and performing data conversion on each sub-data according to the data mode to obtain target sub-data; acquiring the semantics of keywords in each target sub-data, and obtaining a standard operation procedure (SOP) sub-tree corresponding to each target sub-data according to the semantics of the keywords; and establishing an SOP system according to each SOP sub-tree. By determining the data mode of each sub-data in the training data, the corresponding data conversion method is determined, the corresponding target sub-data in the form of characters is obtained, the semantics of the keywords in the target sub-data is extracted, the SOP sub-tree corresponding to each target sub-data is obtained, and then the SOP system is established through the SOP sub-tree, so that the generality and standardization level of the SOP system established from the multi-modal training data are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and apparatus for establishing a standard operating procedure system. Background Technology

[0002] Standard Operating Procedures (SOPs) refine and quantify the key control points in a specific workflow. They describe the standard operating steps and requirements for a particular business in a unified format to guide and standardize business operations, improve operational efficiency, and reduce the probability of errors.

[0003] However, the establishment of Standard Operating Procedures (SOPs) systems is hampered by the complexity and low standardization of training data sources. These data typically involve multimodal data such as text, images, audio, and video. Differences in the experience and focus of technical personnel lead to poor versatility in SOP systems built using multimodal training data. Therefore, improving the versatility of SOP systems built using multimodal training data is crucial. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and apparatus for establishing a standard operating procedure system, which can determine the data modality of each sub-data in the training data, and then determine the corresponding data conversion method to obtain the corresponding target sub-data in text form, and establish an SOP system based on this, thereby improving the universality and standardization level of establishing an SOP system from multimodal training data.

[0005] To address the aforementioned technical problems, the first aspect of this invention discloses a method for establishing a standard operating procedure system, the method comprising:

[0006] Obtain training data;

[0007] Determine the data modality of each sub-data in the training data, and perform data transformation on each sub-data according to the data modality to obtain the target sub-data;

[0008] Obtain the semantics of the keywords in each of the target sub-data, and obtain the standard operating procedure (SOP) subtree corresponding to each of the target sub-data based on the semantics of the keywords;

[0009] Establish an SOP system based on each of the described SOP subtrees.

[0010] As an optional implementation, obtaining the standard operating procedure (SOP) subtree corresponding to each of the target sub-data based on the semantics of the keywords includes:

[0011] Based on the semantics of the keywords, determine action information, and / or input information, and / or output information;

[0012] Based on the action information, and / or input information, and / or output information, determine the standardized SOP point corresponding to each of the target sub-data, and based on the standardized SOP point, obtain the standard operating procedure (SOP) subtree corresponding to each of the target sub-data.

[0013] As an optional implementation, the standard operating procedure (SOP) subtree includes multiple standardized SOP points, and obtaining the SOP subtree corresponding to each of the target sub-data based on the standardized SOP points includes:

[0014] Based on the action information, and / or input information, and / or output information, determine the operating sequence of each of the standardized SOP points;

[0015] Based on the described operating sequence, the standardized SOP points are combined to obtain a standard operating procedure (SOP) subtree corresponding to each of the target sub-data.

[0016] As an optional implementation, the data modality includes text data; the step of performing data conversion on each of the sub-data according to the data modality to obtain the target sub-data includes:

[0017] The target sub-data is obtained according to the first data transformation process;

[0018] The first data conversion process includes:

[0019] Identify the first keyword in the text data;

[0020] Tag the first keyword to obtain the target sub-data tagged with the first keyword.

[0021] As an optional implementation, the data modality includes audio data; the step of performing data conversion on each of the sub-data according to the data modality to obtain the target sub-data includes:

[0022] The target sub-data is obtained according to the second data transformation process;

[0023] The second data conversion process includes:

[0024] The sound data is converted to obtain first text information corresponding to the sound data;

[0025] Based on the first text information, determine the second keyword;

[0026] The second keyword is labeled to obtain the target sub-data labeled with the second keyword.

[0027] As an optional implementation, the data modality includes image data; the step of performing data transformation on each of the sub-data according to the data modality to obtain target sub-data includes:

[0028] The target sub-data is obtained according to the third data transformation process;

[0029] The third data conversion process includes:

[0030] Determine whether the image data is origin image data;

[0031] If it is determined that the image data is origin image data, then based on the image entities in the image data, the second text information corresponding to the image entities is obtained;

[0032] Based on the second text information, a third keyword is determined; and the third keyword is marked to obtain the target sub-data marked with the third keyword.

[0033] If it is determined that the image data is not the origin image data, then the third text information in the image data is obtained; and, based on the image entity in the image data, the fourth text information corresponding to the image entity is obtained.

[0034] Based on the third text information and / or the fourth text information, determine the fourth keyword; and mark the fourth keyword to obtain the target sub-data marked with the fourth keyword.

[0035] As an optional implementation, the data modality includes video data; the step of performing data conversion on each of the sub-data according to the data modality to obtain the target sub-data includes:

[0036] Based on the video data, determine the text sub-data, and / or image sub-data, and / or voice sub-data corresponding to the video data;

[0037] According to the first data conversion process, and / or the second data conversion process, and / or the third data conversion process, the video data is converted to obtain target sub-data corresponding to the video data.

[0038] As an optional implementation, after establishing the SOP system based on each of the SOP subtrees, the method further includes:

[0039] Based on the SOP system, a first SOP system and a second SOP system are determined; wherein, the first SOP system is used for vehicle fault diagnosis, and the second SOP system is used for vehicle automatic maintenance.

[0040] Based on the preset range of various parameters of the target vehicle, the target parameters of the target vehicle are determined through the first SOP system; the target parameters are used to indicate the fault units of the target vehicle.

[0041] The target parameters are processed through the second SOP system.

[0042] As an optional implementation, the processing of the target parameters through the second SOP system includes:

[0043] Determine the type of fault unit indicated by the target parameter; wherein, the type of fault unit includes faulty software or faulty hardware;

[0044] If the fault unit type indicated by the target parameter is faulty software, then the faulty software is processed through the second SOP system until the target parameter falls into the corresponding preset range;

[0045] If the fault unit type indicated by the target parameter is faulty hardware, then the mechanical device for automatic vehicle maintenance is controlled through the second SOP system to process the faulty hardware until the target parameter falls into the corresponding preset range.

[0046] A second aspect of the present invention discloses a standard operating procedure system establishment apparatus, characterized in that the apparatus comprises:

[0047] The data acquisition module is used to acquire training data;

[0048] The modality determination module is used to determine the data modality of each sub-data in the training data, and to perform data transformation on each sub-data according to the data modality to obtain the target sub-data;

[0049] The program determination module is used to obtain the semantics of keywords in each of the target sub-data, and obtain the standard operating procedure (SOP) subtree corresponding to each of the target sub-data based on the semantics of the keywords;

[0050] The program determination module is also used to establish an SOP system based on each of the SOP subtrees.

[0051] As an optional implementation, the program determining module obtains the specific method of the standard operating procedure (SOP) subtree corresponding to each of the target sub-data based on the semantics of the keywords, including:

[0052] Based on the semantics of the keywords, determine action information, and / or input information, and / or output information;

[0053] Based on the action information, and / or input information, and / or output information, determine the standardized SOP point corresponding to each of the target sub-data, and based on the standardized SOP point, obtain the standard operating procedure (SOP) subtree corresponding to each of the target sub-data.

[0054] As an optional implementation, the Standard Operating Procedure (SOP) subtree includes multiple standardized SOP points. The program determination module obtains the specific method of the SOP subtree corresponding to each target sub-data based on the standardized SOP points, including:

[0055] Based on the action information, and / or input information, and / or output information, determine the operating sequence of each of the standardized SOP points;

[0056] Based on the described operating sequence, the standardized SOP points are combined to obtain a standard operating procedure (SOP) subtree corresponding to each of the target sub-data.

[0057] As an optional implementation, the data modality includes text data; the modality determination module performs data transformation on each of the sub-data according to the data modality to obtain the target sub-data in the following specific ways:

[0058] The target sub-data is obtained according to the first data transformation process;

[0059] The first data conversion process includes:

[0060] Identify the first keyword in the text data;

[0061] Tag the first keyword to obtain the target sub-data tagged with the first keyword.

[0062] As an optional implementation, the data modality includes audio data; the modality determination module performs data conversion on each of the sub-data according to the data modality to obtain the target sub-data, and the specific method further includes:

[0063] The target sub-data is obtained according to the second data transformation process;

[0064] The second data conversion process includes:

[0065] The sound data is converted to obtain first text information corresponding to the sound data;

[0066] Based on the first text information, determine the second keyword;

[0067] The second keyword is labeled to obtain the target sub-data labeled with the second keyword.

[0068] As an optional implementation, the data modality includes image data; the modality determination module performs data transformation on each of the sub-data according to the data modality to obtain the target sub-data, and the specific method further includes:

[0069] The target sub-data is obtained according to the third data transformation process;

[0070] The third data conversion process includes:

[0071] Determine whether the image data is origin image data;

[0072] If it is determined that the image data is origin image data, then based on the image entities in the image data, the second text information corresponding to the image entities is obtained;

[0073] Based on the second text information, a third keyword is determined; and the third keyword is marked to obtain the target sub-data marked with the third keyword.

[0074] If it is determined that the image data is not the origin image data, then the third text information in the image data is obtained; and, based on the image entity in the image data, the fourth text information corresponding to the image entity is obtained.

[0075] Based on the third text information and / or the fourth text information, determine the fourth keyword; and mark the fourth keyword to obtain the target sub-data marked with the fourth keyword.

[0076] As an optional implementation, the data modality includes video data; the modality determination module performs data conversion on each of the sub-data according to the data modality to obtain the target sub-data, and the specific method further includes:

[0077] Based on the video data, determine the text sub-data, and / or image sub-data, and / or voice sub-data corresponding to the video data;

[0078] According to the first data conversion process, and / or the second data conversion process, and / or the third data conversion process, the video data is converted to obtain target sub-data corresponding to the video data.

[0079] As an optional implementation, the apparatus further includes an SOP application module, used after the program determination module establishes the SOP system according to each of the SOP subtrees,

[0080] Based on the SOP system, a first SOP system and a second SOP system are determined; wherein, the first SOP system is used for vehicle fault diagnosis, and the second SOP system is used for vehicle automatic maintenance.

[0081] Based on the preset range of various parameters of the target vehicle, the target parameters of the target vehicle are determined through the first SOP system; the target parameters are used to indicate the fault units of the target vehicle.

[0082] The target parameters are processed through the second SOP system.

[0083] As an optional implementation, the specific method by which the SOP application module processes the target parameters through the second SOP system includes:

[0084] Determine the type of fault unit indicated by the target parameter; wherein, the type of fault unit includes faulty software or faulty hardware;

[0085] If the fault unit type indicated by the target parameter is faulty software, then the faulty software is processed through the second SOP system until the target parameter falls into the corresponding preset range;

[0086] If the fault unit type indicated by the target parameter is faulty hardware, then the mechanical device for automatic vehicle maintenance is controlled through the second SOP system to process the faulty hardware until the target parameter falls into the corresponding preset range.

[0087] A third aspect of the present invention discloses another standard operating procedure system establishment apparatus, the apparatus comprising:

[0088] Memory containing executable program code;

[0089] A processor coupled to the memory;

[0090] The processor calls the executable program code stored in the memory to execute the standard operating procedure system establishment method disclosed in the first aspect of the present invention.

[0091] The fourth aspect of the present invention discloses a computer-storable medium storing computer instructions, which, when invoked, are used to execute the standard operating procedure system establishment method disclosed in the first aspect of the present invention.

[0092] In this embodiment of the invention, by determining the data modality of each sub-data in the training data, and then determining the corresponding data conversion method, the corresponding target sub-data in text form is obtained. The semantics of keywords in the target sub-data are extracted, and the SOP subtree corresponding to each target sub-data is obtained. Then, an SOP system is established through the SOP subtree, thereby improving the universality and standardization level of establishing an SOP system from multimodal training data. Attached Figure Description

[0093] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0094] Figure 1 This is a schematic diagram illustrating an application scenario of a standard operating procedure system establishment method disclosed in an embodiment of the present invention;

[0095] Figure 2 This is a flowchart illustrating a method for establishing a standard operating procedure system according to an embodiment of the present invention;

[0096] Figure 3 This is a flowchart illustrating another method for establishing a standard operating procedure system disclosed in an embodiment of the present invention;

[0097] Figure 4 This is a flowchart illustrating another method for establishing a standard operating procedure system disclosed in an embodiment of the present invention;

[0098] Figure 5 This is a schematic diagram of the structure of a standard operating procedure system establishment device disclosed in an embodiment of the present invention;

[0099] Figure 6 This is a schematic diagram of another standard operating procedure system establishment device disclosed in an embodiment of the present invention;

[0100] Figure 7 This is a schematic diagram of the structure of another standard operating procedure system establishment device disclosed in the embodiments of the present invention. Detailed Implementation

[0101] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0102] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or end that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or ends.

[0103] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0104] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario of a standard operating procedure system creation method disclosed in an embodiment of the present invention. For example... Figure 1 As shown, SOPs refine and quantify the key control points in specific workflows, describing the standard operating procedures and requirements for a particular business in a unified format. This serves to guide and standardize business operations, improve operational efficiency, and reduce the probability of errors.

[0105] In a vehicle diagnostics application scenario, the following functions need to be standardized through an SOP system:

[0106] Basic diagnostic functions include reading version information, reading fault codes, clearing fault codes, reading data streams, freezing frames, and motion testing; commonly used special functions include battery matching, maintenance reset, brake pad replacement, throttle matching, steering angle reset, ABS exhaust, DPF regeneration, airbag reset, suspension matching, tire pressure reset, transmission matching, fuel injector coding, gear learning, transport mode deactivation, headlight matching, sunroof initialization, seat calibration, door and window calibration, start-stop setting, ERG self-learning, odometer adjustment, language setting, tire modification, A / F tuning, electronic water pump activation, NOx reset, urea reset, turbo blade learning, and cylinder balance testing; coding functions include seat preference settings, fog light settings, and automatic window closing; ECU flashing function; and maintenance guidance function.

[0107] To standardize these complex functions through a Standard Operating Procedure (SOP) system, it is necessary to collect image, text, video, or audio data based on internet resources or the operational experience of technical personnel. The content contained in this data is then described in text form, constructing SOP points that include input, action, and output content, thus building the SOP system. Therefore, the development of an SOP system is challenging due to the complexity and low standardization of training data sources, often involving multimodal data. Furthermore, differences in the experience and focus of technical personnel result in poor universality of SOP systems built using multimodal training data. Therefore, improving the universality of SOP systems built using multimodal training data is crucial.

[0108] This invention discloses a method and apparatus for establishing a standard operating procedure (SOP) system. By determining the data modality of each sub-data in the training data, and then determining the corresponding data conversion method, the corresponding target sub-data in text form is obtained. The semantics of keywords in the target sub-data are extracted to obtain SOP subtrees corresponding to each target sub-data. Then, an SOP system is established through the SOP subtrees, thereby improving the universality and standardization level of establishing an SOP system from multimodal training data.

[0109] Example 1

[0110] Please see Figure 2 , Figure 2 This is a flowchart illustrating a method for establishing a standard operating procedure system according to an embodiment of the present invention. Figure 2 As shown, the method for establishing the standard operating procedure system may include the following operations:

[0111] S101. Obtain training data;

[0112] S102. Determine the data mode of each sub-data in the training data, and perform data transformation on each sub-data according to the data mode to obtain the target sub-data;

[0113] Training data can include one or more modalities such as text data, image data, audio data, or video data. The data modality of each sub-data item in the training data can be determined by file naming or file format. Based on the corresponding data modality, a data transformation method is determined to obtain the target sub-data. The data modality of the target sub-data can be uniformly set to text data. In an application scenario requiring the establishment of a Standard Operating Procedure (SOP) system related to vehicle diagnostics, text data, image data, audio data, or video data from network data, industry-standard operating procedures, and self-made resources can be converted into target sub-data with a text modality and then processed accordingly to form an SOP system for the vehicle diagnostics field.

[0114] For text data, keyword recognition and tagging can be performed to improve the efficiency of obtaining semantic content in subsequent steps. For image data, audio data, or video data, the information contained therein can be converted into text form through certain methods. For details, please refer to the relevant description in Example 2.

[0115] S103. Obtain the semantics of the keywords in each of the target sub-data, and obtain the standard operating procedure (SOP) subtree corresponding to each of the target sub-data based on the semantics of the keywords;

[0116] Keywords can be entity words in text data. For example, in the aforementioned field of vehicle diagnostics, keywords can be brand names, category names, attribute names, etc., of vehicles or hardware devices such as engines, batteries, and transmissions within vehicles—words with practical meaning or that can describe specific objects. The identification of entity words in text data can be performed through text encoding networks to segment characters and determine the category of each segment, thereby identifying keywords in each target sub-data. Keyword identification can be performed in S102 or S103. Then, based on the semantics of the keywords, a Standard Operating Procedure (SOP) subtree corresponding to each target sub-data is obtained. The SOP subtree may contain only one standard SOP point or a standard SOP chain composed of multiple standard SOP points.

[0117] SOPs may include one or more of input information, action information, and output information, but should include at least action information.

[0118] S104. Establish an SOP system based on each of the SOP subtrees.

[0119] As an optional implementation, establishing an SOP system based on each of the SOP subtrees includes:

[0120] Determine the application requirements of the SOP system;

[0121] Based on the application requirements and the operational sequence of each standardized SOP point in each SOP subtree, the SOP system is established.

[0122] Based on the specific application area of ​​the SOP system, the application requirements corresponding to the SOP system can be determined. For example, in the field of vehicle diagnostics, the application requirements and corresponding processes related to "battery matching" can be determined. Furthermore, based on the application requirements and the semantic and logical relationships between each standardized SOP point, the operating order between each standardized SOP point in each SOP subtree can be determined. Finally, the SOP system is established according to the specific application requirements.

[0123] As an optional implementation, after establishing the SOP system based on each of the SOP subtrees, the method further includes:

[0124] Based on the SOP system, a first SOP system and a second SOP system are determined; wherein, the first SOP system is used for vehicle fault diagnosis, and the second SOP system is used for vehicle automatic maintenance.

[0125] This implementation method is a specific application scenario in the field of vehicle diagnostics. Through the established SOP system, the first SOP system for vehicle fault diagnosis and the second SOP system for automatic vehicle maintenance can be determined by further determining each standardized SOP point or by pre-marking in the established SOP system.

[0126] Based on the preset range of various parameters of the target vehicle, the target parameters of the target vehicle are determined through the first SOP system; the target parameters are used to indicate the fault units of the target vehicle.

[0127] Since the first SOP system can be used for vehicle fault diagnosis, the preset range of each parameter of the target vehicle can be used as the diagnostic standard. The first SOP system can determine whether each parameter meets the preset range and take the parameters that do not fall into the corresponding preset range as the target parameters to be processed. The target parameters can be used to indicate the corresponding fault unit. For example, the vehicle load parameter can be used to indicate the engine's operating status. When the vehicle load parameter does not meet the preset range, the corresponding diagnostic result of engine abnormality can be output.

[0128] The target parameters are processed through the second SOP system.

[0129] The second SOP system can be used for automatic maintenance. After fault diagnosis is completed, the second SOP system can make corresponding modifications or control the corresponding hardware to perform corresponding maintenance actions, thereby processing the target parameters until the target parameters return to the preset range.

[0130] By establishing a Standard Operating Procedure (SOP) system based on multimodal data, a first SOP system for vehicle fault diagnosis and a second SOP system for vehicle automated maintenance are further determined. The target parameters are determined through the first SOP system and processed through the second SOP system, thereby improving the practicality of the SOP system established from multimodal training data.

[0131] As an optional implementation, the processing of the target parameters through the second SOP system includes:

[0132] Determine the type of fault unit indicated by the target parameter; wherein, the type of fault unit includes faulty software or faulty hardware;

[0133] If the fault unit type indicated by the target parameter is faulty software, then the faulty software is processed through the second SOP system until the target parameter falls into the corresponding preset range;

[0134] If the fault unit type indicated by the target parameter is faulty hardware, then the mechanical device for automatic vehicle maintenance is controlled through the second SOP system to process the faulty hardware until the target parameter falls into the corresponding preset range.

[0135] As described in the previous embodiment, the second SOP system can be used for automatic maintenance. After the first SOP system completes the fault diagnosis, it can determine whether the fault unit type indicated by the target parameter is faulty software or faulty hardware. Then, the second SOP system can make corresponding modification actions for the faulty software, or control the mechanical device used for automatic vehicle maintenance to make corresponding maintenance actions, process the faulty hardware, and then process the target parameter until the target parameter returns to the preset range.

[0136] By determining the type of fault unit indicated by the target parameters and performing corresponding software or hardware processing according to the second SOP, the practicality of the SOP system established from multimodal training data is improved.

[0137] As an optional implementation, obtaining the standard operating procedure (SOP) subtree corresponding to each of the target sub-data based on the semantics of the keywords includes:

[0138] Based on the semantics of the keywords, determine action information, and / or input information, and / or output information;

[0139] As mentioned earlier, a Standard Operating Procedure (SOP) can include one or more of input information, action information, and output information, but should at least include action information. A standardized SOP should simultaneously include all three aspects: input information, action information, and output information. Therefore, a standardized SOP can only be directly determined when the action information, input information, and output information can be determined based on the semantics of the keywords. Otherwise, the determined SOP needs further supplementation with relevant information to generate the corresponding standardized SOP.

[0140] Based on the action information, and / or input information, and / or output information, determine the standardized SOP point corresponding to each of the target sub-data, and based on the standardized SOP point, obtain the standard operating procedure (SOP) subtree corresponding to each of the target sub-data.

[0141] When there is only one standardized SOP point corresponding to the target sub-data, it can be directly used as the SOP subtree. If there are multiple standardized SOP points corresponding to the target sub-data, it is also necessary to determine the logical and semantic relationships between the standardized SOP points, and then mark the execution order to generate a standardized SOP chain, thereby obtaining the standard operating procedure SOP subtree corresponding to the target sub-data.

[0142] By analyzing the semantics of keywords in the target sub-data obtained from multimodal data transformation, action information, and / or input information, and / or output information are obtained, thereby determining the standardized SOP points corresponding to each target sub-data. This improves the efficiency of establishing the SOP system and enhances the universality and standardization of establishing the SOP system using multimodal training data.

[0143] As an optional implementation, the standard operating procedure (SOP) subtree includes multiple standardized SOP points, and obtaining the SOP subtree corresponding to each of the target sub-data based on the standardized SOP points includes:

[0144] Based on the action information, and / or input information, and / or output information, determine the operating sequence of each of the standardized SOP points;

[0145] Based on the described operating sequence, the standardized SOP points are combined to obtain a standard operating procedure (SOP) subtree corresponding to each of the target sub-data.

[0146] As mentioned above, when there are multiple standardized SOP points corresponding to the target sub-data, it is also necessary to determine the logical and semantic relationships between each standardized SOP point. During the process, the running order of each standardized SOP point is determined. Furthermore, according to the running order, each standardized SOP point is combined to generate a standardized SOP chain, thereby obtaining a standard operating procedure (SOP) subtree corresponding to each target sub-data.

[0147] By using the action information, and / or input information, and / or output information of the target sub-data obtained from multimodal data transformation, the running sequence and logical relationship of each standardized SOP point are determined. Then, the standardized SOP points are combined to form a chain, and an SOP subtree is obtained, which improves the efficiency of SOP system establishment and enhances the universality and standardization level of SOP system establishment using multimodal training data.

[0148] This embodiment provides a method for establishing a Standard Operating Procedure (SOP) system. The method includes: acquiring training data; determining the data modality of each sub-data in the training data, and performing data transformation on each sub-data according to the data modality to obtain target sub-data; acquiring the semantics of keywords in each target sub-data, and obtaining a SOP subtree corresponding to each target sub-data according to the semantics of the keywords; and establishing an SOP system based on each SOP subtree. By determining the data modality of each sub-data in the training data, and then determining the corresponding data transformation method, the corresponding text-based target sub-data is obtained. The semantics of keywords in the target sub-data are extracted to obtain a SOP subtree corresponding to each target sub-data. The SOP system is then established using the SOP subtree, improving the universality and standardization level of establishing an SOP system from multimodal training data.

[0149] Example 2

[0150] Please see Figure 3 , Figure 3 This is a flowchart illustrating another method for establishing a standard operating procedure system disclosed in an embodiment of the present invention. Figure 3 As shown, the data modality includes text data, and the standard operating procedure system establishment method may include the following operations:

[0151] S201. Obtain training data;

[0152] S202. Obtain the target sub-data according to the first data conversion process;

[0153] The first data conversion process includes:

[0154] Identify the first keyword in the text data;

[0155] Tag the first keyword to obtain the target sub-data tagged with the first keyword.

[0156] For text data, at least one first keyword can be determined based on the text encoding network described above, and each first keyword is labeled. The labeling process can be regarded as a data transformation process. Through this pre-labeling, the efficiency of SOP system establishment can be improved. The purpose of this step is to obtain the labeled target sub-data. Since the modality of the preset target sub-data can be text data, if the sub-data in the training data is text data, only keyword extraction and labeling are required. However, for data of other modalities, it may involve data modality transformation, unifying the information of different modalities into text information for processing.

[0157] S203. Obtain the semantics of the keywords in each of the target sub-data, and obtain the standard operating procedure (SOP) subtree corresponding to each of the target sub-data based on the semantics of the keywords;

[0158] S204. Establish an SOP system based on each of the SOP subtrees.

[0159] For a description of S201, S203 and S204, please refer to the description of S101, S103 and S104 in Embodiment 1, which will not be repeated here.

[0160] As an optional implementation, the data modality includes audio data; the step of performing data conversion on each of the sub-data according to the data modality to obtain the target sub-data includes:

[0161] The target sub-data is obtained according to the second data transformation process;

[0162] The second data conversion process includes:

[0163] The sound data is converted to obtain first text information corresponding to the sound data;

[0164] Based on the first text information, determine the second keyword;

[0165] The second keyword is labeled to obtain the target sub-data labeled with the second keyword.

[0166] For the process of converting sound data into text data, it is only necessary to use a speech recognition algorithm to obtain the content of the speech, and then obtain the first text information corresponding to the sound data. Alternatively, based on the frequency, intensity, and other features of the speech, the actual meaning corresponding to these features can be parsed to obtain the first text information corresponding to the sound data. After obtaining the first text information, the second keyword can be determined according to the text data processing flow, and the second keyword can be marked to obtain the target sub-data marked with the second keyword.

[0167] By acquiring the first textual information from the audio data and then tagging the keywords in the textual data, the efficiency of acquiring the semantics corresponding to the audio data is improved, which in turn improves the efficiency of establishing the SOP subtree and enhances the practicality and standardization of the SOP system established through audio training data.

[0168] As an optional implementation, the data modality includes image data; the step of performing data transformation on each of the sub-data according to the data modality to obtain target sub-data includes:

[0169] The target sub-data is obtained according to the third data transformation process;

[0170] The third data conversion process includes:

[0171] Determine whether the image data is origin image data;

[0172] If it is determined that the image data is origin image data, then based on the image entities in the image data, the second text information corresponding to the image entities is obtained;

[0173] Based on the second text information, a third keyword is determined; and the third keyword is marked to obtain the target sub-data marked with the third keyword.

[0174] If it is determined that the image data is not the origin image data, then the third text information in the image data is obtained; and, based on the image entity in the image data, the fourth text information corresponding to the image entity is obtained.

[0175] Based on the third text information and / or the fourth text information, determine the fourth keyword; and mark the fourth keyword to obtain the target sub-data marked with the fourth keyword.

[0176] The conversion of image data is more complex than that of sound data. Image data includes two types: one is origin image data composed of pure graphics, and the other is non-origin image data. The image may already contain some text information. It is necessary to separate the text information and graphic information in the image, and then further process the graphic information to parse out the text information corresponding to each image entity.

[0177] By acquiring the text information corresponding to each image entity in the image data, or by acquiring both the text information in the image data and the text information corresponding to each image entity in the image data, and then marking the keywords in the text information, the efficiency of acquiring the semantics corresponding to the image data is improved, thereby improving the efficiency of building the SOP subtree and enhancing the practicality and standardization of the SOP system built through image training data.

[0178] As an optional implementation, the data modality includes video data; the step of performing data conversion on each of the sub-data according to the data modality to obtain the target sub-data includes:

[0179] Based on the video data, determine the text sub-data, and / or image sub-data, and / or voice sub-data corresponding to the video data;

[0180] According to the first data conversion process, and / or the second data conversion process, and / or the third data conversion process, the video data is converted to obtain target sub-data corresponding to the video data.

[0181] Video data is the most complex of the various data modalities. It requires multiple decompositions. First, it needs to be parsed to extract one or more of the following: text, image, and speech sub-data. When all three modalities are present, the text sub-data can be processed directly. The image sub-data needs further decomposition into corresponding graphic and text data. The graphic data is then further converted into the corresponding text data. The same applies to speech sub-data. Based on the semantic content, it can be directly converted into text data. However, the acoustic features require further parsing and conversion to obtain the corresponding text data. For the specific data conversion process, please refer to the aforementioned implementation method.

[0182] It should be noted that each branch in the decomposition process for each modality of data is an optional scheme, while text is the most basic data modality in this application. Video data can be decomposed into one or more of text data, image data, and audio data; audio data can be decomposed into one or more of text data and acoustic feature data, and further acoustic feature data can be converted into text data; image data can be decomposed into one or more of graphic data and text data, and further graphic data can be converted into text data; text data can be directly used for keyword recognition and tagging.

[0183] Please see Figure 4 , Figure 4 This is a flowchart illustrating another method for establishing a standard operating procedure system disclosed in an embodiment of the present invention. Figure 4As shown, in a scenario where a Standard Operating Procedure (SOP) system is established with multi-modal training data, the training data in the data source includes video data, audio data, image data, and text data. As mentioned earlier, before establishing the SOP system, it is necessary to determine the modality of each sub-data in the training data. If it is video data, a first round of content analysis and extraction can be performed to identify data that can be directly used for SOP system establishment, as well as data that requires further analysis and decomposition. Furthermore, for audio and image data decomposed from video data, a second round of content analysis and extraction can be performed to identify data that can be directly used for SOP system establishment, as well as data that requires further analysis and decomposition. For image data, it can also be determined whether it is an origin image that does not contain text data. If the image data is an origin image, it can be directly used to establish the SOP system; if it is not an origin image, it can be further decomposed into text data. A similar decomposition process can be performed for training data that consists of audio, image, or text data. Ultimately, all modal data can be converted into corresponding text data for content analysis and extraction, thereby generating corresponding SOP points and forming an SOP system corresponding to the preset domain and task requirements.

[0184] By performing data transformation on video data, decomposing it into text sub-data and / or image sub-data and / or audio sub-data, and performing data transformation on the decomposed sub-data of video data according to a preset data transformation process, the efficiency of obtaining semantics corresponding to video data is improved, thereby improving the efficiency of SOP subtree establishment, and enhancing the practicality and standardization level of the SOP system established through video training data.

[0185] Based on the various SOP subtrees, after establishing the SOP system, the relevant application processes can be implemented. Specifically, after the SOP system is established, it may contain multiple functions within a domain. For example, the standard operating procedures corresponding to the various sub-functions in the aforementioned vehicle diagnostic function may be integrated into the same SOP system. In this case, the specific function to be performed needs to be entered first, such as through text input, screen clicks, or voice input. Then, through text retrieval, the specific standardized SOP point is located in the SOP system, and the complete process corresponding to the sub-function is executed based on this.

[0186] The SOP system can also be updated in real time. It can capture training data from the network at preset intervals, add, delete or replace some standard SOP points in the SOP system to achieve real-time updates, or establish a new SOP tree.

[0187] The standard operating procedure (SOP) system establishment method provided in this embodiment improves the efficiency of obtaining the semantics corresponding to the text data by marking keywords in the text data, thereby improving the efficiency of SOP subtree establishment and enhancing the practicality and standardization level of the SOP system established through text training data.

[0188] Example 3

[0189] This invention also provides a standard operating procedure system establishment apparatus to implement the aforementioned method. Please refer to [link to relevant documentation]. Figure 5 , Figure 5 This is a schematic diagram of a standard operating procedure system creation device disclosed in an embodiment of the present invention. Figure 5 As shown, based on any other embodiment, the apparatus includes:

[0190] Data acquisition module 31 is used to acquire training data;

[0191] The modality determination module 32 is used to determine the data modality of each sub-data in the training data, and to perform data transformation on each sub-data according to the data modality to obtain the target sub-data;

[0192] The program determination module 33 is used to obtain the semantics of the keywords in each of the target sub-data, and obtain the standard operating procedure (SOP) subtree corresponding to each of the target sub-data based on the semantics of the keywords;

[0193] The program determination module 33 is also used to establish an SOP system based on each of the SOP subtrees.

[0194] By determining the data modality of each sub-data in the training data, and then determining the corresponding data transformation method, the corresponding target sub-data in text form is obtained. The semantics of keywords in the target sub-data are extracted, and the SOP subtree corresponding to each target sub-data is obtained. Then, an SOP system is established through the SOP subtree, which improves the universality and standardization of establishing an SOP system from multimodal training data.

[0195] As an optional implementation, the program determining module 33 obtains the specific method of the standard operating procedure (SOP) subtree corresponding to each of the target sub-data based on the semantics of the keywords, including:

[0196] Based on the semantics of the keywords, determine action information, and / or input information, and / or output information;

[0197] Based on the action information, and / or input information, and / or output information, determine the standardized SOP point corresponding to each of the target sub-data, and based on the standardized SOP point, obtain the standard operating procedure (SOP) subtree corresponding to each of the target sub-data.

[0198] By analyzing the semantics of keywords in the target sub-data obtained from multimodal data transformation, action information, and / or input information, and / or output information are obtained, thereby determining the standardized SOP points corresponding to each target sub-data. This improves the efficiency of establishing the SOP system and enhances the universality and standardization of establishing the SOP system using multimodal training data.

[0199] As an optional implementation, the standard operating procedure (SOP) subtree includes multiple standardized SOP points. The program determination module 33 obtains the specific method of the standard operating procedure (SOP) subtree corresponding to each target subdata based on the standardized SOP points, including:

[0200] Based on the action information, and / or input information, and / or output information, determine the operating sequence of each of the standardized SOP points;

[0201] Based on the described operating sequence, the standardized SOP points are combined to obtain a standard operating procedure (SOP) subtree corresponding to each of the target sub-data.

[0202] By using the action information, and / or input information, and / or output information of the target sub-data obtained from multimodal data transformation, the running sequence and logical relationship of each standardized SOP point are determined. Then, the standardized SOP points are combined to form a chain, and an SOP subtree is obtained, which improves the efficiency of SOP system establishment and enhances the universality and standardization level of SOP system establishment using multimodal training data.

[0203] As an optional implementation, the data modality includes text data; the modality determination module 32 performs data conversion on each of the sub-data according to the data modality to obtain the specific method of the target sub-data, including:

[0204] The target sub-data is obtained according to the first data transformation process;

[0205] The first data conversion process includes:

[0206] Identify the first keyword in the text data;

[0207] Tag the first keyword to obtain the target sub-data tagged with the first keyword.

[0208] By tagging keywords in text data, the efficiency of obtaining semantics corresponding to the text data is improved, which in turn improves the efficiency of building SOP subtrees and enhances the practicality and standardization of SOP systems built from text training data.

[0209] As an optional implementation, the data modality includes audio data; the modality determination module 32 performs data conversion on each of the sub-data according to the data modality to obtain the target sub-data, and the specific method further includes:

[0210] The target sub-data is obtained according to the second data transformation process;

[0211] The second data conversion process includes:

[0212] The sound data is converted to obtain first text information corresponding to the sound data;

[0213] Based on the first text information, determine the second keyword;

[0214] The second keyword is labeled to obtain the target sub-data labeled with the second keyword.

[0215] By acquiring the first textual information from the audio data and then tagging the keywords in the textual data, the efficiency of acquiring the semantics corresponding to the audio data is improved, which in turn improves the efficiency of establishing the SOP subtree and enhances the practicality and standardization of the SOP system established through audio training data.

[0216] As an optional implementation, the data modality includes image data; the modality determination module 32 performs data transformation on each of the sub-data according to the data modality to obtain the target sub-data, and the specific method further includes:

[0217] The target sub-data is obtained according to the third data transformation process;

[0218] The third data conversion process includes:

[0219] Determine whether the image data is origin image data;

[0220] If it is determined that the image data is origin image data, then based on the image entities in the image data, the second text information corresponding to the image entities is obtained;

[0221] Based on the second text information, a third keyword is determined; and the third keyword is marked to obtain the target sub-data marked with the third keyword.

[0222] If it is determined that the image data is not the origin image data, then the third text information in the image data is obtained; and, based on the image entity in the image data, the fourth text information corresponding to the image entity is obtained.

[0223] Based on the third text information and / or the fourth text information, determine the fourth keyword; and mark the fourth keyword to obtain the target sub-data marked with the fourth keyword.

[0224] By acquiring the text information corresponding to each image entity in the image data, or by acquiring both the text information in the image data and the text information corresponding to each image entity in the image data, and then marking the keywords in the text information, the efficiency of acquiring the semantics corresponding to the image data is improved, thereby improving the efficiency of building the SOP subtree and enhancing the practicality and standardization of the SOP system built through image training data.

[0225] As an optional implementation, the data modality includes video data; the modality determination module 32 performs data conversion on each of the sub-data according to the data modality to obtain the target sub-data, and the specific method further includes:

[0226] Based on the video data, determine the text sub-data, and / or image sub-data, and / or voice sub-data corresponding to the video data;

[0227] According to the first data conversion process, and / or the second data conversion process, and / or the third data conversion process, the video data is converted to obtain target sub-data corresponding to the video data.

[0228] By performing data transformation on video data, decomposing it into text sub-data and / or image sub-data and / or audio sub-data, and performing data transformation on the decomposed sub-data of video data according to a preset data transformation process, the efficiency of obtaining semantics corresponding to video data is improved, thereby improving the efficiency of SOP subtree establishment, and enhancing the practicality and standardization level of the SOP system established through video training data.

[0229] Please see Figure 6 , Figure 6 This is a schematic diagram of another standard operating procedure system creation device disclosed in an embodiment of the present invention. Figure 6 As shown, as an optional implementation, the device further includes an SOP application module 34, used after the program determination module 33 establishes the SOP system according to each of the SOP subtrees,

[0230] Based on the SOP system, a first SOP system and a second SOP system are determined; wherein, the first SOP system is used for vehicle fault diagnosis, and the second SOP system is used for vehicle automatic maintenance.

[0231] Based on the preset range of various parameters of the target vehicle, the target parameters of the target vehicle are determined through the first SOP system; the target parameters are used to indicate the fault units of the target vehicle.

[0232] The target parameters are processed through the second SOP system.

[0233] By establishing a Standard Operating Procedure (SOP) system based on multimodal data, a first SOP system for vehicle fault diagnosis and a second SOP system for vehicle automated maintenance are further determined. The target parameters are determined through the first SOP system and processed through the second SOP system, thereby improving the practicality of the SOP system established from multimodal training data.

[0234] As an optional implementation, the specific method by which the SOP application module 34 processes the target parameters through the second SOP system includes:

[0235] Determine the type of fault unit indicated by the target parameter; wherein, the type of fault unit includes faulty software or faulty hardware;

[0236] If the fault unit type indicated by the target parameter is faulty software, then the faulty software is processed through the second SOP system until the target parameter falls into the corresponding preset range;

[0237] If the fault unit type indicated by the target parameter is faulty hardware, then the mechanical device for automatic vehicle maintenance is controlled through the second SOP system to process the faulty hardware until the target parameter falls into the corresponding preset range.

[0238] By determining the type of fault unit indicated by the target parameters and performing corresponding software or hardware processing according to the second SOP, the practicality of the SOP system established from multimodal training data is improved.

[0239] Example 4

[0240] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of another standard operating procedure system establishment device disclosed in an embodiment of the present invention. For example... Figure 7 As shown, the standard operating procedure system establishment device may include:

[0241] The device includes a processor 291 and a memory 292 storing executable program code; it may also include a communication interface 293 and a bus 294. The processor 291, memory 292, and communication interface 293 can communicate with each other via the bus 294. The communication interface 293 can be used for information transmission. The processor 291 is coupled to the memory 292, and the processor 291 can call logical instructions (executable program code) in the memory 292 to execute the standard operating procedure system establishment method described in any of the above embodiments.

[0242] Furthermore, the logic instructions in the aforementioned memory 292 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0243] The memory 292, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this application. The processor 291 executes functional applications and data processing by running the software programs, instructions, and modules stored in the memory 292, thereby implementing the methods in the above-described method embodiments.

[0244] The memory 292 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 292 may include high-speed random access memory and may also include non-volatile memory.

[0245] This invention also provides a computer-readable storage medium storing computer-executable instructions, which, when invoked, are used to implement the method described in any of the embodiments.

[0246] This invention also discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps in the standard operating procedure system establishment method described in any embodiment.

[0247] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0248] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0249] Finally, it should be noted that the standard operating procedure system establishment method and apparatus disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for establishing a standard operating procedure system, characterized in that, The method includes: Obtain training data; Determine the data modality of each sub-data in the training data, and perform data transformation on each sub-data according to the data modality to obtain the target sub-data; Obtain the semantics of the keywords in each of the target sub-data, and obtain the standard operating procedure (SOP) subtree corresponding to each of the target sub-data based on the semantics of the keywords; Based on each of the SOP subtrees, establish an SOP system; After establishing the SOP system based on each of the SOP subtrees, the method further includes: Based on the SOP system, a first SOP system and a second SOP system are determined; wherein, the first SOP system is used for vehicle fault diagnosis, and the second SOP system is used for automatic vehicle maintenance. Based on the preset range of various parameters of the target vehicle, the target parameters of the target vehicle are determined through the first SOP system; the target parameters are used to indicate the fault units of the target vehicle. The target parameters are processed through the second SOP system; And, the processing of the target parameters through the second SOP system includes: Determine the type of fault unit indicated by the target parameter; wherein, the type of fault unit includes faulty software or faulty hardware; If the fault unit type indicated by the target parameter is faulty software, then the faulty software is processed through the second SOP system until the target parameter falls into the corresponding preset range; If the fault unit type indicated by the target parameter is faulty hardware, then the mechanical device for automatic vehicle maintenance is controlled through the second SOP system to process the faulty hardware until the target parameter falls into the corresponding preset range.

2. The method according to claim 1, characterized in that, The step of obtaining the standard operating procedure (SOP) subtree corresponding to each of the target sub-data based on the semantics of the keywords includes: Based on the semantics of the keywords, determine action information, and / or input information, and / or output information; Based on the action information, and / or input information, and / or output information, determine the standardized SOP point corresponding to each of the target sub-data, and based on the standardized SOP point, obtain the standard operating procedure (SOP) subtree corresponding to each of the target sub-data.

3. The method according to claim 2, characterized in that, The Standard Operating Procedure (SOP) subtree includes multiple standardized SOP points. Obtaining the SOP subtree corresponding to each of the target sub-data based on the standardized SOP points includes: Based on the action information, and / or input information, and / or output information, determine the operating sequence of each of the standardized SOP points; Based on the described operating sequence, the standardized SOP points are combined to obtain a standard operating procedure (SOP) subtree corresponding to each of the target sub-data.

4. The method according to claim 1, characterized in that, The data modality includes text data; the step of performing data transformation on each of the sub-data according to the data modality to obtain the target sub-data includes: The target sub-data is obtained according to the first data transformation process; The first data conversion process includes: Identify the first keyword in the text data; Tag the first keyword to obtain the target sub-data tagged with the first keyword.

5. The method according to claim 4, characterized in that, The data modality includes audio data; the step of performing data transformation on each of the sub-data according to the data modality to obtain target sub-data includes: The target sub-data is obtained according to the second data transformation process; The second data conversion process includes: The sound data is converted to obtain first text information corresponding to the sound data; Based on the first text information, determine the second keyword; The second keyword is labeled to obtain the target sub-data labeled with the second keyword.

6. The method according to claim 5, characterized in that, The data modality includes image data, which includes origin image data composed purely of graphics or non-origin image data containing text information; the step of performing data transformation on each of the sub-data according to the data modality to obtain target sub-data includes: The target sub-data is obtained according to the third data transformation process; The third data conversion process includes: Determine whether the image data is origin image data; If it is determined that the image data is origin image data, then based on the image entities in the image data, the second text information corresponding to the image entities is obtained; Based on the second text information, a third keyword is determined; and the third keyword is marked to obtain the target sub-data marked with the third keyword. If it is determined that the image data is not the origin image data, then the third text information in the image data is obtained; and, based on the image entity in the image data, the fourth text information corresponding to the image entity is obtained. Based on the third text information and / or the fourth text information, determine the fourth keyword; and mark the fourth keyword to obtain the target sub-data marked with the fourth keyword.

7. The method according to claim 6, characterized in that, The data modality includes video data; the step of performing data transformation on each of the sub-data according to the data modality to obtain target sub-data includes: Based on the video data, determine the text sub-data, and / or image sub-data, and / or voice sub-data corresponding to the video data; According to the first data conversion process, and / or the second data conversion process, and / or the third data conversion process, the video data is converted to obtain target sub-data corresponding to the video data.

8. A standard operating procedure system establishment device, characterized in that, The apparatus is used to execute the standard operating procedure system establishment method as described in any one of claims 1-7; and the apparatus comprises: The data acquisition module is used to acquire training data; The modality determination module is used to determine the data modality of each sub-data in the training data, and to perform data transformation on each sub-data according to the data modality to obtain the target sub-data; The program determination module is used to obtain the semantics of keywords in each of the target sub-data, and obtain the standard operating procedure (SOP) subtree corresponding to each of the target sub-data based on the semantics of the keywords; The program determination module is also used to establish an SOP system based on each of the SOP subtrees.

9. A standard operating procedure system establishment device, characterized in that, The device includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the standard operating procedure system establishment method as described in any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer storage medium stores computer instructions, which, when invoked, are used to execute the standard operating procedure system establishment method as described in any one of claims 1-7.

Citation Information

Patent Citations

  • Knowledge graph construction method and device and electronic device

    CN109885698A

  • Entity identification method, apparatus and device, and storage medium

    CN114399701A

  • Information processing method, computer readable storage medium and electronic equipment

    CN114817584A

  • Vehicle fault maintenance method based on knowledge graph

    CN115310630A