Semantic data processing method and device, middleware, storage medium and product

Abstract

The invention relates to the technical field of vehicles, in particular to a semantic data processing method and device, middleware, a storage medium and a product, and the method comprises the steps: receiving semantic data issued by a sender; analyzing the semantic data according to the semantic protocol of the sender, identifying target application software corresponding to the analyzed semantic data, and converting the analyzed semantic data to generate a control instruction executable by the target application software; the control instruction is sent to the target application software, and the target application software executes the target operation according to the control instruction. Therefore, the problems of high development cost, low efficiency and the like due to the fact that semantic data generated by different sensing end hardware devices and control end software products follow respective semantic protocols and a large amount of technical protocol docking work needs to be carried out during voice development are solved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a method, device, middleware, storage medium and product for processing semantic data. Background Art

[0002] With the rapid development of hardware and software technologies in the field of smart car cockpits, modern smart cars can support increasingly complex human-computer interaction scenarios. Voice control, as one of the key technologies, can already cover the control range of the entire smart cockpit. However, different sensing hardware devices and control software products in the smart cockpit are purchased from different suppliers, with different implementation technologies. The semantic data generated by the voice content also complies with their own semantic protocols, and it is impossible to achieve standard unification.

[0003] Therefore, whenever a new voice control function is needed or improved, it is necessary to coordinate with the technical staff of each supplier to connect the technical protocol and perform customized development for different semantic protocols. This not only increases the initial product development cost, but also requires a lot of manpower and material resources in the later maintenance stage to ensure normal operation. Summary of the invention

[0004] The present application provides a method, device, middleware, storage medium and product for processing semantic data to solve the problem that the semantic data generated by different perception-end hardware devices and control-end software products follow their own semantic protocols, a large amount of technical protocol docking work is required during voice development, and there are problems such as high development cost and low efficiency.

[0005] The first aspect of the present application provides a method for processing semantic data, which is applied to middleware, and the middleware is a background service process, comprising the following steps: receiving semantic data sent by a sender; parsing the semantic data according to the semantic protocol of the sender, and identifying the target application software corresponding to the parsed semantic data, converting the parsed semantic data to generate control instructions executable by the target application software; sending the control instructions to the target application software, wherein the target application software executes the target operation according to the control instructions.

[0006] Optionally, converting the parsed semantic data to generate control instructions executable by the target application software includes: acquiring a target communication protocol; converting the parsed semantic data based on the target communication protocol to generate control instructions executable by the target application software.

[0007] Optionally, before sending the control instruction to the target application software, the method further includes: encapsulating the control instruction into target format data using a communication protocol of the target application software.

[0008] Optionally, before converting the parsed semantic data to generate control instructions executable by the target application software, it also includes: reading all application software of the vehicle; if the target application software does not exist in all application software, intercepting the parsed semantic data and generating an interception prompt.

[0009] Optionally, the target application software is a foreground service process.

[0010] The second aspect of the present application provides a semantic data processing device, which is applied to middleware, and the middleware is a background service process, including: a receiving module, which is used to receive semantic data sent by a sender; a processing module, which is used to parse the semantic data according to the semantic protocol of the sender, and identify the target application software corresponding to the parsed semantic data, and convert the parsed semantic data to generate control instructions executable by the target application software; a sending module, which is used to send control instructions to the target application software, wherein the target application software executes the target operation according to the control instructions.

[0011] Optionally, the processing module is further used to: obtain a target communication protocol; convert the parsed semantic data based on the target communication protocol to generate control instructions executable by the target application software.

[0012] Optionally, the semantic data processing device further includes: an encapsulation module, which is used to encapsulate the control instruction into target format data using the communication protocol of the target application software before sending the control instruction to the target application software.

[0013] Optionally, the semantic data processing device also includes: a reading module, which is used to read all application software installed in the vehicle before converting the parsed semantic data to generate control instructions executable by the target application software; an interception module, which is used to intercept the parsed semantic data and generate an interception prompt when the target application software does not exist in all the application software.

[0014] Optionally, the target application software is a foreground service process.

[0015] The third aspect of the present application provides a middleware, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a method for processing semantic data as described in the above embodiment.

[0016] The fourth aspect of the present application provides a computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed, it is used to implement the semantic data processing method as described in the above embodiment.

[0017] The fifth aspect of the present application provides a computer program product, including: a computer program or instructions, which, when executed, implements the method for processing semantic data as described in the above embodiment.

[0018] Therefore, this application has at least the following beneficial effects:

[0019] The embodiment of the present application receives semantic data sent by the sender and converts various heterogeneous semantic data into unified control instructions executable by the application software. There is no need to perform separate protocol docking for each different speech recognition service or third-party application software, which greatly simplifies the development and integration process. It not only improves the reusability of product functions, but also greatly reduces the development cost of enterprise smart cockpit products.

[0020] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A flowchart of a method for processing semantic data provided according to an embodiment of the present application;

[0023] Figure 2 A communication relationship diagram between the middleware, the speech recognition service, and the operating system provided according to an embodiment of the present application;

[0024] Figure 3 An example diagram of a method for processing semantic data provided according to an embodiment of the present application;

[0025] Figure 4 is a block diagram of a device for processing semantic data according to an embodiment of the present application;

[0026] Figure 5 It is a schematic diagram of the structure of the middleware according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0028] The following describes the semantic data processing method, device, middleware and storage medium of the embodiment of the present application with reference to the accompanying drawings. In response to the problems mentioned in the above background technology, the present application provides a method for processing semantic data, in which the semantic data sent by the sender is received, and various heterogeneous semantic data are converted into unified control instructions executable by the application software. There is no need to perform separate protocol docking for each different speech recognition service or third-party application software, which greatly simplifies the development and integration process, not only improves the reusability of product functions, but also greatly reduces the development cost of enterprise smart cockpit products.

[0029] Specifically, Figure 1 A flowchart of a method for processing semantic data provided in an embodiment of the present application.

[0030] like Figure 1 As shown, the semantic data processing method is applied to the middleware, which is a background service process, and includes the following steps:

[0031] In step S101, semantic data sent by a sender is received.

[0032] It can be understood that the speech recognition service includes hardware equipment for capturing speech and software services for converting speech into semantics. The speech recognition service is connected to the smart car cockpit system of this application, and the speech recognition service uses respective semantic protocols to generate semantic data.

[0033] In step S102, the semantic data is parsed according to the semantic protocol of the sender, and the target application software corresponding to the parsed semantic data is identified, and the parsed semantic data is converted to generate control instructions executable by the target application software.

[0034] The target application software is a foreground service process, which displays an interface in the foreground and interacts with the user.

[0035] In the embodiment of the present application, different speech recognition services may use different semantic protocols to express the same intention or command. Therefore, the embodiment of the present application can parse various heterogeneous input semantic data and output them as unified instruction operations executable by application software. In the actual execution process, the embodiment of the present application calculates the target application software that needs to process the instructions contained in the semantic data according to the business rules.

[0036] In one embodiment of the present application, the parsed semantic data is converted to generate control instructions executable by the target application software, including: obtaining the target communication protocol; converting the parsed semantic data based on the target communication protocol to generate control instructions executable by the target application software.

[0037] For ease of understanding, the embodiments of the present application can be combined with Figure 2 Describe the specific implementation method in detail, such as Figure 2 As shown, 102 is a middleware for executing the processing method of the semantic data of the embodiment of the present application, which runs as an independent process in the operating system. The semantic processing middleware is a background service process, and there is no foreground interface to interact with the user. 100-1 to 100-N refer to the application software independently developed by the current company, which runs in the foreground as an independent process and can interact with the user in the foreground display interface. The semantic data communication between 101-1 to 101-N and the semantic middleware complies with the custom inter-process communication protocol. The custom communication protocol usually includes a control information description, a data structure or format, and a description of the order of events. 101-1 to 101-N refer to the application software that is not independently developed by the current company and is connected to the smart car cockpit system, here referred to as third-party application software, which can interact with the user in the foreground display interface, such as the common ones: WeChat, NetEase Cloud Music, etc. The third-party application software may have an information communication protocol specified by the third-party company. In this case, the semantic processing middleware needs to process the semantic data according to the information communication protocol provided by the third-party company. 103-1 to 103-N refer to the voice recognition service connected to the smart car cockpit system. 104 refers to the operating system of smart cars. Figure 1 All processes and services in the system run on the operating system. Semantic processing middleware services need to be adapted and developed on different operating systems to achieve the portability and functional reusability of semantic middleware services on different systems.

[0038] The embodiment of the present application can calculate the application software 100 - 1 to 100 -N or 101 - 1 to 101 -N that needs to process the instructions included in the semantic data according to the business rules.

[0039] For self-developed application software, the semantic processing middleware 102 uses a custom communication protocol to repackage the semantic data output into a unified instruction operation executable by the application software, and sends it to one or more application software 100-1 to 100-N. For third-party application software, the semantic processing middleware 102 uses the information communication protocol specified by the third-party company to repackage the semantic data output into a unified instruction operation executable by the application software, and sends it to one or more third-party application software 101-1 to 101-N.

[0040] In one embodiment of the present application, before converting the parsed semantic data to generate control instructions executable by the target application software, it also includes: reading all application software of the vehicle; if the target application software does not exist in all application software, intercepting the parsed semantic data and generating an interception prompt.

[0041] It is understandable that the embodiment of the present application can read the list of all installed application software in the current smart car operating system, check whether the target application software corresponding to the parsed semantic data exists in the list, and if it is found that the target application software does not exist in the vehicle's operating system, that is, when the user attempts to call an uninstalled or unavailable application through a voice command, the semantic processing middleware will intercept the semantic data and generate a corresponding interception prompt. In the actual execution process, the prompt can be fed back to the user in the form of voice, informing the user that the specified application software is not found or cannot be used, avoiding potential errors or abnormal situations caused by attempting to call a non-existent application. If the target application software exists, the subsequent data conversion and instruction generation process will continue, and finally the control instruction will be sent to the corresponding target application software.

[0042] In step S103, a control instruction is sent to the target application software, wherein the target application software performs a target operation according to the control instruction.

[0043] In one embodiment of the present application, before sending the control instruction to the target application software, it also includes: using the communication protocol of the target application software to encapsulate the control instruction into target format data.

[0044] In the embodiment of the present application, for the independently developed application software (100-1 to 100-N), the semantic processing middleware repackages the control instructions based on a custom data protocol (such as a standard JSON format). For third-party application software (101-1 to 101-N), if it is necessary to encapsulate according to the data protocol of a third-party company, the control instructions are repackaged according to their specific data structure and communication protocol. Thus, a smooth and efficient conversion is ensured from the user's voice input to the final operation execution, so that the control instructions can be correctly transmitted and processed, improving the response speed and user experience of the smart car cockpit system.

[0045] Combine the following Figure 3 The method for processing semantic data in the embodiment of the present application is described in detail, including the following steps:

[0046] Step 1: The human language audio data in the physical world, i.e., voice input, is collected by hardware devices (such as microphones, etc.).

[0047] Step 2: The speech recognition service recognizes the collected speech data, generates semantic data based on the semantic protocol, and sends the semantic data to the semantic processing middleware.

[0048] Step 3: The semantic processing middleware can actively query the semantic recognition service to obtain semantic data, or passively wait to receive semantic data.

[0049] Step 4: The semantic processing middleware parses the semantic data based on the semantic protocol of the semantic data sender.

[0050] Step 5: The semantic processing middleware determines whether the semantic data needs to be sent to the self-developed application software.

[0051] Step 6: According to the judgment result in step 5, if the answer is yes, the semantic processing middleware repackages the control instruction into standard JSON format data based on the self-developed data protocol and sends it to the self-developed application software.

[0052] Step 7: The self-developed application software receives the JSON format data and performs corresponding operations to complete the data consumption.

[0053] Step 8: According to the judgment result in step 5, if it is no, the semantic processing middleware needs to read the system-related configuration, query the target third-party application software, and determine whether it is installed on the current operating system.

[0054] Step 9: According to the judgment result in step 8, if it is no, the semantic processing middleware intercepts the semantic data and uses voice to remind the user that the third-party application software does not exist in the operating system. The data consumption is completed.

[0055] Step 10: According to the judgment result in step 8, if yes, the semantic processing middleware determines whether it is necessary to re-encapsulate the control instruction according to the data protocol provided by the third-party application software.

[0056] Step 11: According to the judgment result in step 10, if it is no, the semantic processing middleware repackages the control instruction into standard JSON format data based on the self-developed data protocol.

[0057] Step 12: The semantic processing middleware uses a self-developed data protocol communication method to send JSON format data to third-party application software.

[0058] Step 13: The third-party application receives the JSON format data and performs corresponding operations, completing the data consumption.

[0059] Step 14: According to the judgment result in step 10, if yes, the semantic processing middleware repackages the control instruction based on the third-party application software data protocol.

[0060] Step 15: The semantic processing middleware uses the third-party application software data protocol communication method to send the encapsulated data to the third-party application software.

[0061] Step 16: The third-party application software receives the protocol encapsulated data and performs corresponding operations, completing the data consumption.

[0062] According to the method for processing semantic data proposed in the embodiment of the present application, semantic data sent by the sender is received, and various heterogeneous semantic data are converted into unified control instructions executable by the application software. There is no need to perform separate protocol docking for each different speech recognition service or third-party application software, which greatly simplifies the development and integration process, not only improves the reusability of product functions, but also greatly reduces the development cost of enterprise smart cockpit products.

[0063] Next, a device for processing semantic data according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0064] Figure 4 It is a block diagram of a device for processing semantic data according to an embodiment of the present application.

[0065] like Figure 4 As shown, the semantic data processing device 10 includes: a receiving module 100, a processing module 200 and a sending module 300.

[0066] Among them, the receiving module 100 is used to receive the semantic data sent by the sender; the processing module 200 is used to parse the semantic data according to the semantic protocol of the sender, and identify the target application software corresponding to the parsed semantic data, and convert the parsed semantic data to generate control instructions executable by the target application software; the sending module 300 is used to send control instructions to the target application software, wherein the target application software executes the target operation according to the control instructions.

[0067] In one embodiment of the present application, the processing module is further used to: obtain a target communication protocol; convert the parsed semantic data based on the target communication protocol to generate control instructions executable by the target application software.

[0068] In one embodiment of the present application, the semantic data processing device further includes: an encapsulation module, which is used to encapsulate the control instruction into target format data using the communication protocol of the target application software before sending the control instruction to the target application software.

[0069] In one embodiment of the present application, the semantic data processing device 10 also includes: a reading module, which is used to read all application software of the vehicle before converting the parsed semantic data to generate control instructions executable by the target application software; an interception module, which is used to intercept the parsed semantic data and generate an interception prompt when the target application software does not exist in all the application software.

[0070] In one embodiment of the present application, the target application software is a foreground service process.

[0071] It should be noted that the above explanation of the embodiment of the method for processing semantic data is also applicable to the apparatus for processing semantic data of this embodiment, and will not be repeated here.

[0072] According to the semantic data processing device proposed in the embodiment of the present application, the semantic data sent by the sender is received, and various heterogeneous semantic data are converted into unified control instructions executable by the application software. There is no need to perform separate protocol docking for each different speech recognition service or third-party application software, which greatly simplifies the development and integration process, not only improves the reusability of product functions, but also greatly reduces the development cost of enterprise smart cockpit products.

[0073] Figure 5 A schematic diagram of the structure of the middleware provided in the embodiment of the present application. The middleware may include:

[0074] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0075] When the processor 502 executes the program, the method for processing semantic data provided in the above embodiment is implemented.

[0076] Furthermore, the middleware also includes:

[0077] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0078] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0079] The memory 501 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0080] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0081] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0082] The processor 502 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0083] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for processing semantic data as described above is implemented.

[0084] An embodiment of the present application also provides a computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed, it is used to implement the above-mentioned method for processing semantic data.

[0085] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0086] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0087] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0088] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0089] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0090] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for processing semantic data, characterized in that: The method is applied to middleware, which is a background service process, and includes the following steps: Receive semantic data sent by the sender; Parsing the semantic data according to the semantic protocol of the sender, identifying the target application software corresponding to the parsed semantic data, and converting the parsed semantic data to generate control instructions executable by the target application software; The control instruction is sent to the target application software, wherein the target application software performs a target operation according to the control instruction.

2. The method for processing semantic data according to claim 1, characterized in that: Converting the parsed semantic data to generate control instructions executable by the target application software includes: Get the target communication protocol; The parsed semantic data is converted based on the target communication protocol to generate control instructions executable by the target application software.

3. The method for processing semantic data according to claim 2, characterized in that: Before sending the control instruction to the target application software, the method further includes: The control instruction is encapsulated into the target format data using the communication protocol of the target application software.

4. The method for processing semantic data according to claim 1, characterized in that: Before converting the parsed semantic data to generate control instructions executable by the target application software, the method further includes: Read all application software installed in the vehicle; If the target application software does not exist in all the application software, the parsed semantic data is intercepted and an interception prompt is generated.

5. The method for processing semantic data according to any one of claims 1 to 4, characterized in that: The target application software is a foreground service process.

6. A semantic data processing device, characterized in that: The device is applied to middleware, which is a background service process and includes: A receiving module, used for receiving semantic data sent by the sender; A processing module, used to parse the semantic data according to the semantic protocol of the sender, identify the target application software corresponding to the parsed semantic data, and convert the parsed semantic data to generate control instructions executable by the target application software; The sending module is used to send the control instruction to the target application software, wherein the target application software performs a target operation according to the control instruction.

7. The semantic data processing device according to claim 6, characterized in that: The processing module is further configured to: Get the target communication protocol; The parsed semantic data is converted based on the target communication protocol to generate control instructions executable by the target application software.

8. A middleware, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for processing semantic data according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed, the method for processing semantic data described in any one of claims 1 to 5 is implemented.

10. A computer program product comprising: A computer program or instruction, characterized in that when the computer program or instruction is executed, it implements the method for processing semantic data described in any one of claims 1-5.

Images