Universal panel auxiliary operation method and system

Abstract

The present application discloses a universal panel auxiliary operation method and system, comprising: searching for a corresponding cooking plan according to a target object and a user instruction; searching for cooking steps involving the target object in the cooking plan; determining a target operation template according to the cooking steps, and confirming information about interactive buttons involved according to the target operation template; determining an operation mode and an operation metric of each interactive button according to the cooking steps and the information about the interactive buttons; generating an operation schematic diagram of each interactive button based on the style of the interactive button according to the operation mode and the operation metric, the operation schematic diagram including a mark for the operation mode of the interactive button; splicing and symbolizing each operation schematic diagram in sequence according to the sequence represented by the operation template to obtain a reference legend; and inserting the reference legend before or after the cooking steps involving the target object in the cooking plan according to the cooking plan.

Description

Technical Field

[0001] The present application relates to content recognition and content generation technology, and in particular to a universal panel auxiliary operation method and system. Background Art

[0002] Operation panels are used in various electrical appliances, especially kitchen appliances, which have many functions. For users who do not use them frequently or for functions that are not used frequently, users often need to look through the manual. Although it is convenient to check the manual of some products, which can be queried through QR codes, there is still a certain gap between the operations actually required by users. In addition, for some older devices, users cannot find the manual.

[0003] Based on this, the inventor believes that a more convenient panel-assisted operation solution can be provided to users. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention provides a universal panel auxiliary operation method and system to facilitate users to use equipment.

[0005] On the one hand, an embodiment of the present application provides a general panel auxiliary operation method, including:

[0006] Determine the user command and the target object, the target object being a device having an operation panel;

[0007] Look up the corresponding cooking plan according to the target object and user instructions;

[0008] searching for a cooking step related to a target object in the cooking recipe;

[0009] Determine a target operation template according to the cooking steps, and confirm information of the interactive buttons involved according to the target operation template;

[0010] Determining the operation mode and operation metric of each interactive button according to the cooking steps and information of the interactive buttons;

[0011] According to the operation mode and the operation metric, generating an operation schematic diagram of each interactive button based on the style of the interactive button, the operation schematic diagram including a mark of the operation mode of the interactive button;

[0012] According to the sequence represented by the operation template, each operation schematic diagram is spliced ​​and annotated with symbols in sequence to obtain a reference legend;

[0013] According to the cooking scheme, inserting the reference legend before or after a cooking step involving a target object in the cooking scheme;

[0014] The cooking steps involving the target object generate specific operation methods according to the operation template and integrate them into the content of the target object.

[0015] In some embodiments, the target object is determined based on an image taken by the user, or based on a list of devices bound to the user.

[0016] In some embodiments, when the target object is determined based on an image taken by a user, interactive button features in the image are extracted, interactive button type features and distribution features are determined based on the interactive button features, and the target object is retrieved from a database based on the type features and distribution features.

[0017] In some embodiments, when the target object is determined according to the device list bound by the user, the retrieved cooking plans are filtered according to the devices in the device list, and the cooking plans that can be completed by one or more devices in the device list are filtered out. When there are multiple cooking plans, they are arranged in ascending order according to the number of devices used; the cooking plans are pre-marked with the designed device type;

[0018] Filter out the cooking plan that uses the least amount of equipment, and use the equipment used in the cooking plan as the target object.

[0019] In some embodiments, searching the cooking scheme for cooking steps related to the target object specifically includes:

[0020] Sentences belonging to specific cooking actions are identified from the text of the cooking plan, and the identified sentences are classified into cooking operation types. According to the classification results, corresponding operation templates are found from the operation template library of the target object.

[0021] In some embodiments, the operation template includes information about the interactive buttons required for the operation and the operation sequence between the interactive buttons.

[0022] In some embodiments, the following steps are also included: switching the content displayed in the current cooking plan from the original content and the processed content according to a user switching instruction; or highlighting the processed content from the cooking plan.

[0023] In some embodiments, the operation schematic diagrams are spliced ​​and annotated with symbols in order according to the order represented by the operation template to obtain a reference legend:

[0024] The number of operation schematic diagrams is determined, and the corresponding templates are applied to insert each operation schematic diagram into the operation template according to the order represented in the operation template to generate the reference example diagram.

[0025] In some embodiments, the following steps are also included:

[0026] According to user instructions, generate content links and share them with other users.

[0027] On the other hand, an embodiment of the present application provides a universal panel auxiliary operating system, including:

[0028] Memory, used to store programs;

[0029] The processor is used for loading the program to execute the universal panel auxiliary operating system.

[0030] Through the embodiment of the present application, by determining the user instruction and the target object device, the corresponding cooking plan is searched according to the target object and the user instruction, that is, the present solution can retrieve the cooking plan based on the target object device and the instruction provided by the user; then search the cooking steps involving the target object in the cooking plan; determine the target operation template according to the cooking steps, and confirm the information of the interactive buttons involved according to the target operation template; determine the operation mode and operation measurement of each interactive button according to the information of the cooking steps and the interactive buttons; the solution can find the relevant cooking steps through the cooking plan, and then determine the operation template according to the target object required for the cooking steps, so as to determine the required buttons and the operation amount of the buttons; then, based on the operation mode and operation measurement, generate a An operation diagram for each interactive button is generated, and the operation diagram includes a mark for the operation mode of the interactive button; according to the order represented by the operation template, each operation diagram is spliced ​​and annotated with symbols in order to obtain a reference legend; according to the cooking plan, the reference legend is inserted before or after the cooking steps involving the target object in the cooking plan; for the cooking steps involving the target object, a specific operation mode is generated according to the operation template and integrated into the content of the target object; by generating a reference diagram and inserting the reference diagram into a relevant position in the cooking plan, the content of the cooking plan is combined with the content generated by the plan, so as to guide the user's operation in a specific cooking case, which integrates the cooking plan with the operation plan of the equipment specified or owned by the user, so that the user can learn the operation more conveniently. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments.

[0032] Figure 1 is a flow chart of a method provided by an embodiment of the present application;

[0033] Figure 2 is a schematic diagram of a cooking plan after content generation provided by an embodiment of the present application;

[0034] Figure 3 It is a block diagram of a system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described through the implementation method with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0036] Reference Figure 1 and Figure 2 The embodiment of the present application provides a universal panel auxiliary operation method. It can be understood that the application scenario of this embodiment is that when a user wants to cook, he searches through a mobile phone. The server regenerates the cooking plan based on the user's search request and the device owned by the user or the device the user wants to use, so that the user can obtain a combination of cooking plan and device operation mode. The method includes the following steps:

[0037] S1. Determine the user instructions and the target object, where the target object is a device with an operation panel. In this step, the user can input the desired cooking by text input or voice input. For example, the user can say, "I want to make fried chicken wings." The target object can be determined by the user, or by the server based on the device associated with the user and the content the user wants to cook. In some embodiments, the user can specify the device to be used, and then the server queries the device model bound to the user to determine the target object. The user can also submit the target object to the server through image recognition. Of course, if the user has registered the device, the device that can complete the cooking plan will be determined as the target object based on the retrieved cooking plan.

[0038] S2. Look up the corresponding cooking plan according to the target object and the user instruction.

[0039] When the target object is clear, the cooking scheme is determined based on the target object as a screening condition. The cooking scheme can be marked with a specific tag, such as the type of equipment involved. When the equipment is clear, the cooking scheme can be screened in combination with the equipment.

[0040] In the case of an unclear target object, the cooking scheme is first retrieved, and then the cooking scheme is screened based on the retrieved results and the device bound to the user. It is understandable that the scheme can be adapted according to different situations through this step.

[0041] S3. Searching for cooking steps related to the target object in the cooking plan.

[0042] Some of the implementation plans involve cooking steps that require the use of equipment, such as steaming, baking, frying, roasting, etc. These steps generally carry specific cooking keywords and related temperature and time data. For example, putting in an oven and adjusting to 180 degrees Celsius for 30 minutes is a typical cooking step. The relevant cooking requirements can be determined by identifying keywords in the sentence.

[0043] S4. Determine a target operation template according to the cooking steps, and confirm information of the interactive buttons involved according to the target operation template.

[0044] When the target object is determined, the operation templates for different functions of the target object can be matched. For example, the template for baking needs to adjust the temperature button, mode button and time button. Then, when the target object needs to be used for baking, the information of the interactive buttons involved will be determined according to the template. It can be understood that there can be multiple different modes for a device. For example, a microwave oven can include modes such as thawing, baking, and normal heating.

[0045] Taking a microwave oven as an example, the function template can be as shown in the following table.

[0046] Function template Related keywords Grilling Mode button (baking) + time button Grill, roast, heat, bake… thaw Mode button (defrost) + time button Defrosting, thawing by heating... heating Mode button (high / medium / low heating) + time button Heat on high / medium / low heat, heat...

[0047] It is understandable that for devices with more complex function buttons, the template may involve more buttons, which will not be described one by one here.

[0048] S5. Determine the operation mode and operation metric of each interactive button according to the cooking steps and the information of the interactive buttons.

[0049] Among them, each interactive button can be in different forms. Some buttons are in the form of switches, that is, buttons with only two states, some buttons can be in the form of knobs, and some buttons can be option buttons, that is, only one can be selected among multiple buttons. There will be different operation metrics for different buttons. For example, the metric of a knob is the angle of rotation, and the metric of a button can be the on and off state, or the number of times it is pressed. Through the above information, the generated content is more referenceable.

[0050] S6. Generate an operation diagram of each interactive button based on the style of the interactive button according to the operation mode and the operation metric, wherein the operation diagram includes a mark of the operation mode of the interactive button.

[0051] For example, the heating function of a microwave oven involves a mode knob and a time knob. The operation mode of the two knobs is rotation, and the measurement is determined according to the location of the heating function and the required heating time. In this step, the style of the interactive button can be called, and the corresponding guide icon can be added according to the operation mode of the button to generate an operation diagram. Among them, the mark described in this embodiment can be an operation symbol, etc.

[0052] S7. According to the order represented by the operation template, each operation schematic diagram is spliced ​​and annotated with symbols in order to obtain a reference legend. For the operation template, the operation order between different buttons can be marked. For example, taking a microwave oven as an example, when heating at high heat for 10 minutes, you first need to rotate the mode knob to the high heat mode, and then rotate the time knob to the 10 minute position. There is a sequence between these two interactive buttons. The schematic diagrams can be spliced ​​according to the sequence. In some embodiments, the operation schematic diagrams can be edited with sequence numbers.

[0053] S8. According to the cooking plan, insert the reference legend before or after the cooking step involving the target object in the cooking plan.

[0054] It is understandable that this solution actually adapts the device owned by the user to combine with the cooking plan to generate content. The user operation guide is integrated into the original cooking plan so that the user can directly see the operation method from the cooking plan. Even if the user rarely uses the relevant functions, he can quickly learn the operation method. In this step, reference examples can be inserted before and after the relevant cooking steps to facilitate the user to intuitively confirm the operation method.

[0055] S9, the cooking steps involving the target object, generating a specific operation method according to the operation template and integrating it into the content of the target object.

[0056] In this step, a description of the operation can be further added. The description can be combined with the operation of the interactive button to operate the measurement to generate text and integrate it into the corresponding content. For example, when using a microwave oven to heat for 10 minutes, a certain text template can be used to generate the following description: turn the mode knob to high fire mode, and then turn the time knob about two grids to 10 minutes. The text template can be defined separately according to the device.

[0057] In some embodiments, the target object is determined based on an image taken by the user, or based on a list of devices bound by the user. It is understandable that in some embodiments, the target object can be determined by the user taking a photo. For example, the user can take a photo of a microwave oven panel, and the server determines the type and model of the device taken by the user based on the panel.

[0058] In some embodiments, when the target object is determined based on an image taken by a user, interactive button features in the image are extracted, interactive button type features and distribution features are determined based on the interactive button features, and the target object is retrieved from a database based on the type features and distribution features.

[0059] In this step, when the user chooses to determine the target object by taking an image, the features of the interactive button can be extracted, and then the target object can be retrieved from the database based on the type and distribution characteristics of the interactive button. It is understandable that in the retrieval process, the device registered by the user can also be retrieved first. For devices, their panel designs have different styles, and their button designs also have characteristics. Through the combination of characteristics and distribution, different devices can be effectively determined. For a certain product manufacturer, the design categories are limited, so the recognition error rate will be relatively low. Therefore, the above solution has a better recognition rate.

[0060] In some embodiments, when the target object is determined based on the device list bound by the user, the retrieved cooking plans are filtered according to the devices in the device list, and the cooking plans that can be completed using one or more devices in the device list are filtered out. When there are multiple cooking plans, they are arranged in ascending order according to the number of devices used; the cooking plans are pre-marked with the designed equipment types; the cooking plan with the smallest number of devices is selected, and the equipment used by the cooking plan is used as the target object.

[0061] It is understandable that in this solution, the user does not specify the equipment to be used, but only states what he wants to cook. The system retrieves the corresponding cooking plan based on the content the user wants to cook, and then selects the plan that can be processed with the least user equipment, so as to facilitate the user's cooking. It is understandable that the cooking plan can be marked, and the equipment involved can be marked. Then, after the cooking plan is retrieved through the cooking content, combined with the list of equipment owned by the user, the cooking plan that can be implemented by the user's current equipment can be screened, and then the content of the cooking plan can be generated.

[0062] In some embodiments, searching the cooking scheme for cooking steps related to the target object specifically includes:

[0063] Identify sentences belonging to specific cooking actions from the text of the cooking plan, and classify the identified sentences into cooking operation types, and find the corresponding operation template from the operation template library of the target object according to the classification results. It can be understood that the type of cooking can be identified from the sentence by training the model, so as to match the corresponding template. Of course, the classification of cooking can also be determined by keyword matching, and then the operation template is determined.

[0064] In some embodiments, the operation template includes information about the interactive buttons required for the operation and the operation sequence between the interactive buttons.

[0065] In some embodiments, the following steps are also included: according to the user switching instruction, the content displayed in the current cooking scheme is switched from the original content and the processed content; or the processed content is highlighted from the cooking scheme. It can be understood that by distinguishing the display or allowing the user to switch the content before and after generation through this scheme, it can help users who are familiar with the operation of the device to restore the original cooking scheme. It can also provide convenience for users to find unreasonable operations.

[0066] In some embodiments, the operation schematic diagrams are spliced ​​and annotated with symbols in order according to the order represented by the operation template to obtain a reference legend:

[0067] The number of operation diagrams is determined, and the corresponding template is applied to insert each operation diagram into the operation template according to the order represented in the operation template to generate the reference example diagram. In this embodiment, different templates can be adapted to different numbers of images so that the reference example diagram can adapt to the graphic content of the cooking solution, making the layout more optimized.

[0068] In some embodiments, the following steps are also included:

[0069] According to the user's instructions, the content link is generated and shared with other users. It is understandable that the user can share the solution with family and friends.

[0070] Reference Figure 3 , the embodiment of the present application provides a universal panel auxiliary operating system, including:

[0071] Memory, used to store programs;

[0072] The processor is used for loading the program to execute the universal panel auxiliary operating system.

[0073] It should be noted that the historical operation behavior data involved in this application (including but not limited to the historical operation behavior data of the target user, the historical operation behavior data of the sample user, etc.) are all information and data authorized by the user or fully authorized by all parties.

[0074] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0075] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A general panel auxiliary operation method, characterized in that: include: Determine a user instruction and a target object, wherein the target object is a device having an operation panel; Look up the corresponding cooking plan according to the target object and user instructions; searching for a cooking step related to a target object in the cooking recipe; Determine a target operation template according to the cooking steps, and confirm information of the interactive buttons involved according to the target operation template; Determining the operation mode and operation metric of each interactive button according to the cooking steps and information of the interactive buttons; According to the operation mode and the operation metric, generating an operation schematic diagram of each interactive button based on the style of the interactive button, the operation schematic diagram including a mark of the operation mode of the interactive button; According to the sequence represented by the operation template, each operation schematic diagram is spliced ​​and annotated with symbols in sequence to obtain a reference legend; According to the cooking scheme, inserting the reference legend before or after a cooking step involving a target object in the cooking scheme; Regarding the cooking steps of the target object, a text description of the specific operation method is generated according to the operation template and integrated into the content of the target object.

2. A universal panel auxiliary operation method according to claim 1, characterized in that: The target object is determined according to an image taken by the user, or according to a list of devices bound to the user.

3. A universal panel auxiliary operation method according to claim 2, characterized in that: When the target object is determined based on an image taken by a user, interactive button features in the image are extracted, interactive button type features and distribution features are determined based on the interactive button features, and the target object is retrieved from a database based on the type features and distribution features.

4. A universal panel auxiliary operation method according to claim 2, characterized in that: When the target object is determined according to the device list bound by the user, the retrieved cooking plans are filtered according to the devices in the device list, and the cooking plans that can be completed by one or more devices in the device list are filtered out. When there are multiple cooking plans, they are arranged in ascending order according to the number of devices used; the cooking plans are pre-marked with the designed device type; Filter out the cooking plan that uses the least amount of equipment, and use the equipment used in the cooking plan as the target object.

5. A universal panel auxiliary operation method according to claim 1, characterized in that: The searching for cooking steps related to the target object in the cooking scheme specifically includes: Sentences belonging to specific cooking actions are identified from the text of the cooking plan, and the identified sentences are classified into cooking operation types. According to the classification results, corresponding operation templates are found from the operation template library of the target object.

6. A universal panel auxiliary operation method according to claim 5, characterized in that: The operation template includes information about the interaction buttons required for the operation and the operation sequence between the interaction buttons.

7. A universal panel auxiliary operation method according to claim 1, characterized in that: The following steps are also included: Switching the content displayed on the current cooking plan from the original content to the processed content according to a user switching instruction; Alternatively, the processed content may be highlighted from the cooking recipe.

8. A universal panel auxiliary operation method according to claim 1, characterized in that: According to the order represented by the operation template, each operation schematic diagram is spliced ​​and annotated with symbols in order to obtain a reference legend: The number of operation schematic diagrams is determined, and the corresponding templates are applied to insert each operation schematic diagram into the operation template according to the order represented in the operation template to generate the reference legend.

9. A universal panel auxiliary operation method according to claim 1, characterized in that: The following steps are also included: According to user instructions, generate content links and share them with other users.

10. A universal panel-assisted operating system, characterized in that: include: Memory, used to store programs; A processor, used for loading the program to execute the universal panel auxiliary operating system as described in any one of claims 1-9.

Images