Method and terminal for converting cooking information in a home based on dual-device cooperation

Abstract

The application discloses a household cooking information conversion processing method and terminal based on double-device cooperation, relates to the technical field of intelligent terminal cooking information processing, and comprises the following steps: extracting and analyzing the visual features of food materials in the key frames of a target cooking video, constructing a structured food material list and an appliance list; generating a service type mapping logic on a display device, providing a selectable service path; triggering a corresponding service chain according to user selection, and converting video content semantics into specific execution instructions; establishing a task flow transfer channel between the display device and a mobile terminal; when it is detected that a user selects a self-made path for purchasing food materials, the display device pushes the structured food material list to the mobile terminal bound to the user, and the mobile terminal calls an e-commerce platform interface to generate a to-be-confirmed order after receiving; the mobile terminal preloads step data of a complete recipe and cuts the step data into independent voice segments according to a cooking process; and the next instruction sequence of a user is triggered to be broadcasted through voice recognition and listening; and hands-free operation in the whole process can be realized.

Description

Technical Field

[0001] This invention relates to the field of smart terminal cooking information processing technology, and in particular to a method, apparatus, smart terminal and storage medium for converting and processing home cooking information based on dual-device collaboration. Background Technology

[0002] In existing technologies, smart display devices such as TV cooking programs are disconnected from actual cooking practices. After watching a TV cooking program, users need to manually browse their phones to find the corresponding recipes, manually compile shopping lists and record the required ingredients, and frequently manually operate their phones to check the cooking steps during the cooking process.

[0003] The existing smart display devices and mobile terminals for sharing cooking information have the following problems: 1) When cooking, you need to manually browse the recipes on your mobile phone, which is inconvenient and poses a safety hazard; 2) Shopping lists need to be organized manually, and ingredients are easily missed; 3) Recording dietary health data depends on manual input; 4) There is a disconnect between TV food programs and cooking practice.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method, device, smart terminal and storage medium for home cooking information conversion and processing based on dual-device collaboration, which addresses the shortcomings of the prior art. The present invention achieves a seamless conversion of food preparation "from seeing to doing" through dual-device collaboration, and provides convenience for users by creating a closed loop of TV content → food identification → one-click shopping, while ensuring the safety and convenience of the cooking process.

[0006] The technical solution adopted by this invention to solve the problem is as follows: A method for converting and processing home cooking information based on dual-device collaboration, comprising: Extract visual features of ingredients from keyframes of the target cooking video, parse the ingredient name entities in the video subtitles or audio, identify the types of ingredients required for the target dish, and extract and identify the types of cooking utensils through keyframes to obtain the required cooking equipment, and construct a structured ingredient list and utensil list. Based on the identified structured ingredient list, service type mapping logic is generated on the display device, providing optional service paths of ordering takeout for immediate enjoyment and purchasing ingredients to cook at home; the corresponding service chain is triggered according to the user's selection, and the semantics of the video content are converted into specific execution instructions; A task flow channel is established between the display device and the mobile terminal. When the user selects the path of purchasing ingredients to make their own food, the display device pushes the structured ingredient list to the user's bound mobile terminal. After receiving the list, the mobile terminal calls the e-commerce platform interface to generate an order to be confirmed. The mobile terminal preloads the complete recipe's step data and cuts it into independent voice segments according to the cooking process; it also uses voice recognition to listen to the user's next instruction and trigger the broadcast sequentially; at the same time, it adopts a keyword-based wake-up interruption questioning mechanism. When a cooking-related question is detected, the broadcast of the current step is paused, the answer is matched based on the recipe knowledge base, and the answer is returned via voice, realizing hands-free operation throughout the entire process.

[0007] The aforementioned method for converting and processing home cooking information based on dual-device collaboration further includes, after the step of the mobile terminal preloading the step data of the complete recipe and cutting it into independent audio segments according to the cooking process, the method further includes: Once the cooking process is detected as complete, the system uses the identified dish name to generate a social media copywriting template, automatically filling in the dish name and cooking text, and automatically synchronizing it to the user's preset social media platforms for posting.

[0008] The aforementioned method for converting and processing home cooking information based on dual-device collaboration further includes the following steps in constructing a structured ingredient list and utensil list: Based on the constructed structured ingredient list, a health analysis model using a preset ingredient weight and nutrient element conversion matrix converts cooking data into health data, enabling automated estimation of the nutritional components of dishes. Combined with the automatic storage of user health records and the periodic report generation mechanism, it provides users with continuous tracking and trend analysis of nutritional intake, assisting users in scientific dietary health management.

[0009] The aforementioned method for converting and processing home cooking information based on dual-device collaboration, wherein the mobile terminal preloads the step data of the complete recipe and cuts it into independent audio segments according to the cooking process; and the step of triggering the broadcast by listening to the user's next instruction sequence through voice recognition further includes: By adding a voiceprint fingerprint recognition unit to the audio input layer, the speaker's voiceprint feature vector is extracted simultaneously when voice commands are collected, and a voiceprint database of commonly used family members is established to realize real-time binding of voice control commands with the identity of the operating user.

[0010] The aforementioned method for converting and processing home cooking information based on dual-device collaboration, wherein the mobile terminal preloads the step data of the complete recipe and cuts it into independent audio segments according to the cooking process; and the step of triggering the broadcast by listening to the user's next instruction sequence through voice recognition further includes: A real-time shared buffer for cooking status is pre-established, a distributed status synchronization architecture based on a home LAN or Bluetooth network is constructed, and a shared memory pool is set up to record key cooking status parameters; the shared buffer adopts a publish-subscribe mechanism to ensure that the terminal devices of all authorized family members can read the latest key cooking status parameters in real time. The key cooking status parameters recorded include: the current step number, the real-time temperature inside the pot transmitted by the smart cookware sensor, the cumulative weight of the ingredients and seasonings added, and the remaining countdown for the current step.

[0011] The aforementioned method for converting and processing home cooking information based on dual-device collaboration, wherein the mobile terminal preloads the step data of the complete recipe and cuts it into independent audio segments according to the cooking process; and the step of triggering the broadcast by listening to the user's next instruction sequence through voice recognition further includes: A dynamic mapping table of user preferences is pre-built to create an independent cooking habit profile for each family member, recording personalized parameters including taste preferences, saltiness threshold, and heat control mode; Based on the voiceprint recognition results, the current user's preference profile is dynamically loaded, and the logic for generating subsequent voice guidance content is adjusted in real time.

[0012] The aforementioned method for converting and processing home cooking information based on dual-device collaboration, wherein the mobile terminal preloads the step data of the complete recipe and cuts it into independent audio segments according to the cooking process; and the step of triggering the broadcast by listening to the user's next instruction sequence through voice recognition further includes: When the voiceprint feature extraction module detects that the similarity between the voiceprint feature of the voice command and the previous command is lower than a preset threshold, the arbitration process is automatically triggered: the TTS broadcast of the current step is paused, and the current cooking status is read from the shared buffer to generate a current status summary. The generated current status summary will be broadcast to the successor for confirmation via voice. At the same time, switch to the preference profile of the newly identified user and adjust the parameter suggestions for subsequent steps.

[0013] A home cooking information conversion and processing device based on dual-device collaboration, wherein the device includes: The video semantic recognition module is used to extract the visual features of ingredients in key frames of the target cooking video, parse the ingredient name entities in the video subtitles or audio, identify the types of ingredients required for the target dish, and extract and identify the types of cooking utensils through key frames to obtain the required cooking equipment and construct a structured ingredient list and utensil list. The service type mapping and distribution module is used to generate service type mapping logic on the display device based on the identified structured ingredient list, providing optional service paths of ordering takeout for immediate consumption and purchasing ingredients to make the food; it triggers the corresponding service chain according to the user's selection, and converts the semantics of the video content into specific execution instructions; The cross-screen execution module is used to establish a task flow channel between the display device and the mobile terminal. When the display device detects that the user selects the path of purchasing ingredients to make the food, it pushes the structured ingredient list to the user's bound mobile terminal. After receiving the list, the mobile terminal calls the e-commerce platform interface to generate an order to be confirmed. The segmented hands-free voice interaction module is used to preload the complete recipe's step data into mobile terminals and cut it into independent voice segments according to the cooking process; it also uses voice recognition to listen to the user's next instruction and trigger the broadcast sequentially; at the same time, it adopts a keyword-based wake-up interruption questioning mechanism. When a cooking-related question is detected, the broadcast of the current step is paused, the answer is matched based on the recipe knowledge base, and the answer is returned via voice, realizing hands-free operation throughout the entire process.

[0014] A smart terminal includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors, the one or more programs comprising the method for performing any one of the methods.

[0015] A computer-readable storage medium, wherein, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform any of the methods described herein.

[0016] The beneficial effects of this invention are as follows: This invention provides a method, device, smart terminal, and storage medium for converting and processing home cooking information based on dual-device collaboration. This invention adopts a cross-device cooking service chain, which can realize a closed loop of TV content → ingredient recognition → one-click shopping; and establishes a voice-driven hands-free recipe guidance system; it can also realize automated social sharing and health recording of cooking results. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the home cooking information conversion and processing method based on dual-device collaboration provided in Embodiment 1 of the present invention.

[0018] Figure 2 A schematic diagram of an embodiment of a home cooking information conversion and processing device based on dual-device collaboration provided by the present invention.

[0019] Figure 3 This is a block diagram illustrating the internal structure of a smart terminal provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0021] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0022] People sometimes see delicious food on TV and then use their phones to look up the ingredients and cooking methods. However, TV cooking programs are disconnected from actual cooking. After watching a cooking show, users need to manually search for recipes on their phones, compile shopping lists, and record the required ingredients. Furthermore, they frequently need to manually check their phones to view the cooking steps during the cooking process.

[0023] This fragmented state leads to three specific technical problems: First, the efficiency of cross-media information conversion is low. Television video content cannot be automatically converted into structured cooking data, requiring users to rely on memory or manual search to complete the "from seeing to knowing" conversion, easily overlooking key ingredients or steps. Second, cross-device task collaboration is lacking. Users need to obtain cooking inspiration on the TV, then manually rebuild the shopping list on their mobile phone and jump to an e-commerce application, resulting in a fragmented and cumbersome process. Third, there are safety hazards during the cooking process. Traditional mobile phone recipes require touch operation, but during cooking, users' hands are often covered in grease or food, making mobile phone operation inconvenient, posing a risk of device contamination and slippage, and making it difficult to obtain targeted guidance (such as adjusting heat) during cooking. The root cause of these technical problems lies in the lack of a technical architecture that automatically maps video semantics into executable services, and the lack of a cross-device task flow protocol that supports hands-free operation.

[0024] To address this, this application proposes a method for converting and processing home cooking information based on dual-device collaboration. For example... Figure 1 As shown, an embodiment 1 of the present invention provides a method for converting and processing home cooking information based on dual-device collaboration, comprising the following steps: S100. Extract the visual features of ingredients from the key frames of the target cooking video, parse the ingredient name entities in the video subtitles or audio, identify the types of ingredients required for the target dish, and identify the types of cooking utensils by extracting key frames to obtain the required cooking equipment, and construct a structured ingredient list and utensil list. In this embodiment, the target cooking video refers to a television program source or streaming video clip that the user is watching and that contains the complete process of making a dish. It is the input source for all subsequent automated processing.

[0025] In this embodiment, keyframes refer to a number of representative still image frames extracted from the target cooking video based on scene changes, time intervals, or salience detection. These frames are used to carry visual information such as ingredients and utensils, thus avoiding the computational overhead of processing the entire video.

[0026] In this embodiment, the visual features of food ingredients refer to the feature vectors of food objects in keyframes in dimensions such as color, texture, and outline, which are used to identify the food category through machine learning models, such as "tomato" and "onion".

[0027] In this embodiment, the food ingredient name entity refers to a noun or noun phrase that specifically refers to the food ingredient, extracted from the embedded subtitles or speech recognition results of the target cooking video through natural language processing. For example, "beef tenderloin" is extracted from "we need to prepare 200 grams of beef tenderloin".

[0028] The structured ingredient list and utensil list in this embodiment refer to a machine-readable data object containing fields such as ingredient name, recommended dosage, and unit, and fields such as utensil name and specifications. It is stored in JSON or XML format to provide standardized input for downstream automation services.

[0029] In the specific implementation of step S100, firstly, a convolutional neural network (CNN) can be used to acquire the cooking video being watched by the user in real time, capturing key frames periodically or dynamically. Image recognition is then performed on the key frames to extract the visual features of the ingredients, such as the red, round features of tomatoes and the oval features of eggs. These are then compared with an ingredient knowledge graph to determine the types of ingredients appearing in the video, such as tomatoes and eggs. Simultaneously, image-to-text recognition (OCR) is used to embed subtitles into the video, and automatic speech recognition (ASR) is used to transcribe speech. Named entity recognition (NER) algorithms are then used to extract the names of the mentioned ingredients and their possible quantities. The visual recognition results are then fused and disambiguated with the semantic extraction results, such as distinguishing between "green bell pepper" and "red bell pepper," ultimately generating a list of the "ingredient types" required for the dish.

[0030] Based on a cooking utensil database, the system matches images of kitchen utensils in keyframes, detecting and classifying utensils such as pots, stoves, and ovens in the video feed. It then infers the required heat source type (e.g., a gas stove for a wok, an electric heat source for an oven), creating a list of necessary "cooking equipment." Finally, a structured data package is generated, containing information on ingredient types, estimated weights, and required utensils. The entire process outputs a structured list of ingredients and a structured list of utensils, transforming unstructured video signals into machine-parseable structured cooking data.

[0031] This embodiment addresses the pain point of low efficiency in cross-media information conversion in existing technologies; it automates the process from seeing food to knowing the required ingredients, eliminating reliance on human memory and preventing the omission of key ingredients due to forgetting, thus completely eliminating the need for manual memory and manual search.

[0032] Furthermore, this invention integrates visual and semantic dual-channel recognition, which is more comprehensive than relying solely on human memory. It can also extract essential ingredients and utensils for cooking that may not be explicitly stated in subtitles or audio, thus improving the completeness and accuracy of information extraction.

[0033] Moreover, the structured list of ingredients and utensils generated by this invention becomes a unified data exchange format, enabling the video semantics on the TV to be directly consumed by subsequent service chains such as ordering and recipe advancement, laying a data foundation for subsequent service flows.

[0034] S200: Based on the identified structured ingredient list, generate service type mapping logic on the display device to provide optional service paths of ordering takeout for immediate enjoyment and purchasing ingredients to make the food; trigger the corresponding service chain according to the user's selection, and convert the semantics of the video content into specific execution instructions; The display device in this embodiment refers to a smart TV or a screen mirroring display terminal, which is the decision-making and interaction entry point for users when watching cooking videos and generating a desire to cook; this embodiment uses a smart TV as an example for explanation.

[0035] The service type mapping logic in this embodiment refers to a set of pre-set decision rules or mapping tables that can map the semantics of a dish plus a structured ingredient list to specific service categories that users can choose from. For example, it can map two processing paths: one for takeout and one for purchasing fresh produce and preparing the food themselves.

[0036] The optional service paths in this embodiment refer to the different subsequent processing methods presented to the user on the display interface, generated by the service type mapping logic. Here, they refer to the two specific service branches: "Buy takeout and enjoy immediately" and "Buy ingredients and make your own".

[0037] The service chain in this embodiment refers to a sequence of backend task calls triggered after the user makes a path selection, such as starting the food delivery platform service process or starting the e-commerce procurement service process.

[0038] The execution instructions in this embodiment refer to specific and operable control commands that are ultimately issued by the display device or executed in coordination with other devices, such as "injecting the ingredient list of this dish into the search box of the food delivery platform" or "packing the ingredient list into a shopping cart and requesting to send it to the mobile device".

[0039] In the specific implementation of step S200, the local service mapping table is queried based on the structured ingredient list output in step S100 to determine whether the dish supports delivery (based on geographical location and matching with the merchant's menu) or home cooking (based on ingredient availability). A layered selection interface is then rendered on the TV. When the user selects the "home cooking" option via remote control or voice, the TV generates a task token containing ingredient SKU information and the user's identity token, which is transmitted to the mobile device via an encrypted cross-device channel (such as an authorization protocol based on OAuth 2.0). After the mobile device parses the token, it is automatically filled into the shopping cart of the e-commerce application. The user only needs to complete biometric identification or password confirmation on the mobile device to complete the purchase loop, avoiding the risk of omissions from manually compiling the list.

[0040] This step S200 addresses the shortcomings of lack of cross-device task collaboration and broken operation processes, and brings the following benefits: 1) Instant conversion of intent, reducing operational breakpoints: After a user has a desire to cook on the TV, they no longer need to switch their mind and device to think about "what should I do next". All options are presented on the same screen, instantly turning inspiration into an actionable task.

[0041] 2) Provide differentiated service paths to enhance user experience: Through intelligent mapping, the system can provide a dual choice of "order takeout and eat on the spot" and "buy ingredients and cook at home" based on the complexity of the dishes and the user's historical preferences. This satisfies both the pursuit of convenience and the desire to cook. Moreover, the entire decision-making and triggering process is completed on the same interface without interruption.

[0042] 3) Establishing the instruction source for multi-device task collaboration: The execution instructions generated by S200 are the triggers for subsequent cross-device collaboration, making the TV not only a content player, but also a central node for home service scheduling, which in fact realizes the automatic mapping technology architecture from "video semantics to executable services".

[0043] S300. Establish a task flow channel between the display device and the mobile terminal. When the user selects the path of purchasing ingredients to make their own food, the display device pushes the structured ingredient list to the mobile terminal bound to the user. After receiving the list, the mobile terminal calls the e-commerce platform interface to generate an order to be confirmed. The task flow channel in this embodiment refers to a secure, low-latency data push link established between the display device (TV) and the mobile terminal (phone). It can be implemented through a direct LAN connection, cloud message queue, or Bluetooth / Near Field Communication, and is used to transmit structured task data packets.

[0044] In this embodiment, the mobile terminal refers to a portable device bound to the user, such as the user's smartphone.

[0045] The e-commerce platform interface in this embodiment refers to the open API provided by a third-party fresh food e-commerce platform or comprehensive shopping platform. By calling this interface, the mobile terminal converts the names and quantities of ingredients in the ingredient list into product items and shopping cart data that the platform can recognize.

[0046] The pending order refers to a pre-filled order generated by the system after automatically filling the e-commerce platform with the ingredient list. It includes the selected goods, quantity and estimated price, but payment has not yet been submitted and is waiting for the user's final confirmation and authorization.

[0047] Step S300 implements a cross-device task flow protocol that is triggered on the TV and seamlessly continued on the mobile phone. When the user selects the "buy ingredients to cook" path on the TV, the system of this embodiment first detects whether the user's pre-bound mobile terminal is online and accessible, and establishes a task flow channel. The display device packages the structured ingredient list (which may also include dosage and personalized recommendation information) generated in S100 into task data and pushes it to the mobile terminal through this flow channel. After receiving the list, the background service on the mobile terminal automatically calls the interface of the locally installed e-commerce application or cloud e-commerce platform to parse each ingredient in the list into a corresponding product, add it to the shopping cart, and generate a complete order to be confirmed. At this time, the mobile phone only needs to pop up a notification or display an order confirmation page. The user only needs to check at a glance and authorize payment to complete the complete closed loop from TV cooking program to ingredients arriving at home.

[0048] In this embodiment of the invention, step S200 constructs a semantic service gateway on the display device (smart TV) that converts video semantics into commercial services. After S100 generates a structured ingredient list, the TV's operating system loads service type mapping logic based on the list and dish information, generating an interactive overlay containing two buttons or options: "Buy takeout and enjoy immediately" and "Buy ingredients to make your own." The user can select one via remote control, voice, or gesture. Upon receiving the selection, the system converts the intention along with the structured list into a specific execution instruction: if "takeout" is selected, the instruction triggers the takeout application to search for the corresponding dish and place an order; if "make your own" is selected, the instruction encapsulates the ingredient list and task trigger signal, preparing to transmit it to the mobile terminal. This process seamlessly distributes the semantics of "want to make / eat this dish" contained within the video content to different real-world service paths.

[0049] Step S300 addresses the lack of cross-device task collaboration and the resulting cumbersome operations and potential security risks. Its benefits include: 1) It can eliminate the operational breaks of manually rebuilding the list across devices: Users no longer need to manually enter the names of ingredients one by one on their mobile phones, organize the list, and then jump to multiple applications. The whole process is initiated and filled by the system automatically, and the operation process is completed in one go, which greatly shortens the path from "wanting to buy" to "placing an order".

[0050] 2) Ensuring hand and equipment hygiene, reducing safety hazards: In traditional methods, if users are preparing ingredients in the kitchen while trying to recall what they need, and then operate their phones with oily hands, it can lead to equipment contamination and the risk of slipping. However, the solution in this invention allows users to remotely place orders while watching TV, and even push the shopping list to their phones before cooking begins. Users can perform a simple "glance confirmation" operation with clean hands, significantly reducing the need to touch their phones during cooking and improving kitchen safety and hygiene.

[0051] 3) This invention can build a complete cross-device service closed loop: the TV becomes the home service entry point, and the mobile phone becomes the execution confirmation terminal. The two collaborate through a standardized task flow protocol, truly opening up the cross-screen service link of "film and television content consumption → consumption decision → e-commerce execution", which is a reconstruction of the existing fragmented ecosystem.

[0052] The S400 and mobile terminal preload complete recipe step data and cut it into independent voice segments according to the cooking process; and use voice recognition to listen to the user's next instruction sequence to trigger the broadcast; at the same time, it adopts a keyword wake-up interruption questioning mechanism. When a cooking-related question is detected, the broadcast of the current step is paused, the answer is matched based on the recipe knowledge base and returned by voice, realizing hands-free operation throughout the entire process.

[0053] In this embodiment, after receiving the user's confirmation to start cooking, the user's mobile phone loads the complete recipe data into the memory buffer and uses a streaming segmented transmission strategy to cut the long text recipe into a triplet of step-action-parameter, such as step 1-action: pour oil-parameter: 30 ml.

[0054] The system also uses voice recognition to listen to the user's next instruction sequence and triggers playback accordingly. A TTS (Text-to-Speech) engine synthesizes the speech sequentially, but only plays the current step's content; subsequent steps are stored in memory as text awaiting triggering. The invention also continuously monitors ambient speech via a microphone array. When a "next step" instruction is detected, the next segment of speech is played. When a question or specific keyword such as "heat" or "time" is detected, the current audio stream is immediately paused, and a priority response mechanism is activated: the question text is matched with a local recipe knowledge base containing information such as the recipe's heat curve and time parameters; the answer is extracted and played via TTS priority; then, the previous step playback is resumed or the user is redirected to the relevant step.

[0055] This embodiment of the solution ensures that users do not need to touch the device with their hands during cooking, thus avoiding grease contamination of the phone and allowing them to obtain cooking guidance in real time while their hands are occupied (such as when stirring ingredients), significantly reducing operation interruptions and safety risks.

[0056] The present invention will be further described in detail below through a specific application embodiment: In this specific embodiment, taking the application scenario of scrambled eggs with tomatoes as an example, the specific implementation process is as follows: S1. A user is watching a cooking program on a smart TV in the living room. The program shows a chef pouring beaten egg liquid into a wok. At this moment, the content recognition engine deployed on the TV or in the cloud in this invention is activated: a convolutional neural network (CNN) model extracts keyframes of the current video, identifying red tomato chunks, yellow egg liquid, and a black wok in the image; simultaneously, it uses a named entity recognition (NER) model to analyze the program's subtitle text "Prepare two tomatoes, three eggs, and a little vegetable oil," extracting the three food entities: "tomato," "egg," and "vegetable oil." The system in this embodiment confirms the food list through multimodal matching as: 2 tomatoes, 3 eggs, 30ml vegetable oil; and infers the need for a gas stove based on the wok.

[0057] S2. A service selection pop-up window appears in the lower right corner of the TV screen, displaying two options: "[Takeout] Enjoy Tomato and Egg Stir-fry Now" and "[Cooking] Purchase Ingredients to Make Your Own Food." After selecting the "[Cooking]" option using the remote, the TV packages the ingredient list and user account information and pushes it to the user's linked mobile phone via a cross-screen protocol. This mobile phone is pre-linked and authorized with the TV account. Upon receiving the data, the mobile app automatically opens the associated fresh food e-commerce application, adds tomatoes, eggs, and vegetable oil to the shopping cart, and generates a payment order. The user simply clicks "Confirm Payment" on their phone to complete the ingredient purchase.

[0058] S3. The next day, after the ingredients are delivered, the user opens the mobile application and selects "Start Cooking". The mobile app pre-loads the complete recipe for scrambled eggs with tomatoes and divides it into five voice steps. In this embodiment, the system first announces the steps loaded on the phone: "Step 1: Cut the tomatoes into chunks and beat the eggs." After the user completes this step, they say "Next step," and the system continues: "Step 2: Heat the pan and pour in 30 ml of vegetable oil." When the user pours in the oil and asks, "How do I adjust the heat?", the system in this embodiment recognizes the keyword "heat" and immediately pauses the subsequent announcements for step 2. After consulting the knowledge base, it answers: "Scrambled eggs with tomatoes need to be heated over medium heat. Pour in the egg mixture when the oil is about 70% hot." After answering, it automatically resumes: "Please confirm that the oil is about 70% hot, and then say 'Next step' to continue." In this way, the user does not need to touch the phone screen at all. Even when cutting ingredients or stirring the pan, they can get real-time guidance through voice until the dish is finished.

[0059] In a further embodiment of the present invention, the home cooking information conversion and processing method based on dual-device collaboration, wherein after the step of the mobile terminal preloading the step data of the complete recipe and cutting it into independent audio segments according to the cooking process, the method further includes: Once the cooking process is detected as complete, the system uses the identified dish name to generate a social media copywriting template, automatically filling in the dish name and cooking text, and automatically synchronizing it to the user's preset social media platforms for posting.

[0060] In this embodiment of the invention, after completing the identification, procurement and cooking guidance of ingredients through the above steps, the dish names such as scrambled eggs with tomatoes and health analysis data output by the video semantic recognition engine can be used to generate standardized social sharing content through a preset template and trigger a cross-platform publishing mechanism to achieve automated connection between cooking behavior and social sharing.

[0061] For example, after detecting that the user has completed making scrambled eggs with tomatoes, the system will call the social copywriting generation template based on the identified dish name "scrambled eggs with tomatoes" and automatically fill in the text "Scrambled eggs with tomatoes made successfully, AI assistant chef's debut", and automatically synchronize it to the user's preset Weibo or WeChat Moments, without the user having to manually enter text or select a posting platform.

[0062] This solves the cumbersome problem of users having to manually edit sharing text and select social media platforms in existing technologies. This embodiment uses template-based generation technology, such as a format like "dish name, successful preparation, health label," to ensure the standardization and consistency of sharing text, reducing user steps. Simultaneously, the addition of health labels enhances the interactivity and information content of the shared content, improving the efficiency of social media dissemination. The automated process of this invention lowers the barrier to sharing for users and encourages the social dissemination of culinary achievements.

[0063] In another embodiment of the present invention, the method for converting and processing home cooking information based on dual-device collaboration further includes the step of constructing a structured ingredient list and an appliance list, wherein: S201. Based on the constructed structured ingredient list, a health analysis model using a preset ingredient weight and nutrient element conversion matrix converts cooking data into health data, enabling automated estimation of the nutritional components of dishes. The food weight-nutrient conversion matrix in this embodiment refers to a multidimensional mathematical mapping model used to establish a correspondence between the standard serving size of a food ingredient (e.g., 100 grams) and the content of specific nutrients (e.g., calories, protein, fat, carbohydrates, sodium, etc.). For example, the matrix records the mapping relationship for "eggs" as: 100 grams ≈ 144 kcal, 13.3 grams of protein, and 8.8 grams of fat.

[0064] The health analysis model refers to a statistical algorithm that combines the weight of ingredients, the oil absorption / dehydration coefficient of cooking methods, the penetration ratio of compound seasonings, and the nutrient conversion matrix to derive overall nutritional data from cooking data, rather than simply summing the nutritional components of ingredients.

[0065] The cooking data refers to structured parameters identified from the video, which include a complete set of production information such as the type of ingredients, recommended dosage, cooking method (frying, stir-frying, deep-frying, steaming), and the amount of oil and salt used.

[0066] The nutritional components of a dish refer to the final output, which is the total calories, macronutrients, and micronutrients contained in a complete dish per adult's average intake. This information can be directly used for health assessments or dietary recommendations.

[0067] This embodiment constructs an automated analysis layer that converts an ingredient list into nutrient data. After the system obtains a structured ingredient list through the aforementioned steps, it supplements this with the recommended weight for each ingredient and the thermophysical properties of the cooking method, such as the 35% fat absorption coefficient for "frying". At this point, the system calls a pre-set ingredient weight-nutrient conversion matrix, multiplying the mass parameter of each ingredient by the corresponding nutrient value per 100 grams in the matrix to calculate the basic nutritional contribution of each ingredient. Simultaneously, the health analysis model further processes implicit nutrient increases and decreases: for example, adjusting for additional oil oxidation based on the cooking method, the proportion of vitamin loss at high temperatures, and the implicit sugar and salt in compound sauces. After model calibration, the final output is a precise total nutrient estimate of a dish, corrected for cooking losses and additions, completing the automated conversion from cooking data to health data.

[0068] S202, combined with the automatic storage and periodic report generation mechanism of user health records, provides users with continuous tracking and trend analysis of nutritional intake, assisting users in scientific dietary health management.

[0069] In this embodiment, the user health profile refers to a multi-dimensional health data pool that is encrypted and stored on the terminal or trusted cloud, centered on the individual user. It includes, but is not limited to, basic physiological indicators such as height, weight, and age, as well as dietary preferences, allergens, chronic disease markers, and cumulative records of nutritional calculations for each dish.

[0070] The automatic storage mechanism refers to a background service that silently and in real-time writes the corresponding nutrition data package into the user's health record whenever the system completes a nutritional estimate of a dish or a user completes a purchase of real ingredients through an e-commerce platform (presumably to be cooked and consumed in the future), without requiring manual entry.

[0071] The periodic report generation mechanism in this embodiment refers to a timed task (such as generating every Sunday evening) built into the system, which periodically reads all dietary and nutritional data accumulated in the health record and automatically outputs a periodic nutritional analysis report combining text and graphics through trend calculation and visualization templates.

[0072] The continuous nutrient intake tracking and trend analysis in this embodiment refers to the cumulative and dynamic trend fitting of all the nutritional data of the dishes that the user expects to consume through this system within a certain time window (such as a week or a month) to determine whether the user's macronutrient energy supply structure is reasonable and whether there is a deficiency of micronutrients.

[0073] In the specific implementation of this step, firstly, each time a user triggers the "food nutrition estimation" or completes a grocery purchase through an e-commerce platform, the system automatically stores the corresponding nutritional dataset for that food into the user's personal health record, achieving zero-operation automated recording of dietary intake data. Subsequently, when the set periodic node is reached, the system's built-in periodic report generator is scheduled to start, automatically extracting all accumulated nutritional data within the current period from the health record, organizing it into a continuous nutritional intake sequence, and using trend analysis algorithms to generate a change curve compared to the previous period, a pie chart of nutrient energy contribution ratios, and possible dietary recommendations. This report can be directly pushed to the user's mobile phone or display device, helping the user intuitively understand recent dietary patterns and areas for improvement, ultimately achieving a leap from passively receiving food information to actively and scientifically managing their own dietary health.

[0074] For example, in this embodiment of the invention, based on the identified ingredient list (2 tomatoes, approximately 300 grams; 3 eggs, approximately 150 grams; and 30 ml of vegetable oil), the system calculates through the transformation matrix of the health analysis model that the "tomato and egg stir-fry" contains approximately 350 kcal of energy, 18 grams of protein, and 25 grams of fat. The data is automatically stored in the user's health record. On Sunday, the system automatically generates and pushes a weekly report: "This week's protein intake met the standard, but fat intake was slightly high. It is recommended to reduce oil consumption next week."

[0075] Thus, this embodiment of the invention solves the problems of data omission and untimely recording caused by the reliance on manual input of dietary health data in the prior art. By constructing a conversion matrix from ingredient weight to nutritional elements such as calories, protein, and fat, the nutritional composition of dishes can be automatically estimated. Combined with the automatic storage of health records and a periodic report generation mechanism such as weekly reports, users are provided with continuous tracking of nutritional intake and trend analysis, such as "This week's protein intake met the target," assisting users in scientific dietary health management and significantly improving the convenience and accuracy of health management.

[0076] Furthermore, in collaborative cooking scenarios within a home kitchen, particularly the "relay cooking" model common in modern dual-income families, the first operator, such as the husband who arrives home first, starts cooking and completes the pre-processing and initial stir-frying of ingredients. If they need to leave the kitchen due to unforeseen circumstances, the second operator, such as the wife who arrives later, takes over to complete the subsequent seasoning and sauce-reducing stages. This presents specific technical challenges related to synchronizing cooking status across users and seamlessly transferring permissions.

[0077] Furthermore, cooking preference parameters for different family members, such as texture preference, saltiness threshold, and heat control habits, are statically bound to the initial login account and cannot be dynamically switched based on the current user. This results in subsequent voice guidance potentially contradicting the cooking habits of the person taking over, such as the system continuing to suggest "stir-fry over high heat" while the person taking over prefers "simmer over low heat." The root cause of this problem lies in the system architecture's lack of multi-user identity awareness and distributed state synchronization mechanisms, leading to mismatches between cooking instructions and operator preferences, and gaps in process state information when multiple people take turns operating the system.

[0078] Therefore, in order to further solve the above problems, the optimized embodiment of the present invention for the home cooking information conversion and processing method based on dual-device collaboration further includes the following steps: S910: By adding a voiceprint fingerprint recognition unit to the audio input layer, the speaker's voiceprint feature vector is extracted simultaneously when voice commands are collected, and a voiceprint database of commonly used family members is established to realize real-time binding of voice control commands with the identity of the operating user.

[0079] The voiceprint fingerprint recognition unit in this embodiment refers to the hardware logic or underlying software algorithm embedded in the audio input front end, such as a microphone array or a smart speaker pickup module, which can extract unique biometric features that characterize the speaker's vocal tract physiological structure and pronunciation habits from the voice signal and compress them into an irreversible mathematical vector.

[0080] In this embodiment, the voiceprint feature vector refers to a multi-dimensional floating-point array that represents the unique characteristics of the speaker's voice, such as the fundamental frequency curve and cepstral coefficients, and is used for voiceprint similarity calculation and difference comparison.

[0081] The family member voiceprint database in this embodiment refers to a lightweight voiceprint registry deployed on a local edge gateway or trusted cloud, which stores the pre-registered voiceprint feature vectors and corresponding identity tags of each family member, and supports real-time retrieval and comparison.

[0082] In the specific implementation of step S910, a voiceprint fingerprint recognition unit is integrated into the audio input component used to receive cooking voice commands. When any family member issues a voice command such as "Next," "Turn up the heat," or "Add minced garlic," this unit simultaneously extracts the voiceprint feature vector of the current speaker while acquiring the audio stream. In this embodiment, the system compares this vector with a locally pre-established voiceprint database of commonly used family members to determine the speaker's identity, such as "Dad," "Mom," or "Child." Thus, each voice control command, at the moment it generates a cooking operation, is bound in real time to a specific user identity, forming an identity tagging link of "who is speaking, who is controlling."

[0083] In this way, the present invention can eliminate identity confusion when multiple people share a single voice system, because traditional voice assistants cannot distinguish the speaker, leading to conflicting instructions, such as one person saying "flip the noodles" and another saying "add salt." This step, through underlying voiceprint binding, enables the system to distinguish the source of the instruction, laying the foundation for subsequent personalized responses and multi-user arbitration.

[0084] Furthermore, this invention enables natural identity identification during hands-free operation, eliminating the need for users to manually switch accounts or enter passwords. It allows for speaker recognition in a completely seamless manner while users are busy cooking, meeting the requirements for high safety and convenience in the cooking process.

[0085] S920. A real-time shared buffer for cooking status is pre-established, and a distributed status synchronization architecture based on a home LAN or Bluetooth network is constructed. A shared memory pool is set up to record key cooking status parameters. The shared buffer adopts a publish-subscribe mechanism to ensure that the terminal devices of all authorized family members can read the latest key cooking status parameters in real time. The key cooking status parameters recorded include: the current execution step number, the real-time temperature inside the pot transmitted by the smart cooker sensor, the cumulative weight of the ingredients and seasonings added, and the remaining countdown for the current step.

[0086] The real-time shared buffer for cooking status in this embodiment refers to a high-speed data exchange area designed based on a publish / subscribe model. Physically, it can be a memory page of a smart TV, a cache of a smart speaker, or the memory of a separate home gateway, used to synchronize the latest cooking process parameters among multiple members and multiple devices.

[0087] The distributed state synchronization architecture refers to a decentralized data synchronization logic that utilizes a home LAN or Bluetooth network to enable each authorized device, such as a kitchen tablet, living room TV, or mobile phone, to read and write data equally and be notified of state changes, without relying on a single central server.

[0088] The shared memory pool refers to a physical memory region that is mapped to the virtual address spaces of multiple processes or devices. In this context, it refers to a logical block used to store core cooking parameters, the contents of which are considered to be the latest and consistent values ​​by all visitors.

[0089] The key cooking status parameters include digital parameters that can fully reconstruct the progress of the cooking process, such as the current execution step number, the real-time temperature inside the pot transmitted by the smart cookware sensor, the cumulative weight of the ingredients and seasonings already added, and the remaining countdown for the current step.

[0090] In step S920, during system initialization, a real-time shared buffer for cooking status is established using the home network environment. All authorized devices participating in the cooking process, such as the main speaker, mobile phone, and cooktop display, can form a distributed state synchronization architecture via LAN or Bluetooth, jointly subscribing to a shared memory pool. During cooking, various state data sources continuously publish key state parameters to this pool: for example, smart cookware pushes real-time temperature and total weight changes into the buffer pool, and the recipe execution module writes the current step number and the countdown for that step into a single location. When any family member views the progress or interacts with the device, the device instantly retrieves the latest consistent state view from the buffer pool. Through the publish-subscribe mechanism, any change in a parameter is broadcast to all subscribed devices within milliseconds, ensuring that everyone in the family sees the same pot of food at the same instant.

[0091] This invention breaks down information silos in the kitchen, enabling synchronized information across multiple devices. It solves the problem in traditional solutions where only a single device tracks progress, preventing others from participating or repeatedly asking questions. Couples can each monitor the cooking progress simultaneously using their own devices, avoiding repetitive or erroneous operations due to a lack of context.

[0092] This invention uses a publish-subscribe approach instead of polling, which reduces network overhead and state lag. Even when kitchen grease interferes with some sensors, the shared pool still retains the latest valid records, making the entire distributed cooking guidance system more robust and improving the fault tolerance of the distributed system.

[0093] S930: Pre-build a dynamic mapping table of user preferences to create an independent cooking habit profile for each family member and record personalized parameters, including taste preferences, saltiness threshold, and heat control mode. The user preference dynamic mapping table in this embodiment refers to a key-value mapping structure that can learn and update over time, pointing user identifiers such as voiceprint IDs to a set of cooking preference parameters covering taste preferences and heating habits, supporting dynamic correction and fine-grained configuration.

[0094] The cooking habit profile refers to a structured data package at the individual level, which stores various highly variable parameters related to cooking taste and operation accumulated through initial questionnaires, historical feedback corrections, and explicit evaluations.

[0095] The taste preference in this embodiment describes an individual's preference for the texture, crispness, or smoothness of the finished product, such as "preferring crisp vegetables" or "preferring tender stewed meat." The saltiness threshold refers to a quantitative indicator of an individual's sensitivity to sodium ions, used to dynamically adjust the recommended amount of sodium-containing seasonings such as salt and soy sauce in the recipe, such as "too bland, so that the salt content is reduced to 80% of the standard value."

[0096] The heat control mode reflects the approximate heating intensity and time rhythm that the user prefers, such as "slow simmering" or "fast stir-frying", and may even be related to the specific stove's heat setting habits.

[0097] In step S930, a unique cooking habit profile is pre-established for each registered voiceprint family member. This profile is stored in a dynamic user preference mapping table. The personalized parameters recorded in the profile include, but are not limited to, three core indicators: taste preferences, saltiness thresholds, and heat control modes. These parameters can be dynamically updated and refined through initial user settings, feedback analysis of adherence to historical cooking guidelines, and implicit evaluations in the voice (such as "a little salty"). Thus, each person possesses a digital avatar of cooking habits that evolves according to their changing tastes.

[0098] This step demonstrates how to evolve from a one-size-fits-all recipe guide to a precisely tailored approach for each individual. Traditional recipe videos and apps cater to the general public and cannot simultaneously meet the diverse needs of a family: elderly people prefer soft and tender dishes, children dislike spicy food, and middle-aged people control their salt intake. This step, through independent profiles, allows the same recipe to provide drastically different parameter suggestions to different users, precisely matching their individual tastes.

[0099] Furthermore, embodiments of the present invention can also be linked to health models, such as automatically recommending lower sodium levels in all recipes for fathers with hypertension, making dietary management more continuous and preventative.

[0100] Moreover, this invention can reduce the cognitive burden of manual adjustments for users. Users no longer need to recall and verbally correct their preferences before each cooking session, such as adding less salt or cooking a little longer. The system automatically retrieves preferences from the archives, making voice interaction more natural and commands shorter.

[0101] S940: Dynamically load the current user's preference profile based on the voiceprint recognition result, and adjust the content generation logic of subsequent voice guidance in real time.

[0102] The preference configuration file in this embodiment corresponds to the cooking habit file in S930. It is a set of runtime preference parameters that the system reads from the mapping table and loads into memory after detecting the voiceprint identity, and is directly called by the subsequent guidance algorithm.

[0103] The content generation logic refers to the core algorithm responsible for generating TTS (text-to-speech) broadcast content and on-screen prompts. This logic dynamically modifies style elements such as quantity, duration, and tips based on the current recipe step template and parameters in the preference configuration file.

[0104] In practice, this step takes effect immediately once the S910's voiceprint recognition unit identifies the speaker of the current voice command. Based on the recognition result, the system reads the corresponding preference profile from the user preference dynamic mapping table and instantly injects these parameters into the intelligent recipe content generation logic. For example, if the speaker is identified as the mother, and her profile shows a "saltiness threshold" of reduced sodium and a "heat control mode" of simmering, the standard voice instruction "Add 1.5 teaspoons of salt, turn up the heat to reduce the sauce" will be rewritten in real time and announced as "Add slightly less than one level teaspoon of salt, turn down the heat to low and simmer until the sauce thickens; this will better suit your taste." The entire adjustment process is completed in the background within milliseconds, and the user is completely unaware of the switch.

[0105] Through this step-by-step embodiment, seamless personalized service flow switching can be achieved. When multiple members take turns operating, the voice guidance can be adjusted instantly according to the change of the operating user's identity, without the need for any manual account switching or login, and the voice interaction experience is smooth and consistent.

[0106] Furthermore, this invention instantly translates preference data into usable operational information, avoiding the disconnect of knowing user preferences but still providing standard values ​​as guidance. This ensures that previously accumulated preference profiles truly impact cooking behavior, directly affecting the taste of the finished product and creating a value loop. It also improves the overall cooking success rate.

[0107] S950. When the voiceprint feature extraction module detects that the similarity between the voiceprint feature of the voice command and the previous command is lower than the preset threshold, the arbitration process is automatically triggered: the TTS broadcast of the current step is paused, and the current cooking status is read from the shared buffer to generate a current status summary. In this embodiment, the similarity is lower than the preset threshold, meaning that the voiceprint feature vector extracted from the current voice command and the voiceprint vector of the previous command do not reach the minimum threshold for the system to determine that they are the same speaker in terms of cosine similarity or probability ratio distance, which is used to determine that the chef may have been replaced.

[0108] The arbitration process in this embodiment is a set of protective and transitional processing logic activated by the system when it detects that commands may be initiated by different users, preventing command conflicts, state loss, or the incorrect inheritance of dangerous operations. The TTS broadcast is a text-to-speech output, referring here to the audio stream of recipe guidance steps and operation prompts.

[0109] The current state summary refers to a concise natural language text that the system quickly extracts and organizes from the real-time shared buffer described in S920, covering the core context of the current step, such as: "Currently, it is the third step of frying steak. The pan temperature is 200 degrees. The steak has been in the pan for 2 minutes and 30 seconds. There are 30 seconds left before flipping it."

[0110] In the specific implementation of step S950, the voiceprint feature extraction module continuously monitors during the interactive cooking process. When a new voice command is received, if the similarity between its voiceprint and the previous operator's voiceprint falls below a preset threshold, this embodiment of the invention determines that a chef change has occurred. At this point, an arbitration process is immediately triggered: First, all ongoing TTS broadcasts are immediately paused to prevent the new chef from being misled by instructions not fully heard by the previous chef. Next, all key cooking state parameters are rapidly extracted from the real-time shared buffer of the cooking state, and a natural language generation submodule is invoked to semantically encapsulate them, forming a concise summary of the current state, ready to be used to inform the new successor.

[0111] This ensures cooking safety and operational continuity, as the biggest risk during multi-person handover is that the successor is unaware of the cooking time or the countdown, resulting in a gap in the workflow. The arbitration process employed in this invention forcibly halts the process and proactively generates context, effectively preventing safety incidents such as burning and spills, and reducing the risk of human error during handover to a level controllable by the system.

[0112] Furthermore, this invention can construct responsible handover semantics because by pausing old instructions and issuing status to new users, it avoids instruction overlap or conflict, making the entire multi-person collaborative cooking process as orderly as a relay race.

[0113] Moreover, the entire process of this invention is automatically triggered without manual intervention. This is because the handover action is automatically detected and activated by voiceprint differences. The chef does not need to ask for help or manually check the status. Both hands can remain on the spatula, thus maintaining hygiene and safety during operation.

[0114] S960, and announce the generated current status summary to the successor via voice for confirmation; at the same time, switch to the newly identified user's preference profile and adjust the parameter suggestions for subsequent steps.

[0115] The confirmation of the successor refers to the system in this embodiment of the invention broadcasting the current status summary generated in step S950 to the newly identified user in voice form. In essence, it is to synchronize the context with the successor and implicitly wait for his cognitive feedback to ensure that the successor has understood the progress on site.

[0116] The preference profile switching refers to the process where, after the context handover is completed, the background abandons the previous user's preference profile and instead loads and activates the parameters in the current successor's personal cooking habit profile, thus completing the full transition of the guidance strategy.

[0117] In this step, immediately following step S950, the generated current status summary is announced via voice to the newly assigned chef, such as, "You have been identified. This is step four. The pot temperature has dropped slightly. Stew for another 3 minutes. The previous chef added carrots and half the salt." This serves as a confirmation to the new chef. Almost simultaneously, the preference profile referenced by the TTS engine and content generation logic is instantly switched to the one mapped by the newly identified user's voiceprint. Once the confirmation announcement is complete and the new chef gives the order, subsequent heat and seasoning suggestions will be adjusted based entirely on the new chef's own preferences, such as "high heat, quick stir-fry" and "less oil, mild spiciness," without retaining the previous chef's preference settings.

[0118] In this way, the present invention simulates the natural dialogue mode of "handing over the pot" in a human kitchen through the atomized switching of status briefings and personalized configurations—the experienced chef informs the chef of the progress and hands over the apron, while the new chef continues to season according to their own habits. The whole process is extremely natural, achieving a seamless and humanized chef handover ceremony. Furthermore, through a complete hot switch of preferences, it ensures that the final flavor of the meal is highly consistent with the preferences of the person who performed the work and their main customers; it can prevent preference contamination and ensure that the taste of the finished product meets the expectations of the final chef.

[0119] As can be seen from the above, the optimized embodiment of the present invention achieves immediate perception of operator changes through real-time comparison of voiceprint features, solving the problem of identity recognition gaps; it breaks the limitations of local device storage by utilizing a distributed shared buffer, making the cooking process status transparent and visible to multiple users, solving the information synchronization problem; and it ensures that the cooking instructions always conform to the habits of the current actual user through dynamic loading of preference profiles and remapping of instruction parameters, solving the problem of personalized mismatch. When the first operator says "Next" and leaves after completing steps S910-S940, and the second operator enters the kitchen and says "Next," the system identifies the user change through voiceprint differences, immediately queries the shared buffer to obtain the status "hot oil is available in the pot waiting for eggs," and simultaneously loads the setting "prefers larger egg pieces" from the second operator's preference profile, dynamically adjusting the original instruction "quickly break into small pieces" to "gently push to form large pieces," and broadcasts the current status prompt via voice, achieving seamless connection with zero interruption in multi-person relay scenarios.

[0120] Exemplary device like Figure 2 As shown, this embodiment of the invention provides a home cooking information conversion and processing device based on dual-device collaboration, comprising: The video semantic recognition module 310 is used to extract the visual features of ingredients in the key frames of the target cooking video, parse the ingredient name entities in the video subtitles or audio, identify the types of ingredients required for the target dish, and extract and identify the types of cooking utensils through key frames to obtain the required cooking equipment and construct a structured ingredient list and utensil list. The service type mapping and distribution module 320 is used to generate service type mapping logic on the display device based on the identified structured ingredient list, providing optional service paths of ordering takeout for immediate consumption and purchasing ingredients to make the food; and triggering the corresponding service chain according to the user's selection, converting the semantics of the video content into specific execution instructions; The cross-screen execution module 330 is used to establish a task flow channel between the display device and the mobile terminal. When the display device detects that the user selects the path of purchasing ingredients to make the food, it pushes the structured ingredient list to the mobile terminal bound to the user. After receiving the list, the mobile terminal calls the e-commerce platform interface to generate an order to be confirmed. The segmented hands-free voice interaction module 340 is used to preload the complete recipe's step data into the mobile terminal and cut it into independent voice segments according to the cooking process; and to listen to the user's next instruction sequence through voice recognition and trigger the broadcast; at the same time, it adopts a keyword-based wake-up interruption questioning mechanism. When a cooking-related question is detected, the current step broadcast is paused, the answer is matched based on the recipe knowledge base and returned through voice, realizing hands-free operation throughout the entire process, as described above.

[0121] Based on the above embodiments, the present invention also provides a smart terminal, the principle block diagram of which can be as follows: Figure 3As shown, the smart terminal can be a smartphone, smart TV, smart large screen, etc. The smart terminal includes a processor, memory, network interface, display screen, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the smart terminal is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a home cooking information conversion and processing method based on dual-device collaboration. The database of the smart terminal stores the home cooking information conversion and processing program based on dual-device collaboration.

[0122] Those skilled in the art will understand that Figure 3 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the smart terminal to which the present invention is applied. A specific smart terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0123] In one embodiment, a smart terminal is provided, including a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations: Extract visual features of ingredients from keyframes of the target cooking video, parse the ingredient name entities in the video subtitles or audio, identify the types of ingredients required for the target dish, and extract and identify the types of cooking utensils through keyframes to obtain the required cooking equipment, and construct a structured ingredient list and utensil list. Based on the identified structured ingredient list, service type mapping logic is generated on the display device, providing optional service paths of ordering takeout for immediate enjoyment and purchasing ingredients to cook at home; the corresponding service chain is triggered according to the user's selection, and the semantics of the video content are converted into specific execution instructions; A task flow channel is established between the display device and the mobile terminal. When the user selects the path of purchasing ingredients to make their own food, the display device pushes the structured ingredient list to the user's bound mobile terminal. After receiving the list, the mobile terminal calls the e-commerce platform interface to generate an order to be confirmed. The mobile terminal preloads the complete recipe's step data and cuts it into independent voice segments according to the cooking process; it also uses voice recognition to listen to the user's next instruction and trigger the broadcast sequentially; at the same time, it adopts a keyword-based wake-up interruption questioning mechanism. When a cooking-related question is detected, the broadcast of the current step is paused, the answer is matched based on the recipe knowledge base, and the answer is returned via voice, realizing hands-free operation throughout the entire process, as described above.

[0124] The step of preloading the complete recipe's step data on the mobile terminal and cutting it into independent audio segments according to the cooking process also includes: Once the cooking process is detected as complete, the system uses the identified dish name to generate a social media copywriting template, automatically filling in the dish name and cooking text, and automatically synchronizing it to the user's preset social media platforms for posting.

[0125] The steps of constructing the structured ingredient list and utensil list also include: Based on the constructed structured ingredient list, a health analysis model using a preset ingredient weight and nutrient element conversion matrix converts cooking data into health data, enabling automated estimation of the nutritional components of dishes. Combined with the automatic storage of user health records and the periodic report generation mechanism, it provides users with continuous tracking and trend analysis of nutritional intake, assisting users in scientific dietary health management.

[0126] The steps of preloading complete recipe step data on the mobile terminal and cutting it into independent audio segments according to the cooking process, and triggering the broadcast by listening to the user's next instruction sequence through voice recognition, also include: By adding a voiceprint fingerprint recognition unit to the audio input layer, the speaker's voiceprint feature vector is extracted simultaneously when voice commands are collected, and a voiceprint database of commonly used family members is established to realize real-time binding of voice control commands with the identity of the operating user.

[0127] The steps of preloading complete recipe step data on the mobile terminal and cutting it into independent audio segments according to the cooking process, and triggering the broadcast by listening to the user's next instruction sequence through voice recognition, also include: A real-time shared buffer for cooking status is pre-established, a distributed status synchronization architecture based on a home LAN or Bluetooth network is constructed, and a shared memory pool is set up to record key cooking status parameters; the shared buffer adopts a publish-subscribe mechanism to ensure that the terminal devices of all authorized family members can read the latest key cooking status parameters in real time. The key cooking status parameters recorded include: the current step number, the real-time temperature inside the pot transmitted by the smart cookware sensor, the cumulative weight of the ingredients and seasonings added, and the remaining countdown for the current step.

[0128] The steps of preloading complete recipe step data on the mobile terminal and cutting it into independent audio segments according to the cooking process, and triggering the broadcast by listening to the user's next instruction sequence through voice recognition, also include: A dynamic mapping table of user preferences is pre-built to create an independent cooking habit profile for each family member, recording personalized parameters including taste preferences, saltiness threshold, and heat control mode; Based on the voiceprint recognition results, the current user's preference profile is dynamically loaded, and the logic for generating subsequent voice guidance content is adjusted in real time.

[0129] The steps of preloading complete recipe step data on the mobile terminal and cutting it into independent audio segments according to the cooking process, and triggering the broadcast by listening to the user's next instruction sequence through voice recognition, also include: When the voiceprint feature extraction module detects that the similarity between the voiceprint feature of the voice command and the previous command is lower than a preset threshold, the arbitration process is automatically triggered: the TTS broadcast of the current step is paused, and the current cooking status is read from the shared buffer to generate a current status summary. The generated current status summary will be broadcast to the successor for confirmation via voice. At the same time, switch to the preference profile of the newly identified user and adjust the parameter suggestions for subsequent steps, as described above.

[0130] In other embodiments, this application proposes a computer-readable storage medium that, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the above-described method.

[0131] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0132] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for converting and processing home cooking information based on dual-device collaboration, characterized in that, include: Extract visual features of ingredients from keyframes of the target cooking video, parse the ingredient name entities in the video subtitles or audio, identify the types of ingredients required for the target dish, and extract and identify the types of cooking utensils through keyframes to obtain the required cooking equipment, and construct a structured ingredient list and utensil list. Based on the identified structured ingredient list, service type mapping logic is generated on the display device, providing optional service paths of ordering takeout for immediate enjoyment and purchasing ingredients to cook at home; the corresponding service chain is triggered according to the user's selection, and the semantics of the video content are converted into specific execution instructions; A task flow channel is established between the display device and the mobile terminal. When the user selects the path of purchasing ingredients to make their own food, the display device pushes the structured ingredient list to the user's bound mobile terminal. After receiving the list, the mobile terminal calls the e-commerce platform interface to generate an order to be confirmed. The mobile terminal preloads the complete recipe's step data and cuts it into independent voice segments according to the cooking process; it also uses voice recognition to listen to the user's next instruction and trigger the broadcast sequentially; at the same time, it adopts a keyword-based wake-up interruption questioning mechanism. When a cooking-related question is detected, the broadcast of the current step is paused, the answer is matched based on the recipe knowledge base, and the answer is returned via voice, realizing hands-free operation throughout the entire process.

2. The method for converting and processing home cooking information based on dual-device collaboration according to claim 1, characterized in that, After the step of preloading the complete recipe's step data on the mobile terminal and cutting it into independent audio segments according to the cooking process, the following is also included: Once the cooking process is detected as complete, the system uses the identified dish name to generate a social media copywriting template, automatically filling in the dish name and cooking text, and automatically synchronizing it to the user's preset social media platforms for posting.

3. The method for converting and processing home cooking information based on dual-device collaboration according to claim 1, characterized in that, The steps for constructing the structured ingredient list and utensil list also include: Based on the constructed structured ingredient list, a health analysis model using a preset ingredient weight and nutrient element conversion matrix converts cooking data into health data, enabling automated estimation of the nutritional components of dishes. Combined with the automatic storage of user health records and the periodic report generation mechanism, it provides users with continuous tracking and trend analysis of nutritional intake, assisting users in scientific dietary health management.

4. The method for converting and processing home cooking information based on dual-device collaboration according to claim 1, characterized in that, The mobile terminal preloads the step data of the complete recipe and cuts it into independent audio segments according to the cooking process; The steps of triggering the broadcast by listening to the user's next command sequence through voice recognition also include: By adding a voiceprint fingerprint recognition unit to the audio input layer, the speaker's voiceprint feature vector is extracted simultaneously when voice commands are collected, and a voiceprint database of commonly used family members is established to realize real-time binding of voice control commands with the identity of the operating user.

5. The method for converting and processing home cooking information based on dual-device collaboration according to claim 4, characterized in that, The mobile terminal preloads the step data of the complete recipe and cuts it into independent audio segments according to the cooking process; The steps of triggering the broadcast by listening to the user's next command sequence through voice recognition also include: A real-time shared buffer for cooking status is pre-established, a distributed status synchronization architecture based on a home LAN or Bluetooth network is constructed, and a shared memory pool is set up to record key cooking status parameters; the shared buffer adopts a publish-subscribe mechanism to ensure that the terminal devices of all authorized family members can read the latest key cooking status parameters in real time. The key cooking status parameters recorded include: the current step number, the real-time temperature inside the pot transmitted by the smart cookware sensor, the cumulative weight of the ingredients and seasonings added, and the remaining countdown for the current step.

6. The method for converting and processing home cooking information based on dual-device collaboration according to claim 5, characterized in that, The mobile terminal preloads the step data of the complete recipe and cuts it into independent audio segments according to the cooking process; The steps of triggering the broadcast by listening to the user's next command sequence through voice recognition also include: A dynamic mapping table of user preferences is pre-built to create an independent cooking habit profile for each family member, recording personalized parameters including taste preferences, saltiness threshold, and heat control mode; Based on the voiceprint recognition results, the current user's preference profile is dynamically loaded, and the logic for generating subsequent voice guidance content is adjusted in real time.

7. The method for converting and processing home cooking information based on dual-device collaboration according to claim 6, characterized in that, The mobile terminal preloads the step data of the complete recipe and cuts it into independent audio segments according to the cooking process; The steps of triggering the broadcast by listening to the user's next command sequence through voice recognition also include: When the voiceprint feature extraction module detects that the similarity between the voiceprint feature of the voice command and the previous command is lower than a preset threshold, the arbitration process is automatically triggered: the TTS broadcast of the current step is paused, and the current cooking status is read from the shared buffer to generate a current status summary. The generated current status summary will be broadcast to the successor for confirmation via voice. At the same time, switch to the preference profile of the newly identified user and adjust the parameter suggestions for subsequent steps.

8. A home cooking information conversion and processing device based on dual-device collaboration, characterized in that, The device includes: The video semantic recognition module is used to extract the visual features of ingredients in key frames of the target cooking video, parse the ingredient name entities in the video subtitles or audio, identify the types of ingredients required for the target dish, and extract and identify the types of cooking utensils through key frames to obtain the required cooking equipment and construct a structured ingredient list and utensil list. The service type mapping and distribution module is used to generate service type mapping logic on the display device based on the identified structured ingredient list, providing optional service paths of ordering takeout for immediate consumption and purchasing ingredients to make the food; it triggers the corresponding service chain according to the user's selection, and converts the semantics of the video content into specific execution instructions; The cross-screen execution module is used to establish a task flow channel between the display device and the mobile terminal. When the display device detects that the user selects the path of purchasing ingredients to make the food, it pushes the structured ingredient list to the user's bound mobile terminal. After receiving the list, the mobile terminal calls the e-commerce platform interface to generate an order to be confirmed. The segmented hands-free voice interaction module is used to preload the complete recipe's step data into mobile terminals and cut it into independent voice segments according to the cooking process; it also uses voice recognition to listen to the user's next instruction and trigger the broadcast sequentially; at the same time, it adopts a keyword-based wake-up interruption questioning mechanism. When a cooking-related question is detected, the broadcast of the current step is paused, the answer is matched based on the recipe knowledge base, and the answer is returned via voice, realizing hands-free operation throughout the entire process.

9. A smart terminal, characterized in that, It includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors, wherein the one or more programs include methods for performing any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1-7.

Images