Calorific intake monitoring method and device and medium

Abstract

The invention discloses a calorie intake monitoring method and device and a medium, and relates to the technical field of head-mounted devices. The method is applied to a head-mounted device, and comprises the following steps: periodically acquiring a first environment image according to a first time interval until the first environment image comprises a food sub-image; from the moment when the first environment image comprises the food sub-image, a second environment image is obtained according to a second time interval and a fixed period, the calorie of food corresponding to the food sub-image contained in the second environment image is recognized, and a food calorie value corresponding to the second environment image is obtained; and determining the heat intake of the wearer of the head-mounted equipment according to the sequentially obtained food heat values. According to the method, the heat intake of the wearer can be automatically and noninductively monitored, and manual triggering of the wearer is not needed.

Description

Calorie intake monitoring methods, equipment and media Technical Field

[0001] This application relates to the field of head-mounted device technology, and more specifically, to a method, device, and medium for monitoring calorie intake. Background Technology

[0002] With the rapid development of smart wearable devices, they are now being used more and more widely. At the same time, as living standards improve, people are paying increasing attention to their health, especially their daily calorie intake. Therefore, how to use smart head-mounted devices to monitor users' daily calorie intake has become a pressing technical problem to be solved. Summary of the Invention

[0003] One objective of this application is to provide a new technical solution for monitoring calorie intake.

[0004] According to a first aspect of this application, a method for monitoring calorie intake is provided, applied to a head-mounted device. The method includes: periodically acquiring a first environmental image at a first time interval until the first environmental image includes a food sub-image; starting from the moment when the first environmental image includes a food sub-image, periodically acquiring a second environmental image at a second time interval, and identifying the calorie content of the food corresponding to the food sub-image contained in the second environmental image to obtain a calorie value of the food corresponding to the second environmental image; and determining the calorie intake of the wearer of the head-mounted device based on the sequentially obtained calorie values ​​of the food.

[0005] Optionally, after acquiring a second environmental image periodically at a second time interval from the moment when the first environmental image includes a food sub-image, identifying the calories of the food corresponding to the food sub-image contained in the second environmental image, and obtaining the calorie value of the food corresponding to the second environmental image, the method further includes: when the second environmental image does not include a food sub-image, repeating the step of acquiring the first environmental image periodically at a first time interval until the first environmental image includes a food sub-image.

[0006] Optionally, the step of periodically acquiring the first environmental image at a first time interval until the first environmental image includes a food sub-image includes: acquiring the wearer's motion data; and, if it is determined from the motion data that the wearer's action conforms to an eating action, periodically acquiring the first environmental image at a first time interval until the first environmental image includes a food sub-image.

[0007] Optionally, before acquiring the first environmental image periodically according to the first time interval until the first environmental image includes a food sub-image, the method further includes: updating the first time interval to a third time interval when the current time is within at least one preset time period, the preset time period being the wearer's regular eating time period; and updating the first time interval to a fourth time interval when the current time is outside any of the preset time periods, wherein the duration of the fourth time interval is greater than the duration of the third time interval.

[0008] Optionally, identifying the calories of the food corresponding to the food sub-image contained in the second environmental image includes: inputting the second environmental image into a preset calorie recognition model, and having the preset calorie recognition model identify the calories of the food corresponding to the food sub-image contained in the second environmental image; wherein the preset calorie recognition model is trained from a training sample set, the training sample set includes multiple training samples, each training sample includes a food sample image and the calorie value and calorie-related factors corresponding to the food sample image, and the training sample set includes open-source training samples and real training samples.

[0009] Optionally, the heat-related factors include at least one of type, size, quantity, and cooking method.

[0010] Optionally, before determining the calorie intake of the wearer of the head-mounted device based on the sequentially obtained food calorie values, the method further includes: determining a food change trend based on a first sub-environment image and a second sub-environment image, wherein the first sub-environment image and the second sub-environment image are adjacent second environment images, and the second sub-environment image is the next second environment image after the first sub-environment image; determining a food calorie change trend based on the food calorie values ​​corresponding to the first sub-environment image and the food calorie values ​​corresponding to the second sub-environment image; and, if the food calorie change trend does not match the food change trend, discarding the food calorie values ​​corresponding to the second sub-environment image.

[0011] Optionally, after determining the calorie intake of the wearer of the head-mounted device based on the sequentially obtained calorie values ​​of the food, the method further includes: determining the wearer's eating time period based on the identification time corresponding to the sequentially obtained calorie values ​​of the food; outputting eating suggestions based on the eating time period and the calorie intake; and / or determining the eating time period as a preset time period.

[0012] According to a second aspect of this application, a head-mounted device is provided, including a memory and a processor, the memory being configured to store computer instructions, and the processor being configured to invoke the computer instructions from the memory to perform the method as described in any one of the first aspects.

[0013] According to a third aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of the first aspects.

[0014] Based on the above, this application provides a calorie intake monitoring method applied to a head-mounted device. The method includes: periodically acquiring a first environmental image at a first time interval until the first environmental image includes a food sub-image; starting from the moment the first environmental image includes a food sub-image, periodically acquiring a second environmental image at a second time interval, and identifying the calorie content of the food corresponding to the food sub-image in the second environmental image to obtain the calorie value of the food corresponding to the second environmental image; and determining the calorie intake of the wearer of the head-mounted device based on the sequentially obtained calorie values ​​of the food. This method can automatically and imperceptibly monitor the wearer's calorie intake without requiring manual triggering by the wearer. Furthermore, the calorie intake monitoring method provided by this application can be implemented using the image sensor on the head-mounted device, thus eliminating the need for an additional sensor module in the head-mounted device and preventing any increase in the weight and size of the head-mounted device.

[0015] Other features and advantages of this application will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the present application and, together with their description, serve to explain the principles of the present application.

[0017] Figure 1 is a flowchart illustrating a calorie intake monitoring method provided in an embodiment of this application; Figure 2 is a structural diagram illustrating a calorie intake monitoring device provided in an embodiment of this application; Figure 3 is a structural diagram illustrating a head-mounted device provided in an embodiment of this application. Detailed Implementation

[0018] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0019] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0020] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0021] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0022] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0023] This application provides a method for monitoring calorie intake, which is applied to a head-mounted device. The head-mounted device can specifically be AR / MR glasses or a helmet. Of course, the head-mounted device can also be smart glasses capable of taking photos, etc. That is, this application does not limit the specific form of the head-mounted device.

[0024] As shown in Figure 1, the calorie intake monitoring method provided in this application includes the following steps S1100 to S1300.

[0025] Step S1100: Acquire the first environmental image periodically according to the first time interval until the first environmental image includes a food sub-image.

[0026] The head-mounted device includes an image sensor for acquiring environmental images. The first environmental image is used to determine whether it includes a food sub-image. The first time interval is the time interval for acquiring the first environmental image. In one example, the first time interval is 30 minutes.

[0027] In step S1100 above, the head-mounted device acquires a first environmental image periodically at a first time interval, and exemplarily uses a 1M-level CNN network to perform binary classification on the first environmental image to determine whether the first environmental image contains food. If a food sub-image exists in the first environmental image, the acquisition of the first environmental image periodically at the first time interval is stopped, and step S1200 is further executed. Otherwise, the acquisition of the first environmental image periodically at the first time interval continues.

[0028] Since users of head-mounted devices are typically close to food when eating, the first environmental image can be downsampled to 64×64 resolution before determining the presence of a food sub-image. Furthermore, the backbone network of the CNN can be, for example, MobileNetv2, where the computational cost of the CNN is less than 0.1 GFLOPs. These methods all contribute to reducing the resource and power consumption of the head-mounted device.

[0029] Step S1200: Starting from the moment when the food sub-image is included in the first environmental image, the second environmental image is acquired periodically according to the second time interval, and the calories of the food corresponding to the food sub-image contained in the second environmental image are identified to obtain the calorie value of the food corresponding to the second environmental image.

[0030] The second environmental image is used to identify the calorie value of the food corresponding to the food sub-images contained within it. The second time interval is the time interval for acquiring the second environmental image. It should be noted that the second time interval is usually shorter than the first time interval; in one example, the second time interval is 2 minutes.

[0031] If the first environmental image includes a food sub-image, it is determined that the wearer is likely to eat. At this time, an environmental image is acquired according to the second time interval.

[0032] After acquiring the second environmental image, the calorie values ​​of the food corresponding to the food sub-images contained in the second environmental image are identified to obtain the calorie values ​​of the food corresponding to the second environmental image.

[0033] If the second environmental image is acquired periodically according to the second time interval, a food calorie value can be obtained every second time interval.

[0034] Step S1300: Determine the calorie intake of the wearer of the head-mounted device based on the calorie values ​​of the food obtained sequentially.

[0035] In one embodiment of this application, the specific implementation of step S1300 can be: subtracting the earliest obtained food calorie value from the latest obtained food calorie value to obtain the wearer's calorie intake.

[0036] In another embodiment of this application, in order to avoid the problem of incorrect determination of calorie intake due to misidentification of food calorie value, the calorie intake monitoring method provided in this application further includes the following steps S1310 to S1312 before the above step S1300.

[0037] Step S1310: Determine the food change trend based on the first sub-environment image and the second sub-environment image. The first sub-environment image and the second sub-environment image are adjacent second environment images, and the second sub-environment image is the next second environment image after the first sub-environment image.

[0038] Specifically, for any two adjacent acquired second environment images, the first acquired second environment image is designated as the first sub-environment image, and the second acquired second environment image is designated as the second sub-environment image. If the amount of food corresponding to the food sub-image in the second sub-environment image increases compared to the amount of food corresponding to the food sub-image in the first sub-environment image, then the food change trend is determined to be increasing; otherwise, the food change trend is determined to be decreasing.

[0039] Step S1311: Determine the food calorie change trend based on the food calorie values ​​corresponding to the first sub-environment image and the food calorie values ​​corresponding to the second sub-environment image.

[0040] Specifically, if the food calorie value corresponding to the second sub-environment image increases compared to the food calorie value corresponding to the first sub-environment image, then the trend of food calorie change is determined to be increasing; otherwise, the trend of food calorie change is determined to be decreasing.

[0041] Step S1312: If the food calorie change trend does not match the food change trend, remove the food calorie value corresponding to the second sub-environment image.

[0042] If the trend of food calorie change does not match the trend of food change, for example, the trend of food calorie change is increasing while the trend of food change is decreasing, or the trend of food calorie change is decreasing while the trend of food change is increasing, then it is determined that the food calorie value was misidentified based on the food calorie value corresponding to the second sub-environment image, and the food calorie value corresponding to the second sub-environment image is removed.

[0043] As can be seen from the above steps S1100 and S1200, the calorie intake monitoring method provided by this application can automatically and imperceptibly monitor the wearer's calorie intake without requiring manual triggering by the wearer. Furthermore, the calorie intake monitoring method provided by this application can be implemented using the image sensor on the head-mounted device, thus eliminating the need for an additional sensor module in the head-mounted device and preventing any increase in its weight or size.

[0044] Based on the above, this application provides a calorie intake monitoring method applied to a head-mounted device. The method includes: periodically acquiring a first environmental image at a first time interval until the first environmental image includes a food sub-image; starting from the moment the first environmental image includes a food sub-image, periodically acquiring a second environmental image at a second time interval, and identifying the calorie content of the food corresponding to the food sub-image in the second environmental image to obtain the calorie value of the food corresponding to the second environmental image; and determining the calorie intake of the wearer of the head-mounted device based on the sequentially obtained calorie values ​​of the food. This method can automatically and imperceptibly monitor the wearer's calorie intake without requiring manual triggering by the wearer. Furthermore, the calorie intake monitoring method provided by this application can be implemented using the image sensor on the head-mounted device, thus eliminating the need for an additional sensor module in the head-mounted device and preventing any increase in the weight and size of the head-mounted device.

[0045] In one embodiment of this application, the calorie intake monitoring method provided in this application further includes the following step S1400 after the above step S1200.

[0046] Step S1400: If the second environment image does not include a food sub-image, repeat the step of acquiring the first environment image periodically according to the first time interval until the first environment image includes a food sub-image.

[0047] If the second environmental image does not include a food sub-image, then it is determined that the wearer's current meal has ended. At this time, step S1100 is repeated to determine the wearer's calorie intake for the next meal.

[0048] In one embodiment of this application, the calorie intake monitoring method provided in this application can be specifically implemented in step S1100 by the following steps S1110 and S1111.

[0049] Step S1110: Obtain the wearer's motion data.

[0050] In this embodiment, a motion sensor is provided on the head-mounted device, which is used to collect the wearer's motion data.

[0051] In one example, the motion sensor is a combination of a 6DoF motion sensor and a gyroscope. Based on this, there is no need to set up an additional motion sensor on the head-mounted device; by reusing the 6DoF motion sensor and gyroscope on the head-mounted device, the wearer's motion data can be acquired.

[0052] Step S1111: If the wearer's movement is determined to be consistent with the eating movement based on the motion data, the first environmental image is acquired periodically according to the first time interval until the first environmental image includes a food sub-image.

[0053] In this embodiment, after acquiring the wearer's motion data, the data is analyzed to determine the wearer's actions. If the determined actions match eating, it indicates that the wearer is likely eating. Then, at a first time interval, a first environmental image is acquired periodically until the first environmental image includes a food sub-image.

[0054] Because eating is characterized by low amplitude, high frequency, and fixed direction, it differs significantly from other actions (such as walking). Therefore, it is possible to determine whether the wearer's actions conform to eating. If the wearer's actions conform to eating, then step S1100 is executed, which reduces the power consumption of the head-mounted device during calorie intake monitoring.

[0055] In one embodiment of this application, the calorie intake monitoring method provided by this application further includes the following steps S1120 and S1121 before the above step S1100.

[0056] Step S1120: If the current time is within at least one preset time period, update the first time interval to the third time interval.

[0057] The preset time period is the wearer's usual mealtime.

[0058] In one embodiment of this application, a lightweight model that is always running in the background can be used to learn the wearer's regular eating times.

[0059] Step S1121: If the current time is outside any preset time period, update the first time interval to the fourth time interval, wherein the duration of the fourth time interval is greater than the duration of the third time interval.

[0060] In this embodiment, if the current time is within at least one preset time period, it indicates that the wearer has a high probability of eating. In this case, the first time interval is set to a relatively small time interval. In this way, when executing step S1100, the first environmental image can be acquired more frequently, thereby monitoring the calorie intake more timely.

[0061] Conversely, if the current time falls outside any preset time period, it indicates a low probability that the wearer has eaten, and in this case, the first time interval is set to a relatively large interval. This further reduces the power consumption of the head-mounted device during calorie intake monitoring.

[0062] In one embodiment of this application, the identification of the calorie content of the food sub-image contained in the second environmental image in step S1200 is specifically achieved through step S1210 below.

[0063] Step S1210: Input the second environmental image into the preset heat recognition model, and the preset heat recognition model identifies the heat of the food corresponding to the food sub-image contained in the second environmental image.

[0064] The preset calorie recognition model is trained from a training sample set, which includes multiple training samples. Each training sample includes a food sample image and the corresponding food calorie value and calorie-related factors. The training sample set includes open-source training samples and real training samples.

[0065] Among them, the calorie-related factors are factors related to the calories in food. In one embodiment, the calorie-related factors include at least one of type, size, quantity, and cooking method.

[0066] In this embodiment, the preset calorie recognition model can be exemplarily the qwen3-2B-VL model. The training samples used to train the preset calorie recognition model include not only food sample images and their corresponding calorie values, but also calorie-related factors. This allows the preset calorie recognition model to learn more food features during training. When identifying the calorie content of food in sub-images within the second environmental image, the preset calorie recognition model can identify more accurate calorie values.

[0067] Furthermore, the training samples used to train the preset calorie recognition model cover both open-source training samples and real training samples. This reduces the need for collecting real training samples, lowering the training difficulty of the preset calorie recognition model. Additionally, the preset calorie recognition model can be quantized to reduce the computational resource consumption of the head-mounted device. In one example, the quantization precision is w4a16.

[0068] In one embodiment of this application, the calorie intake monitoring method provided by this application further includes the following steps S1500 and S1600.

[0069] Step S1500: Determine the wearer's eating time period based on the identification time corresponding to the successively obtained food calorie values.

[0070] In this embodiment, the acquisition time of the second environmental image is determined as the identification time of the corresponding food calorie value. Furthermore, based on the identification time corresponding to the food calorie value, the wearer's eating time period can be determined.

[0071] In one example, the identified times are time 1, time 2, time 3 and time 4, and the food calorie values ​​are calorie value A, calorie value B, calorie value C and calorie value C (no food was eaten from time 3 to time 4). Based on this, the eating period is the time period from time 3 to time 1.

[0072] Step S1600: Based on the eating time period and calorie intake, output eating suggestions, and / or, determine the eating time period as a preset time period.

[0073] For a healthy diet, the rate of calorie intake is typically close to a fixed rate, which can be set based on experience and the wearer's age. Therefore, the rate of calorie intake is determined by the duration of the meal, corresponding to the calorie intake amount. If the rate exceeds this fixed rate, it indicates that the wearer is eating quickly or consuming a large amount of calories in a short period. In this case, eating suggestions can be provided through voice playback or text display, such as suggesting slowing down the eating speed.

[0074] And / or, after obtaining the wearer's eating time period based on step S1500 above, this eating time period is used as a preset time period in step S1120 above. This allows for further learning of the wearer's regular eating time period.

[0075] It should be noted that when the head-mounted device executes the calorie intake monitoring method provided in the application, if other tasks conflict with it, the conflict can be handled according to the pre-set priority for the calorie intake monitoring method provided in the application. For example, if the priority for the calorie intake monitoring method provided in the application is set to the highest level, when the head-mounted device executes the calorie intake monitoring method provided in the application, if other tasks conflict with this method, the head-mounted device will prioritize executing the calorie intake monitoring method provided in the application.

[0076] Based on the above, in one embodiment of this application, the calorie intake monitoring method provided by this application includes the following steps S1001 to S1009: Step S1001, when the current time is within at least one preset time period, the first time interval is updated to a third time interval, wherein the preset time period is the wearer's regular eating time period; Step S1002, when the current time is outside any of the preset time periods, the first time interval is updated to a fourth time interval, wherein the duration of the fourth time interval is greater than the duration of the third time interval; Step S1003, the wearer's movement data is acquired; Step S1004, when it is determined from the movement data that the wearer's movement conforms to eating movement, a first environmental image is acquired periodically according to the first time interval until the first environmental image is obtained. The process includes: Step S1005: Starting from the moment when the first environmental image includes a food sub-image, acquiring a second environmental image periodically according to a second time interval; Step S1006: Inputting the second environmental image into a preset calorie recognition model, which identifies the calorie content of the food corresponding to the food sub-image in the second environmental image, thereby obtaining the calorie value of the food corresponding to the second environmental image; Step S1007: Determining the calorie intake of the wearer of the head-mounted device based on the sequentially obtained calorie values; Step S1008: Determining the eating time period of the wearer based on the recognition time corresponding to the sequentially obtained calorie values; Step S1009: Outputting eating suggestions based on the eating time period and the calorie intake, and / or determining the eating time period as a preset time period.

[0077] As shown in Figure 2, this application also provides a calorie intake monitoring device 200, applied to a head-mounted device. The device 200 includes: a first acquisition module 210, used to acquire a first environmental image at regular intervals according to a first time interval until the first environmental image includes a food sub-image; a second acquisition module 220, used to acquire a second environmental image at regular intervals according to a second time interval starting from the moment when the first environmental image includes a food sub-image; a recognition module 230, used to identify the calories of the food corresponding to the food sub-image contained in the second environmental image, and obtain the calorie value of the food corresponding to the second environmental image; and a monitoring module 240, used to determine the calorie intake of the wearer of the head-mounted device based on the sequentially obtained calorie values ​​of the food.

[0078] In one embodiment of this application, the calorie intake monitoring device 200 provided in this application further includes: a repeating module, used to repeat the step of acquiring the first environmental image at a fixed time interval until the first environmental image includes a food sub-image when the second environmental image does not include a food sub-image.

[0079] In one embodiment of this application, the first acquisition module 210 is specifically used to: acquire the wearer's motion data; and, if it is determined from the motion data that the wearer's actions conform to eating actions, acquire a first environmental image periodically at a first time interval until the first environmental image includes a food sub-image.

[0080] In one embodiment of this application, the calorie intake monitoring device 200 provided in this application further includes: an update module, configured to update the first time interval to a third time interval when the current time is within at least one preset time period, wherein the preset time period is the wearer's regular eating time period; and to update the first time interval to a fourth time interval when the current time is outside any of the preset time periods, wherein the duration of the fourth time interval is greater than the duration of the third time interval.

[0081] In one embodiment of this application, the recognition module 230 is specifically used to: input the second environmental image into a preset calorie recognition model, and have the preset calorie recognition model identify the calorie content of the food corresponding to the food sub-image contained in the second environmental image; wherein, the preset calorie recognition model is trained by a training sample set, the training sample set includes multiple training samples, each training sample includes a food sample image and the calorie value and calorie-related factors corresponding to the food sample image, and the training sample set includes open-source training samples and real training samples.

[0082] In one embodiment of this application, the heat-related factors include at least one of type, size, quantity, and cooking method.

[0083] In one embodiment of this application, the calorie intake monitoring device 200 provided in this application further includes: a rejection module, configured to determine a food change trend based on a first sub-environment image and a second sub-environment image, wherein the first sub-environment image and the second sub-environment image are adjacent second environment images, and the second sub-environment image is the next second environment image after the first sub-environment image; determine a food calorie change trend based on the food calorie value corresponding to the first sub-environment image and the food calorie value corresponding to the second sub-environment image; and reject the food calorie value corresponding to the second sub-environment image if the food calorie change trend does not match the food change trend.

[0084] In one embodiment of this application, the calorie intake monitoring device 200 provided in this application further includes: a determination module, configured to determine the eating time period of the wearer based on the identification time corresponding to the sequentially obtained food calorie values; an output module, configured to output eating suggestions based on the eating time period and the calorie intake; and / or, an update module, further configured to determine the eating time period as a preset time period.

[0085] This application also provides a head-mounted device, which includes any of the calorie intake monitoring devices 200 provided in the above-described device embodiments.

[0086] This application also provides another head-mounted device 300, as shown in FIG3, including a memory 310 and a processor 320, wherein the memory 310 is used to store computer instructions, and the processor 320 is used to call the computer instructions from the memory 310 to perform the method as described in any of the above method embodiments.

[0087] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of the above-described method embodiments.

[0088] This application may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this application.

[0089] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0090] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0091] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing the status information of the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0092] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0093] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0094] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be well known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0096] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this application is defined by the appended claims.

Claims

1. A method for monitoring calorie intake, characterized in that, The method, applied to a head-mounted device, includes: periodically acquiring a first environmental image at a first time interval until the first environmental image includes a food sub-image; starting from the moment when the first environmental image includes a food sub-image, periodically acquiring a second environmental image at a second time interval, and identifying the calories of the food corresponding to the food sub-image contained in the second environmental image to obtain the calorie value of the food corresponding to the second environmental image; and determining the calorie intake of the wearer of the head-mounted device based on the sequentially obtained calorie values ​​of the food.

2. The method according to claim 1, characterized in that, Starting from the moment when the first environmental image includes a food sub-image, the method acquires the second environmental image periodically according to a second time interval, identifies the calories of the food corresponding to the food sub-image contained in the second environmental image, and obtains the calorie value of the food corresponding to the second environmental image. The method further includes: when the second environmental image does not include a food sub-image, repeating the step of acquiring the first environmental image periodically according to a first time interval until the first environmental image includes a food sub-image.

3. The method according to claim 1, characterized in that, The step of periodically acquiring a first environmental image at a first time interval until the first environmental image includes a food sub-image includes: acquiring the wearer's motion data; and, if the motion data determines that the wearer's action is consistent with an eating action, periodically acquiring a first environmental image at a first time interval until the first environmental image includes a food sub-image.

4. The method according to claim 2 or 3, characterized in that, Before acquiring the first environmental image periodically according to the first time interval until the first environmental image includes a food sub-image, the method further includes: updating the first time interval to a third time interval when the current time is within at least one preset time period, the preset time period being the wearer's regular eating time period; updating the first time interval to a fourth time interval when the current time is outside any of the preset time periods, wherein the duration of the fourth time interval is greater than the duration of the third time interval.

5. The method according to claim 1, characterized in that, The step of identifying the calorie content of the food corresponding to the food sub-image contained in the second environmental image includes: inputting the second environmental image into a preset calorie recognition model, and having the preset calorie recognition model identify the calorie content of the food corresponding to the food sub-image contained in the second environmental image; wherein, the preset calorie recognition model is trained from a training sample set, the training sample set includes multiple training samples, each training sample includes a food sample image and the calorie value and calorie-related factors corresponding to the food sample image, and the training sample set includes open-source training samples and real training samples.

6. The method according to claim 5, characterized in that, The heat-related factors include at least one of the following: type, size, quantity, and cooking method.

7. The method according to claim 1, characterized in that, Before determining the calorie intake of the wearer of the head-mounted device based on the sequentially obtained food calorie values, the method further includes: determining a food change trend based on a first sub-environment image and a second sub-environment image, wherein the first sub-environment image and the second sub-environment image are adjacent second environment images, and the second sub-environment image is the next second environment image after the first sub-environment image; determining a food calorie change trend based on the food calorie values ​​corresponding to the first sub-environment image and the food calorie values ​​corresponding to the second sub-environment image; and, if the food calorie change trend does not match the food change trend, discarding the food calorie values ​​corresponding to the second sub-environment image.

8. The method according to claim 4, characterized in that, After determining the calorie intake of the wearer of the head-mounted device based on the sequentially obtained calorie values ​​of the food, the method further includes: determining the wearer's eating time period based on the identification time corresponding to the sequentially obtained calorie values ​​of the food; outputting eating suggestions based on the eating time period and the calorie intake; and / or determining the eating time period as a preset time period.

9. A head-mounted device, characterized in that, It includes a memory and a processor, the memory being used to store computer instructions, and the processor being used to retrieve the computer instructions from the memory to perform the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images