Food information identification method, device, electronic device and computer readable medium

Abstract

The embodiments of the present disclosure disclose a method, device, electronic device and computer-readable medium for identifying food information. A specific implementation of the method includes: acquiring a target food image; inputting the target food image into a first food recognition model to generate first target food information; generating modification information based on the first target food information; generating a second food recognition model based on the modification information; inputting the target food image into the second food recognition model to generate second target food information; and sending the second target food information to a display device of a target terminal for display. This implementation increases the accuracy of food information generation, reduces the waiting time for users in queues and improves the efficiency of food settlement.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a food information recognition method, device, electronic device, and computer-readable medium. Background Art

[0002] In the field of smart canteens, the research on food information recognition technology has become a core issue in this field. At present, commonly used food information recognition technologies often use a fixed training sample set to train the initial food recognition model to generate a food recognition model, thereby identifying food information.

[0003] However, when the above method is used to identify food information, the following technical problems often occur:

[0004] First, since the dishes in the cafeteria change frequently, the method of using a fixed training sample set to train the initial food recognition model to generate a food recognition model has low recognition accuracy for new foods, which increases the waiting time for users and reduces the efficiency of food settlement.

[0005] Second, because traditional food recognition models often do not consider the correlation between multiple labels in food information when performing food recognition, the generated food recognition model has low recognition accuracy for food information, which leads to low food circulation efficiency. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose food information recognition methods, devices, electronic devices and computer-readable media to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a method for identifying food information, the method comprising: obtaining a target food image. Inputting the target food image into a first food identification model to generate first target food information. Based on the first target food information, generating modification information. Based on the modification information, generating a second food identification model. Inputting the target food image into the second food identification model to generate second target food information. Sending the second target food information to a display device of a target terminal for display.

[0009] In a second aspect, some embodiments of the present disclosure provide a food information identification device, the device comprising: an acquisition unit, configured to acquire a target food image. A first generation unit, configured to input the target food image into a first food recognition model to generate first target food information. A second generation unit, configured to generate modification information based on the first target food information. A third generation unit, configured to generate a second food recognition model based on the modification information. A fourth generation unit, configured to input the target food image into the second food recognition model to generate second target food information. A display unit, configured to send the second target food information to a display device of a target terminal for display.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0012] The above-mentioned embodiments of the present disclosure have the following beneficial effects: the food information obtained by the food information recognition method of some embodiments of the present disclosure improves the accuracy of the generated food information, thereby reducing the waiting time of users in line, and further improving the settlement efficiency of food. Specifically, the inventors found that the reason why the results of the relevant food recognition methods are not accurate enough is that since the dishes are dynamically changing, the food recognition model generated by training the initial food recognition model with a fixed training sample set has low recognition accuracy for new dishes. Based on this, the food information recognition method of some embodiments of the present disclosure sets the training samples to be dynamically changing. In the process of dish recognition, if the dish information does not match the dish image, the dish image and the corresponding correct dish information are added to the training sample data set, and the model is retrained. In this way, the generated food recognition model can adapt to the changes of dishes. Through continuous learning, the representation ability of the food recognition model is improved. In turn, the accuracy of food recognition is improved, the waiting time of users in line is reduced, and the settlement efficiency of food is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 is a schematic diagram of an application scenario of the food information recognition method of some embodiments of the present disclosure;

[0015] Figure 2 is a flow chart of some embodiments of the food information identification method according to the present disclosure;

[0016] Figure 3 is a flow chart of some embodiments of a food information identification device according to the present disclosure;

[0017] Figure 4 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0024] Figure 1 It is a schematic diagram of an application scenario of the food information recognition method according to some embodiments of the present disclosure.

[0025] exist Figure 1 In the application scenario diagram, first, the computing device 101 can obtain the target food image 102. Then, the computing device 101 can input the above target food image 102 into the first food recognition model 103 to generate the first target food information 104. Secondly, the computing device 101 can generate modification information 105 based on the above first target food information 104. Furthermore, the computing device 101 can generate a second food recognition model 106 based on the above modification information 105. In addition, the computing device 101 can input the above target food image 102 into the above second food recognition model 106 to generate the second target food information 107. Finally, the computing device 101 can send the above second target food information 107 to the display device 108 of the target terminal for display.

[0026] It should be noted that the computing device 101 can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, or as a single software or software module. No specific limitation is made here.

[0027] It should be understood that Figure 1 The number of user device information is only for illustration purposes. Any number of computing devices may be provided as required.

[0028] Continue to refer Figure 2 , shows a process 200 of some embodiments of the food information identification method according to the present disclosure. The above-mentioned food information identification method comprises the following steps:

[0029] Step 201, obtaining a target food image.

[0030] In some embodiments, the execution subject of the food information identification method (such as Figure 1 The computing device 101 shown in the figure can obtain the above-mentioned target food image through a wired connection or a wireless connection. The above-mentioned target food image can be an image of a dish or an image of a drink.

[0031] Step 202: input the target food image into a first food recognition model to generate first target food information.

[0032] In some embodiments, the execution entity may input the target food image into the first food recognition model to generate the first target food information. The first food recognition model may be a model of various structures. For example, CNN (Convolutional Neural Networks), RNN (Recurrent Neural Networks), etc. Of course, it may also be a model built according to actual needs.

[0033] In some optional implementations of some embodiments, the execution subject may input the target food image into a first food recognition model to generate first target food information, wherein the first food recognition model may be trained and generated by the following steps:

[0034] The first step is to obtain samples to be trained. The samples to be trained may include: samples to be trained images and samples to be trained labels. The samples to be trained labels may include: samples to be trained names, samples to be trained categories and samples to be trained attribute values. The samples to be trained attribute values ​​are used to characterize the value transfer values ​​(e.g., dish prices) of dishes corresponding to the samples to be trained images.

[0035] As an example, the sample to be trained may be [image 0, scrambled eggs with tomatoes, vegetarian dish, 5 yuan].

[0036] In the second step, in response to determining that the sample to be trained is not in the preset training sample set, the sample to be trained is added to the preset training sample set to obtain the training sample set.

[0037] As an example, the preset training sample set may be [[image 1, beef brisket stewed with potatoes, meat dish, 10 yuan], [image 2, sweet and sour pork ribs, meat dish, 12 yuan], [image 3, shredded pork with fish flavor, meat dish, 12 yuan], [image 4, boiled fish, meat dish, 15 yuan], [image 5, Kung Pao chicken, meat dish, 15 yuan], [image 6, Mapo tofu, vegetarian dish, 10 yuan], [image 7, braised pork in brown sauce, meat dish, 20 yuan], [image 8, husband and wife lung slices, meat dish, 16 yuan], [image 9, twice-cooked pork, meat dish, 17 yuan], [image 10, cola chicken wings, meat dish, 15 yuan]]. The sample to be trained may be [image 0, scrambled eggs with tomatoes, vegetarian dish, 5 yuan]. In response to determining that the above-mentioned samples to be trained are not in the preset training sample set, the above-mentioned samples to be trained are added to the above-mentioned preset training sample set, and the obtained training sample set can be [[Image 1, beef brisket stewed with potatoes, meat, 10 yuan], [Image 2, sweet and sour pork ribs, meat, 12 yuan], [Image 3, shredded pork with fish flavor, meat, 12 yuan], [Image 4, boiled fish, meat, 15 yuan], [Image 5, Kung Pao chicken, meat, 15 yuan], [Image 6, Mapo tofu, vegetarian, 10 yuan], [Image 7, braised pork in brown sauce, meat, 20 yuan], [Image 8, husband and wife lung slices, meat, 16 yuan], [Image 9, twice-cooked pork, meat, 17 yuan], [Image 10, cola chicken wings, meat, 15 yuan], [Image 0, scrambled eggs with tomatoes, vegetarian, 5 yuan]].

[0038] The third step is to select at least one training sample from the training sample set to form a training sample group. The training sample may include: a training sample image and a training sample label, and the training sample label may include: a training sample name, a training sample category and a training sample attribute value.

[0039] As an example, at least one training sample is selected from the above training sample set to form a training sample group, which may be [[Image 1, beef brisket stewed with potatoes, meat, 10 yuan], [Image 2, sweet and sour spare ribs, meat, 12 yuan], [Image 3, shredded pork with fish flavor, meat, 12 yuan], [Image 4, boiled fish, meat, 15 yuan]].

[0040] In the fourth step, the training sample images included in each training sample in the training sample group are input into the initial food recognition model to generate a first food information group. The first food information may include: a first food name, a first food category and a first food attribute value. The initial food recognition model may be a CNN (Convolutional Neural Networks) that has not been trained.

[0041] The fifth step is to generate a first name score and a second name score based on the first food name included in each first food information in the above-mentioned first food information group and the training sample name included in each training sample in the above-mentioned training sample group, and obtain a first name score group and a second name score group.

[0042] In the sixth step, based on the first food category included in each first food information in the first food information group and the training sample category included in each training sample in the training sample group, a first category score is generated to obtain a first category score group.

[0043] In the seventh step, based on the first food information group, the training sample group, the first name score group, the second name score group and the first category score group, the total loss value is determined.

[0044] Optionally, a total loss value may be generated based on the first food information group, the training sample group, the first name score group, the second name score group and the first category score group by using the following formula:

[0045]

[0046] Wherein, Loss represents the total loss value. J1 represents the first loss value. J2 represents the second loss value. J3 represents the third loss value. N represents the number of training samples in the above training sample group. i represents the sequence number. v represents the first food attribute value included in the first food information in the above first food information group. u represents the training sample attribute value included in the training sample in the above training sample group. T represents the first category score in the above first category score group. e represents the second name score in the above second name score group. c represents the first name score in the above first name score group. c i represents the i-th first name score in the above first name score group. i represents the score of the ith second name in the above second name score group. i represents the i-th first category score in the above first category score group. t represents the preset output parameter. p represents the preset boundary value. u i Indicates the training sample attribute value included in the i-th training sample of the above training sample group. i Represents the first food attribute value included in the i-th first food information in the above-mentioned first food information group.

[0047] The above formula, as an inventive point of an embodiment of the present disclosure, solves the second technical problem mentioned in the background technology, "Because the traditional food recognition model often does not consider the correlation between multiple labels in the food information when performing food recognition, the generated food recognition model has low recognition accuracy for the food information, thereby making the food circulation efficiency low". The reasons for the low food circulation efficiency are often as follows: the traditional food recognition model often fails to comprehensively consider the relationship between the various labels in the food information during the recognition process, so that the result generation is asynchronous. If the above factors are solved, the recognition accuracy of the food information can be improved, thereby improving the food circulation efficiency. In order to achieve this effect, the present disclosure uses different lightweight loss functions to estimate the loss of food names, food categories and food attribute values, and then sums the obtained loss values. By summing the obtained loss values, the consideration of each label in the food information is achieved. This solves the problem that the traditional food recognition model often does not consider the correlation between multiple labels in the food information when performing food recognition, thereby making the generated food recognition model have low recognition accuracy for the food information, thereby making the food circulation efficiency low.

[0048] In step 8, in response to determining that the total loss value converges to a predetermined threshold, the initial food recognition model is determined as a first food recognition model, wherein the predetermined threshold may be 0.001.

[0049] In step nine, in response to determining that the total loss value does not converge to a predetermined threshold, a candidate food recognition model is generated.

[0050] In the tenth step, the above candidate food recognition model is determined as the initial food recognition model.

[0051] Step 203: Generate modification information based on the first target food information.

[0052] In some embodiments, the above-mentioned execution entity can generate modification information based on the first target food information in various ways.

[0053] Optionally, the execution subject generates the modification information based on the first target food information, which may include the following steps:

[0054] The first step is to send the first target food information to a target terminal, wherein the target terminal may be a computing device with display and sending functions.

[0055] The second step is to respond to receiving the pre-modification information sent by the target terminal, wherein the pre-modification information may be generated after the target device processes the first target food information.

[0056] The third step is to determine the pre-modification information as the modification information.

[0057] As an example, the first target food information may be [scrambled eggs with tomatoes, vegetarian, 20 yuan]. The pre-modified information may be [scrambled eggs with tomatoes, vegetarian, 5 yuan]. The modified information may be [scrambled eggs with tomatoes, vegetarian, 5 yuan].

[0058] Step 204: Generate a second food recognition model based on the modified information

[0059] In some embodiments, the execution entity may generate a second food recognition model in various ways based on the modification information. The second food recognition model may be a model of various structures. For example, CNN (Convolutional Neural Networks), RNN (Recurrent Neural Networks), etc. Of course, it may also be a model built according to actual needs.

[0060] In some optional implementations of some embodiments, the execution subject generates a second food recognition model based on the modification information, which may include the following steps:

[0061] In the first step, the modification information and the target food image are determined as modification samples.

[0062] As an example, the modification information may be [scrambled eggs with tomatoes, vegetarian, 5 yuan]. The target food image may be [image 0]. The modification sample determined by combining the modification information and the target food image may be [image 0, scrambled eggs with tomatoes, vegetarian, 5 yuan].

[0063] The second step is to add the modified samples to the training sample set to obtain a modified training sample set.

[0064] As an example, the modified sample may be [image 0, scrambled eggs with tomatoes, vegetarian dish, 5 yuan]. The training sample set may be [[image 1, beef brisket stewed with potatoes, meat dish, 10 yuan], [image 2, sweet and sour pork ribs, meat dish, 12 yuan], [image 3, shredded pork with fish flavor, meat dish, 12 yuan], [image 4, boiled fish, meat dish, 15 yuan], [image 5, Kung Pao chicken, meat dish, 15 yuan], [image 6, Mapo tofu, vegetarian dish, 10 yuan], [image 7, braised pork in brown sauce, meat dish, 20 yuan], [image 8, husband and wife lung slices, meat dish, 16 yuan], [image 9, twice-cooked pork, meat dish, 17 yuan], [image 10, cola chicken wings, meat dish, 15 yuan], [image 0, scrambled eggs with tomatoes, vegetarian dish, 5 yuan]]. Add the above modified samples to the above training sample set, and the modified training sample set can be [[Image 1, beef brisket stewed with potatoes, meat, 10 yuan], [Image 2, sweet and sour pork ribs, meat, 12 yuan], [Image 3, shredded pork with fish flavor, meat, 12 yuan], [Image 4, boiled fish, meat, 15 yuan], [Image 5, Kung Pao chicken, meat, 15 yuan], [Image 6, Mapo tofu, vegetarian, 10 yuan], [Image 7, braised pork in brown sauce, meat, 20 yuan], [Image 8, husband and wife lung slices, meat, 16 yuan], [Image 9, twice-cooked pork, meat, 17 yuan], [Image 10, cola chicken wings, meat, 15 yuan], [Image 0, scrambled eggs with tomatoes, vegetarian, 5 yuan], [Image 0, scrambled eggs with tomatoes, vegetarian, 5 yuan]].

[0065] The third step is to generate a second food recognition model based on the modified training sample set.

[0066] Among them, the training and generation steps of the above-mentioned second food recognition model can refer to the training and generation steps of the above-mentioned first food recognition model, and will not be repeated here.

[0067] Step 205: input the target food image into a second food recognition model to generate second target food information.

[0068] In some embodiments, the execution entity may input the target food image into the second food recognition model to generate second target food information.

[0069] Step 206: Send the second target food information to a display device of the target terminal for display.

[0070] In some embodiments, the execution subject may send the second target food information to a display device of the target terminal and display the second target food information on the display device of the target terminal, wherein the display device of the target terminal may be a computer display.

[0071] The above-mentioned embodiments of the present disclosure have the following beneficial effects: the food information obtained by the food information recognition method of some embodiments of the present disclosure improves the accuracy of the generated food information, thereby reducing the waiting time of users in line, and further improving the settlement efficiency of food. Specifically, the inventors found that the reason why the results of the relevant food recognition methods are not accurate enough is that since the dishes are dynamically changing, the food recognition model generated by training the initial food recognition model with a fixed training sample set has low recognition accuracy for new dishes. Based on this, the food information recognition method of some embodiments of the present disclosure sets the training samples to be dynamically changing. In the process of dish recognition, if the dish information does not match the dish image, the dish image and the corresponding correct dish information are added to the training sample data set, and the model is retrained. In this way, the generated food recognition model can adapt to the changes of dishes. Through continuous learning, the representation ability of the food recognition model is improved. Further, the accuracy of food recognition is improved, the waiting time of users in line is reduced, and the settlement efficiency of food is improved.

[0072] Further references Figure 3 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a food information identification device. These device embodiments are similar to Figure 2 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0073] like Figure 3 As shown, in some embodiments, the food information recognition device 300 includes: an acquisition unit 301, a first generation unit 302, a second generation unit 303, a third generation unit 304, a fourth generation unit 305, and a display unit 306. The acquisition unit 301 is configured to acquire a target food image; the first generation unit 302 is configured to input the target food image into a first food recognition model to generate first target food information; the second generation unit 303 is configured to generate modification information based on the first target food information; the third generation unit 304 is configured to generate a second food recognition model based on the modification information; the fourth generation unit 305 is configured to input the target food image into the second food recognition model to generate second target food information; and the display unit 306 is configured to send the second target food information to a display device of a target terminal for display.

[0074] It is understood that the units described in the device 300 are similar to those described in the reference Figure 2Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 300 and the units contained therein, and will not be described in detail here.

[0075] Reference below Figure 4 , which shows an electronic device (eg, Figure 1 Schematic diagram of the structure of the computing device 101)400. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0076] like Figure 4 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 404 is also connected to the bus 404.

[0077] Typically, the following devices may be connected to the I / O interface 404: input devices 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 408 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 409. The communication devices 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 4 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0078] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0079] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0080] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0081] The computer-readable medium may be included in the device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains a target food image; inputs the target food image into a first food recognition model to generate first target food information; generates modification information based on the first target food information; generates a second food recognition model based on the modification information; inputs the target food image into the second food recognition model to generate second target food information; and sends the second target food information to a display device of the target terminal for display.

[0082] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0083] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0084] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including an acquisition unit, a first generation unit, a second generation unit, a third generation unit, a fourth generation unit, and a display unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the acquisition unit may also be described as a "unit for acquiring a target food image".

[0085] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0086] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure to form a technical method.

Claims

1. A method for identifying food information, comprising: Acquire a target food image; inputting the target food image into a first food recognition model to generate first target food information; generating modification information based on the first target food information; generating a second food recognition model based on the modification information; inputting the target food image into the second food recognition model to generate second target food information; sending the second target food information to a display device of a target terminal for display; The first food recognition model is trained and generated by the following steps: Selecting at least one training sample from the training sample set to form a training sample group, wherein the training sample includes: a training sample image and a training sample label, and the training sample label includes: a training sample name, a training sample category and a training sample attribute value; Inputting the training sample images included in each training sample in the training sample group into the initial food recognition model to generate a first food information group, wherein the first food information includes: a first food name, a first food category and a first food attribute value; determining a total loss value of the first food information group; in response to determining that the total loss value converges to a predetermined threshold, determining the initial food recognition model as a first food recognition model; In response to determining that the total loss value does not converge to a predetermined threshold, generating a candidate food recognition model; determining the candidate food recognition model as an initial food recognition model; Wherein, determining the total loss value of the first food information group includes: Based on the first food name included in each first food information in the first food information group and the training sample name included in each training sample in the training sample group, a first name score and a second name score are generated to obtain a first name score group and a second name score group; Based on the first food category included in each first food information in the first food information group and the training sample category included in each training sample in the training sample group, a first category score is generated to obtain a first category score group; Determine the total loss value based on the first food information group, the training sample group, the first name score group, the second name score group and the first category score group; Wherein, determining the total loss value based on the first food information group, the training sample group, the first name score group, the second name score group and the first category score group includes: Based on the first food information group, the training sample group, the first name score group, the second name score group and the first category score group, a total loss value is generated by the following formula: Wherein, Loss represents the total loss value, J1 represents the first loss value, J2 represents the second loss value, J3 represents the third loss value, N represents the number of training samples in the training sample group, i represents the sequence number, v represents the first food attribute value included in the first food information in the first food information group, u represents the training sample attribute value included in the training sample of the training sample group, T represents the first category score in the first category score group, e represents the second name score in the second name score group, c represents the first name score in the first name score group, and c represents the first name score in the first name score group. i represents the score of the i-th first name in the first name score group, e i represents the score of the i-th second name in the second name score group, T i represents the i-th first category score in the first category score group, t represents the preset output parameter, p represents the preset boundary value, u i represents the training sample attribute value included in the i-th training sample of the training sample group, v i Represents the first food attribute value included in the i-th first food information in the first food information group.

2. The method according to claim 1, wherein: The training sample set is generated by the following steps: Acquire a sample to be trained, wherein the sample to be trained includes: a sample image to be trained and a sample label to be trained, and the sample label to be trained includes: a sample name to be trained, a sample category to be trained and a sample attribute value to be trained; In response to determining that the sample to be trained is not in the preset training sample set, the sample to be trained is added to the preset training sample set to obtain the training sample set.

3. The method according to claim 2, wherein: The step of generating a second food recognition model based on the modification information includes: determining the modification information and the target food image as a modification sample; Adding the modified sample to the training sample set to obtain a modified training sample set; Based on the modified training sample set, a second food recognition model is generated.

4. A food information recognition device, comprising: an acquisition unit configured to acquire a target food image; a first generating unit configured to input the target food image into a first food recognition model to generate first target food information; a second generating unit, configured to generate modification information based on the first target food information; a third generating unit, configured to generate a second food recognition model based on the modification information; a fourth generating unit, configured to input the target food image into the second food recognition model to generate second target food information; A display unit, configured to send the second target food information to a display device of a target terminal for display; The first food recognition model is trained and generated by the following steps: Selecting at least one training sample from the training sample set to form a training sample group, wherein the training sample includes: a training sample image and a training sample label, and the training sample label includes: a training sample name, a training sample category and a training sample attribute value; Inputting the training sample images included in each training sample in the training sample group into the initial food recognition model to generate a first food information group, wherein the first food information includes: a first food name, a first food category and a first food attribute value; determining a total loss value of the first food information group; in response to determining that the total loss value converges to a predetermined threshold, determining the initial food recognition model as a first food recognition model; In response to determining that the total loss value does not converge to a predetermined threshold, generating a candidate food recognition model; determining the candidate food recognition model as an initial food recognition model; Wherein, determining the total loss value of the first food information group includes: Based on the first food name included in each first food information in the first food information group and the training sample name included in each training sample in the training sample group, a first name score and a second name score are generated to obtain a first name score group and a second name score group; Based on the first food category included in each first food information in the first food information group and the training sample category included in each training sample in the training sample group, a first category score is generated to obtain a first category score group; Determine the total loss value based on the first food information group, the training sample group, the first name score group, the second name score group and the first category score group; Wherein, determining the total loss value based on the first food information group, the training sample group, the first name score group, the second name score group and the first category score group includes: Based on the first food information group, the training sample group, the first name score group, the second name score group and the first category score group, a total loss value is generated by the following formula: Wherein, Loss represents the total loss value, J1 represents the first loss value, J2 represents the second loss value, J3 represents the third loss value, N represents the number of training samples in the training sample group, i represents the sequence number, v represents the first food attribute value included in the first food information in the first food information group, u represents the training sample attribute value included in the training sample of the training sample group, T represents the first category score in the first category score group, e represents the second name score in the second name score group, c represents the first name score in the first name score group, and c represents the first name score in the first name score group. i represents the score of the i-th first name in the first name score group, e i represents the score of the i-th second name in the second name score group, T i represents the i-th first category score in the first category score group, t represents the preset output parameter, p represents the preset boundary value, u i represents the training sample attribute value included in the i-th training sample of the training sample group, v i Represents the first food attribute value included in the i-th first food information in the first food information group.

5. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 3.

6. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images