List generation method and electronic device

Abstract

This application discloses a list generation method and an electronic device. The list generation method includes: obtaining an image; inputting the image into a target model for inference; obtaining prompt words, which are input into the target model to guide the target model to infer the image; obtaining the inference result of the target model based on the prompt words for the image; whenever the inference result meets the item output condition, generating the output content of an item in the list based on the inference result; and outputting the list to represent the recognition result of the image.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a list generation method and electronic device. Background Technology

[0002] With frequent international cultural exchanges and business dealings, there is an increasing need to consult documents written in foreign languages, such as cargo lists in international business or menus when dining at international restaurants.

[0003] Users typically search for text information manually, but this method requires users to manually input each character, which is very time-consuming and results in a poor user experience. Alternatively, users can use a translator to help translate and understand text information, but this method requires taking a picture of the text information and inputting the corresponding image into the translator. The translator needs to recognize and translate all the text information in the image before it can output the whole thing, which also requires users to wait a long time. Summary of the Invention

[0004] The purpose of this application is to provide a list generation method and an electronic device.

[0005] In a first aspect, embodiments of this application provide a list generation method, including: Obtain an image; the image is used as input to the target model for inference. Obtain prompt words, which are used to input the target model to guide the target model to reason about the image; Obtain the inference result of the target model based on the prompt words for the image; Whenever the reasoning result satisfies the entry output condition, an entry in the list is generated based on the reasoning result; the list is used to output the recognition result of the image.

[0006] In one possible implementation, the reasoning result is a series of consecutively output characters, and the number of entries in the list is increased sequentially as the series of consecutively output characters are displayed.

[0007] In one possible implementation, the step of generating an entry for the list based on the reasoning result whenever the reasoning result satisfies the entry output condition includes: If the reasoning result satisfies the entry output condition each time, the output content of one entry in the list is generated based on the reasoning result, until the target model completes the character output sequentially for the image reasoning process based on the prompt words; Each time the output condition of an entry is met, an entry is added to the list sequentially based on the output content of the generated entry.

[0008] In one possible implementation, the prompt word includes a format sample, and the target model outputs multiple characters consecutively from the format sample based on the inference result of the prompt word for the image.

[0009] In one possible implementation, the step of generating an entry for a list based on the inference result whenever the inference result satisfies the entry output condition further includes: For each character obtained that represents the reasoning result, determine whether the entry output condition is met.

[0010] In one possible implementation, determining whether the entry output condition is satisfied for each character representing the reasoning result obtained includes: Each time a character representing the reasoning result is obtained, the obtained character is processed by a parser. The parser includes multiple judgment conditions, and the entry output condition belongs to the multiple judgment conditions. The multiple judgment conditions are related to the characters included in the format example.

[0011] In one possible implementation, the processing of each obtained character by the parser includes: For each character obtained, determine whether that character belongs to the string; If the character is a string, store it in the buffer and obtain the next character; If it does not belong to a string, determine whether it indicates the end of the string; If the character represents the end of the string, output the string in the output buffer and obtain the next character; If the end of the string is not indicated, determine whether it belongs to a structure character; If it belongs to a structure character, perform the processing corresponding to the structure character and obtain the next character; If it is not a structure character, perform the corresponding processing according to the initialization content and obtain the next character; Continue until all characters are obtained.

[0012] In one possible implementation, before processing each obtained character by the parser, the following is included: Initialize the key-value pairs to be parsed, the structure characters and their corresponding processing, and other characters and their corresponding processing.

[0013] In one possible implementation, the list generation method includes: Obtain the menu image and menu hints; Obtain the menu inference result output by the target model based on the menu prompt words for the menu image; Whenever the menu reasoning result satisfies the item output condition, an item in the list is generated based on the reasoning result; the list is used to output the recognition result of the image. Based on a local database and / or network interface, a mapping relationship is obtained; wherein, the mapping relationship includes at least the correspondence between different languages ​​and the correspondence between different currencies; Based on the mapping relationship, the output content is transformed to obtain the transformed menu image, which is then displayed.

[0014] Secondly, embodiments of this application also provide an electronic device, including a processor and a display interconnected with each other; The processor acquires an image; the image is input to a target model for inference; acquires prompt words, the prompt words are input to the target model to guide the target model inference about the image; acquires the inference result output by the target model based on the prompt words for the image; whenever the inference result satisfies the item output condition, it generates the output content of an item in a list based on the inference result; the list is output to represent the recognition result of the image; and the output content is transmitted to a display. The display shows the output content. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart of a list generation method provided in this application is shown; Figure 2 This application provides a flowchart of a method for processing each obtained character using a parser. Figure 3 This application provides another method for processing each obtained character using a parser; Figure 4 This application provides a complete flowchart from obtaining the menu image to displaying it to the user. Figure 5 This diagram illustrates the output display of the converted menu image in a traditional solution. Figure 6 This illustration shows a schematic diagram of the output display of the converted menu image provided by the generation method of this application; Figure 7 A schematic diagram of the structure of an electronic device provided in this application is shown; Figure 8 A schematic diagram of the structure of a computer device provided in this application is shown. Detailed Implementation

[0017] Various embodiments and features of this application are described herein with reference to the accompanying drawings.

[0018] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.

[0019] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0020] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0021] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application, which have the features described in the claims and are therefore all within the scope of protection defined herein.

[0022] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0023] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.

[0024] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.

[0025] To facilitate understanding of this application, a list generation method provided in this application will be described in detail below. The execution entity of the list generation method in this application can be a processor or controller of an electronic device, etc., and this application does not limit this.

[0026] With the application of visual-language models and large language models, their inference results are usually generated step by step in a continuous output manner, rather than returning a complete structured result all at once. In other words, the inference result needs to wait for the model to form a complete result in a standard output format (such as JSON) and output it, and then be parsed and displayed by the parser. This will result in a long delay in the output of the first piece of usable information, which cannot meet the needs of real-time interactive scenarios. On the other hand, if the continuous output content is directly parsed in a conventional way, it is easy to cause parsing failure, inconsistent results, or waste of system resources because the output content may be in an incomplete structural state at any time.

[0027] For the inference results continuously output by the model, embodiments of this application design a streaming parsing and item-level output mechanism that does not rely on the complete generation of the final result. Specifically, before the inference results output by the target model (e.g., a vision-language model) are formed into a standard output format (e.g., JSON), the item-level output mechanism checks whether the continuously output inference results of the target model constitute an independently usable structured information unit. Each time an item is formed, one entry is displayed and output until the inference results are completely output (i.e., a standard output format is formed).

[0028] As an example, Figure 1 A flowchart of a list generation method provided in an embodiment of this application is shown, wherein the steps specifically include S101-S104.

[0029] S101, Obtain the image; the image is used as input to the target model for inference.

[0030] Optionally, the method of obtaining the image can be determined based on the application scenario. For example, if the application scenario is a user ordering food at a restaurant abroad, and the image is a photograph of the menu, the method of obtaining the image could be to take a picture of the menu using a mobile phone camera or to take a picture of the menu using an image capture device connected to a specific translator. If the application scenario is a user looking up a business list, and the image is a photograph of the business list, the method of obtaining the image could be to take a picture of the business list using a mobile phone camera or to scan the business list using a scanning device connected to a computer, etc.

[0031] Of course, if there are corresponding electronic versions of menus, business lists, etc., you can obtain the electronic versions directly through electronic devices and obtain the images through format conversion or screenshots.

[0032] After obtaining the image, the image is input into the target model so that the target model can perform inference.

[0033] S102, Obtain the prompt words. The prompt words are used to input the target model to guide the target model to perform inference on the image.

[0034] For example, for different application scenarios, corresponding prompts can be pre-set to assist the target model in reasoning and ensure the accuracy of the target model's reasoning. These prompts can include prompts for summarizing or describing, prompts representing text categories, and prompts that restrict the processing rules of the target model.

[0035] After obtaining the prompt words, the prompt words are also input into the target model to guide the target model to make inferences about the image.

[0036] S103, obtain the inference results of the target model based on the prompt words for the image output.

[0037] Optionally, the target model in this application embodiment can be a Vision-Language Model (VLM). The VLM establishes semantic associations between visual information (such as images and videos) and language information (such as text and speech-to-text) to complete cross-modal recognition, matching and reasoning tasks.

[0038] For example, the target model can identify images based on prompt words to output inference results for the images. These inference results are multiple consecutively output characters, which can include characters, numbers, and symbols.

[0039] Optionally, the prompt words include a format example. Based on this, the target model infers multiple characters consecutively from the image output based on the prompt words, satisfying the format example. For example, in the application scenario of an image corresponding to a menu, the format example can be set to {"Dish Name": "Dish Name 1", "Dish Price": Price 1, "DishDescription": "Description 1"}. Based on this, the target model recognizes the text information contained in the obtained image according to the above format example and outputs multiple characters consecutively according to the format example.

[0040] S104, whenever the reasoning result satisfies the entry output condition, generate the output content of one entry in the list based on the reasoning result; the list is used to output the recognition result of the image.

[0041] After obtaining the reasoning result corresponding to the image, it is further determined whether the reasoning result meets the target output condition. For example, when determining whether the reasoning result meets the target output condition, since the output result consists of multiple consecutively output characters, it is possible to determine whether the current reasoning result meets the entry output condition each time a character representing the reasoning result is obtained.

[0042] During the process of obtaining the inference result, and during the process of obtaining each character and judging whether the entry output condition is met, each time the inference result meets the entry output condition, the output content of one entry in the list is generated based on the inference result, until the target model completes the character output sequentially for the image inference process based on the prompt words.

[0043] Each time an item's output condition is met, an item is added to the list based on the output content of that generated item. In other words, the number of items in the list increases sequentially with each consecutively output character, until the target model has completed outputting the characters sequentially based on the prompt words for the image's reasoning process, at which point the list is also generated.

[0044] The list generation method in this application employs streaming parsing and output, eliminating the need to parse all image information. It can output partial image information quickly, and while the user reads the currently output information, it parses and outputs other image information, effectively solving the problem of long user waiting times and improving user retention and work efficiency. Furthermore, in this list generation method, users only need to obtain the image through an electronic device, making it simple to operate and providing a superior user experience.

[0045] Optionally, the electronic device also includes a parser to process each character obtained as a representation of the reasoning result. Of course, if the list generation method of this application requires execution by a specific software program, the parser can be included within that software program or can communicate with it via a communication interface.

[0046] The parser includes multiple judgment conditions, and the item output conditions belong to these multiple judgment conditions, which are related to the characters included in the format example. The parser of the item-level output mechanism in this application differs from the traditional method of unified parsing after the complete text is generated; the parsing process in this application does not depend on the completion of the complete structure (JSON). In this application embodiment, a streaming parsing mechanism based on parsing state maintenance is adopted for the continuously output inference results of the target model. When a complete inference result is not obtained, the parsing mechanism maintains the current parsing state of the parser and determines in real time whether the currently received output constitutes a logically complete item unit, thereby triggering the output of the corresponding item before the structure is fully closed.

[0047] For example, Figure 2 A flowchart of a method for processing each obtained character by a parser is shown, wherein the steps specifically include S201-S208.

[0048] S201: For each character obtained, determine whether the character belongs to the string.

[0049] S202, if it is a string, store the character in the buffer and get the next character.

[0050] S203, if it does not belong to a string, determine whether it indicates the end of the string.

[0051] S204: If the character indicates the end of the string, output the string in the output buffer and obtain the next character.

[0052] S205, if the string does not indicate the end, determine whether it belongs to a structure character.

[0053] S206, if it belongs to a structure character, perform the processing corresponding to the structure character and obtain the next character.

[0054] S207, if it is not a structure character, perform the corresponding processing according to the initialization content and obtain the next character.

[0055] S208, until all characters are obtained.

[0056] For example, before processing each character obtained by the parser, the key-value pairs to be parsed, the structure characters and their corresponding processing, and other characters and their corresponding processing can be initialized. For example, current_string is defined as a temporary string to be parsed, current_key / current_value is defined as a temporary key / value pair to be parsed, number_buffer is defined as a temporary numeric string, target_keys={"Menu Language", "Currency"} is defined as the key names to be extracted during the parsing process, and so on.

[0057] After initialization, each obtained character is processed based on the initialized data. In this embodiment, the characters are used to generate inference results for the target model. Specifically, for each obtained character, it is determined whether the character belongs to a string. If it is determined to belong to a string, the character is stored in a buffer, and the next character is obtained, until the obtained character does not belong to a string. The buffer is used to store characters belonging to the same string, so as to achieve the purpose of processing each character individually, that is, to achieve the technical effect of streaming parsing.

[0058] If it is determined that the character does not belong to the string, then it is determined whether it indicates the end of the string, that is, whether the character is a symbol that indicates the end of the string, such as "double quotes". If it is determined that the character indicates the end of the string, then the string in the character buffer meets the entry output condition, and the string in the buffer is output.

[0059] Once it is determined that the character represents the end of the string and the string in the output buffer is output, the buffer is cleared and the next character is obtained.

[0060] If it is determined that the character does not indicate the end of the string, it is then determined whether it belongs to a structure character; structure characters include, for example, "{", "}", "[", and "]". If the character belongs to a structure character, the corresponding processing is performed, such as determining the current nesting level, so that the output conforms to the structure specified in the initialization. After performing the processing corresponding to the structure character, the next character is obtained.

[0061] If the character is not a structure character, the corresponding processing is performed according to the initialization content. It should be noted that, in addition to double quotes and structure characters mentioned above, initialization is also performed for some specific characters that may appear. For example, for ",", the corresponding processing is to check if number_buffer is empty; if not empty, the numeric value is processed or current_key / current_value is reset, etc.; for ":", since it only serves as a key-value separator, its corresponding processing is to ignore it; for "." or a number, the corresponding processing is to append the character to the buffer corresponding to number_buffer, etc. Of course, specific characters are initialized according to the needs of different application scenarios.

[0062] While performing the corresponding processing based on the initialization content, the next character is obtained, and so on until all characters are obtained, thus completing the parsing and reasoning of the image.

[0063] Taking the application scenario of a user ordering food at a restaurant abroad as an example, the image is a photograph of the menu page, and the menu is in English. Correspondingly, the prompts can be set to include at least the following: '''Extract the dish names and prices from the input menu image and return the content in JSON format. The returned result begins with {. Do notoutput any other natural language except JSON. If the image does not haverelevant information, do not fabricate it. Output "null" in the relevantposition. **Output Example** { "Menu Language": "Chinese" "Currency": "RMB", "dishesList": [ {"Dish Name": "Dish Name 1", "Dish Price": Price 1, "DishDescription": "Description 1"}, {"Dish Name": "Dish Name 2", "Dish Price": Price 2, "DishDescription": "null"}, {"Dish Name": "Dish Name 3", "Dish Price": "null", "DishDescription": "Description 2"} ] } The initialization content may include at least one of the following: SET stack = empty list / / Used to track the current nesting level: 'object' or 'array'; SET current_key = null / / The key name currently being resolved; SET current_value = null / / The key-value pair currently being parsed; SET current_string = "" / / Buffer: Accumulates the current string content; SET in_string = false / / Flag: Whether it is in a string context; SET in_dishes_list = false / / Flag: Whether it is in the dishesList array; SET current_dish = null / / The dish object (dictionary) currently being constructed; SET number_buffer = "" / / Buffer: Accumulates numeric characters (can be integer / floating-point); SET escape_next = false / / Flag: The next character must be escaped (for handling \n, \" etc.); SET target_keys = { "Menu Language", "Currency"} / / Predefine the key fields to be extracted; SET dish_fields = { "Dish Name", "Dish Price", "Dish Description"} / / Dish sub-object fields.

[0064] Furthermore, referring to Figure 3 The flowchart illustrates another method where a parser processes each received character. (See also...) Figure 3 After each character (char) is read, it checks if the current character (char) is already in the string. If a leading "" has been read (i.e., `in_string=True`), it indicates that the current character (char) is in the string, and further determines whether escaping is needed. If `escape_next=True`, it means that the previous character of the current character (char) is an escape character. In this case, the current character (char) is directly added to `current_string`, and `escape_next` is reset to `False`, indicating that the escaping state is over. After that, the subsequent processing of the current character (char) is skipped, and the next character is read directly.

[0065] If `escape_next = False`, it further determines whether the current character `char` needs to be escaped. If the current character `char` is an escape character `\`, then `escape_next = True`, indicating that the next character "needs to be escaped". If the current character `char` is not an escape character, it is directly added to `current_string`, and then subsequent processing of the current character `char` is skipped, and the next character is read directly. During this process, only characters within the string are processed; the escape character ` / ` is not triggered.

[0066] If the current character (char) is not in the string, further determine if the current character is a double quote (""). If it is, it indicates the end of the string. At this point, reverse the state of `in_string`, that is, update `in_string=True` to `in_string=False`. Then, assign the content of `current_string` (the cached string) to `current_key` / `current_value`. Specifically, check if a key already exists to determine whether to assign the string to the "key name" or the "key value". If `current_key` is empty, it means there is no key yet, and the string in `current_string` is assigned to `current_key` as the new key name. If `current_key` is not empty, it means there is already a key name, and the string in `current_string` is assigned to `current_value` as the key value. Then, call the `Process Completed Value()` function to process the paired `current_key` and `current_value`. After confirming that both `current_key` and `current_value` are valid, they can be output, and `current_string` is cleared to read the next character.

[0067] After determining whether the current character is a double quote ("), if the current character is not a double quote ("), meaning it does not indicate the end of the string, it is further determined whether it belongs to a structure character. (See reference...) Figure 3First, determine if the current character `char` is the starting marker "{" for parsing a JSON object. If so, push an "object" marker onto the stack to record that the current object level has been entered (to solve nesting problems, such as a dish object containing child objects). Then, process according to the following condition: if the current state is in the dish list (in_dishes_list=True) and there is no dish object yet (current_dish=null), create an empty dictionary (current_dish) to store the key-value pairs (such as name and price) of this dish. After that, read the next character.

[0068] If the current character char is the closing marker "}" of the JSON object being parsed, then the following conditions are met: if the stack is not empty and the top of the stack is "object" (meaning the current object is within the standard), then the top marker is popped (exiting the object level); furthermore, if current_dish is not empty (indicating the end of the dish object), then the complete dish object is output and current_dish is cleared to prepare for parsing the next dish; then, the next character is read.

[0069] If the current character char is the start marker "[" for parsing a JSON array, then push the "array" marker onto the stack to record that the current array level has been entered. Further, process according to the following conditions: if the current key is dishesList (indicating this is a list of dishes), then set in_dishes_list=True to enable dish parsing mode; if the current key is not dishesList, then clear current_key to prevent interference with the key name; then, read the next character.

[0070] If the current character char is the start marker "]" for parsing a JSON array, then pop the "array" marker from the top of the stack, i.e., exit the array level; if it was previously in dish list mode (i.e., in_dishes_list=True), then set it to in_dishes_list=False, i.e., close the dish parsing mode; then, read the next character.

[0071] Furthermore, if the current character char is not one of the aforementioned "" or structure characters, it is determined whether it is a ",", ":", ".", or a "number". Specifically, if the current character char is a ",", it is further determined whether number_buffer is empty, i.e., whether data exists in the number buffer. If it is not empty, the buffer content is assigned to current_value, ProcessCompletedValue() is called to process and output a valid number, and then the number buffer is cleared. Afterward, current_key and current_value are cleared to prepare for storing the next key-value pair. Finally, the next character is read.

[0072] If the current character char is ":", it indicates that the current character char is the separator between the key name and the key value, and no processing is required; it can be ignored directly. Then, read the next character.

[0073] If the current character char is "." or "number (such as 0-9)", then the current character char is added to the buffer of number_buffer (for example, when parsing 38.5, it is stored in 3→8→.→5 in sequence); then, the next character is read.

[0074] Continue until all characters have been read.

[0075] The entry output conditions described in this application embodiment are used to characterize whether the currently parsed continuous output content constitutes an independently usable structured information unit. In one embodiment, the entry output conditions include: the key fields corresponding to the entry have been parsed (key symbols and keywords in the above example); the current parsing state indicates that the entry can be fully expressed without relying on subsequent output content.

[0076] The target model in this application generates a continuous inference result as an input character stream for each input image. Based on the grammatical structure of the initial content recognition entries (such as objects, arrays, key-value pairs), it immediately outputs when the target field (such as a field that conforms to the format example) is detected. Output can be performed without completing all parsing. This streaming parsing method effectively improves parsing efficiency.

[0077] For example, Figure 4 The flowchart shows the complete process from obtaining the menu image to displaying it to the user, with specific steps including S401-S405.

[0078] S401, obtain the menu image and menu prompts.

[0079] S402, obtain the menu inference results of the target model based on the menu prompts and the menu images.

[0080] S403, whenever the menu reasoning result meets the item output condition, generate the output content of one item in the list based on the reasoning result; the list is used to output the recognition result of the image.

[0081] S404, based on a local database and / or network interface, obtain mapping relationships; wherein, the mapping relationships include at least the correspondence between different languages ​​and the correspondence between different currencies.

[0082] S405, based on the mapping relationship, transforms the output content to obtain the transformed menu image, and then outputs and displays it.

[0083] in, Figure 4 In the example, the paper menu is in French, and the user expects it to be translated into English.

[0084] For example, a user can take a picture of a paper menu using specific software on their mobile phone to obtain a corresponding menu image. The processor of the software or electronic device can then automatically generate prompts based on the menu image or display candidate prompts for the user to choose from.

[0085] Next, the target model identifies and parses the menu image based on menu prompts and outputs the menu inference result. Whenever the menu inference result meets the item output condition, it generates the output content of one item in the list based on the inference result; the list is used to output the image recognition result. Here, each item is a triple format consisting of image-description-price.

[0086] After obtaining each entry, a mapping relationship is generated. This mapping relationship includes at least the correspondence between different languages ​​and between different currencies. For example, the correspondence between French and English, or between the Euro and the US Dollar.

[0087] Optionally, the phone's local database can pre-store mapping relationships, so the mapping relationships can be obtained by directly calling the local database; of course, the network interface can also be called to obtain the mapping relationships over the network. This application embodiment does not limit this.

[0088] Preferably, in scenarios with good network conditions, the network interface is called first to obtain the mapping relationship through the network, ensuring the accuracy and real-time nature of the mapping relationship (such as when the mapping relationship between currencies is updated). This also avoids the waste of resources caused by obtaining the mapping relationship through the network interface instead of the local database, which cannot obtain the complete mapping relationship.

[0089] After obtaining the mapping relationship, the output content of the entry is converted, such as converting the French names and descriptions of dishes to English, and converting the prices of dishes in Euros to US dollars. Then, the converted data is arranged with the corresponding images of the dishes to obtain a converted menu image, which is then displayed on the phone's screen. Of course, this converted menu image only includes one dish and its related information converted to English.

[0090] The system can also generate corresponding review information based on the dish name, description, and price. This review information can be pre-generated using a large language model and can be stored in the electronic device's database or used as prompts input into the target model. For example, qwen3-embedding-8b can be used to embed the description information into the target model for subsequent matching with the dish. Below are some examples of review information:

[0091] Of course, while the user is viewing the dishes and related information that have already been converted, other dishes and related information in the menu image are still undergoing the above process to be converted, until all dishes and related information in the entire menu image have been converted and displayed. Compared to converting all dishes and related information in the entire menu image and then outputting them simultaneously, this embodiment of the application outputs each dish and related information one by one in a streaming manner, which greatly reduces the user's waiting time and provides a better user experience.

[0092] For example, Figure 5 The diagram illustrates the traditional approach of displaying the converted menu image, which simultaneously displays the complete converted menu image, a process that requires the user to wait approximately one minute. Figure 6 The diagram illustrates the output display of the converted menu image provided in this application. It shows a portion of the dishes and related information from the converted menu image one by one until the complete converted menu image is displayed. This process requires the user to wait approximately 4 seconds to view the first dish and related information, with subsequent display intervals of approximately 1.8 seconds. Therefore, the embodiments of this application can significantly reduce the user's waiting time.

[0093] On the other hand, this application also provides an electronic device. Since the principle of solving the problem in the electronic device is similar to the list generation method described above, the implementation of the electronic device can refer to the implementation of the list generation method described above, and the repeated parts will not be described again.

[0094] For example, Figure 7 A schematic diagram of the electronic device is shown, for reference. Figure 7The electronic device includes a processor 701 and a display 702 that are interconnected. The processor acquires an image; the image is input to a target model for inference; acquires prompt words, the prompt words are input to the target model to guide the target model inference about the image; acquires the inference result output by the target model based on the prompt words for the image; whenever the inference result satisfies the item output condition, it generates the output content of an item in a list based on the inference result; the list is output to represent the recognition result of the image; and the output content is transmitted to a display. The display shows the output content.

[0095] In another aspect, embodiments of this application also provide a computer program product, which is a computer-readable medium storing a computer program. When executed by a processor, the computer program implements the method provided in any embodiment of the present invention, including the following steps S11 to S14: S11, Obtain the image; the image is used as input to the target model for inference; S12, obtain prompt words, the prompt words are used to input the target model to guide the target model to reason about the image; S13, obtain the reasoning result of the target model based on the prompt words for the image; S14, whenever the reasoning result satisfies the entry output condition, an entry of the list is generated based on the reasoning result; the list is used to output the recognition result of the image.

[0096] In another aspect, embodiments of this application also provide a computer device, the structural schematic diagram of which can be as follows: Figure 8 As shown, the system includes at least a memory 801 and a processor 802. The memory 801 stores a computer program, and the processor 802 implements the method provided in any embodiment of the present invention when executing the computer program in the memory 801. Exemplarily, the computer program steps of the computer device are as follows: S21 to S24: S21, Obtain the image; the image is used as input to the target model for inference; S22, Obtain prompt words, which are used to input the target model to guide the target model to reason about the image; S23, obtain the reasoning result of the target model based on the prompt words for the image; S24, whenever the reasoning result satisfies the entry output condition, an entry of the list is generated based on the reasoning result; the list is used to output the recognition result of the image.

[0097] In another aspect, embodiments of this application also provide a storage medium carrying one or more computer programs, which, when executed by a processor, implement the method provided in any embodiment of the present invention, including the following steps S31 to S34: S31, Obtain the image; the image is used as input to the target model for inference; S32, obtain prompt words, the prompt words are used to input the target model to guide the target model to reason about the image; S33, obtain the reasoning result of the target model based on the prompt word for the image; S34, whenever the reasoning result satisfies the entry output condition, an entry of the list is generated based on the reasoning result; the list is used to output the recognition result of the image.

[0098] Through the above-mentioned streaming parsing and streaming output mechanism, this application can gradually generate and output structured results while the target model continues to infer and output, thereby avoiding system idle waiting due to waiting for complete inference results and improving the parallelism and resource utilization efficiency of the overall processing flow.

[0099] Optionally, in this embodiment, the aforementioned computer program product and storage medium may include, but are not limited to, various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk. Optionally, in this embodiment, the processor executes the method steps described in the above embodiments according to the program code stored in the computer program product. Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, which will not be repeated here. Obviously, those skilled in the art should understand that the various modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed on a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be executed in a different order than those described here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any specific hardware and software combination.

[0100] Furthermore, although exemplary embodiments have been described herein, their scope includes any and all embodiments based on this application that have equivalent elements, modifications, omissions, combinations (e.g., schemes involving intersections of various embodiments), adaptations, or alterations. Elements in the claims will be interpreted broadly based on the language used in the claims and are not limited to the examples described in this specification or during the implementation of this application, which will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered illustrative only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0101] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. Other embodiments may be used by those skilled in the art upon reading the above description. Furthermore, in the above detailed description, various features may be grouped together to simplify the application. This should not be construed as an intention that a disclosed feature not claimed is necessary for any claim. Rather, the subject matter of this application may be less than all the features of a particular disclosed embodiment. Thus, the following claims are incorporated herein by reference as examples or embodiments, wherein each claim is an independent, separate embodiment, and these embodiments are contemplated as being possible in various combinations or arrangements. The scope of this application should be determined by reference to the appended claims and the full scope of their equivalents.

[0102] The foregoing has described in detail several embodiments of this application, but this application is not limited to these specific embodiments. Those skilled in the art can make various variations and modifications based on the concept of this application, and all such variations and modifications should fall within the scope of protection claimed in this application.

Claims

1. A list generation method, comprising: Get the image; The image is used as input to the target model for inference; Obtain prompt words, which are used to input the target model to guide the target model to reason about the image; Obtain the inference result of the target model based on the prompt words for the image; Whenever the reasoning result satisfies the entry output condition, the output content of one entry in the list is generated based on the reasoning result; The list is used to output the recognition results of the image.

2. The list generation method according to claim 1, The reasoning result is a series of consecutively output characters, and the number of entries in the list increases sequentially as the consecutively output characters are processed.

3. The list generation method according to claim 2, wherein the step of generating an entry for the list based on the reasoning result whenever the reasoning result satisfies the entry output condition includes: If the reasoning result satisfies the entry output condition each time, the output content of one entry in the list is generated based on the reasoning result, until the target model completes the character output sequentially for the image reasoning process based on the prompt words; Each time the output condition of an entry is met, an entry is added to the list sequentially based on the output content of the generated entry.

4. The list generation method according to claim 3, The prompt word includes a format sample, and the target model outputs multiple characters consecutively in the format sample based on the inference result of the prompt word for the image.

5. The list generation method according to claim 2, wherein the step of generating an entry of the list based on the reasoning result whenever the reasoning result satisfies the entry output condition further includes: For each character obtained that represents the reasoning result, determine whether the entry output condition is met.

6. The list generation method according to claim 5, wherein each time a character representing the reasoning result is obtained, determining whether the item output condition is satisfied includes: Each time a character representing the reasoning result is obtained, the obtained character is processed by a parser. The parser includes multiple judgment conditions, and the entry output condition belongs to the multiple judgment conditions. The multiple judgment conditions are related to the characters included in the format example.

7. The list generation method according to claim 6, wherein the step of processing each obtained character by the parser includes: For each character obtained, determine whether that character belongs to the string; If the character is a string, store it in the buffer and obtain the next character; If it does not belong to a string, determine whether it indicates the end of the string; If the character represents the end of the string, output the string in the output buffer and obtain the next character; If the end of the string is not indicated, determine whether it belongs to a structure character; If it belongs to a structure character, perform the processing corresponding to the structure character and obtain the next character; If it is not a structure character, perform the corresponding processing according to the initialization content and obtain the next character; Continue until all characters are obtained.

8. The list generation method according to claim 7, further comprising, before processing each obtained character by the parser: Initialize the key-value pairs to be parsed, the structure characters and their corresponding processing, and other characters and their corresponding processing.

9. The list generation method according to any one of claims 1-8, comprising: Obtain the menu image and menu hints; Obtain the menu inference result output by the target model based on the menu prompt words for the menu image; Whenever the menu reasoning result satisfies the item output condition, the output content of one item in the list is generated based on the reasoning result; The list is used to output the recognition results of the image; Based on a local database and / or network interface, a mapping relationship is obtained; wherein, the mapping relationship includes at least the correspondence between different languages ​​and the correspondence between different currencies; Based on the mapping relationship, the output content is transformed to obtain the transformed menu image, which is then displayed.

10. An electronic device comprising a processor and a display interconnected with each other; The processor obtains an image; the image is used as input to a target model for reasoning; and obtains prompt words, which are used as input to the target model to guide the target model in reasoning about the image. Obtain the inference result of the target model based on the prompt words for the image; Whenever the reasoning result satisfies the entry output condition, the output content of one entry in the list is generated based on the reasoning result; The list is used to output the recognition results of the image; The output content is transmitted to the display. The display shows the output content.

Images