Information processing method and device and electronic equipment

By obtaining the first subject identification information of candidates in the image, the subject recall and recommendation query are determined, the problem of low accuracy of multiple rounds of image dialogue in the prior art is solved, and an efficient and accurate dialogue question-and-answer process is achieved, and users are enhanced. Experience.

CN119089010APending Publication Date: 2024-12-06BEIJING BAIDU NETCOM SCI & TECH CO LTD

Patent Information

Application Number
CN202411109826.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The prior art is difficult to achieve multiple rounds of dialogue on images, and the returned dialogue results are low, resulting in poor user experience.

Method used

By obtaining the first subject identification information of the candidate in the image, performing the subject recall based on the identification information, determining the second subject and its associated recommendation query, sending a query to the client, and having multiple rounds of conversations with the client to obtain the cloud-based large model Dialogue results to improve the efficiency and accuracy of the dialogue process.

Benefits of technology

It improves the efficiency of user dialogue and the accuracy of dialogue results, improves the user experience, and realizes the efficiency of multiple rounds of interactive Q&A on images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089010A_ABST
    Figure CN119089010A_ABST
Patent Text Reader

Abstract

The invention provides an information processing method and device and electronic equipment, relates to the technical field of artificial intelligence, in particular to the technical fields of deep learning, natural language processing, large models and the like, and can be applied to application scenes such as generative search, intelligent assistants, virtual assistants, intelligent e-commerce and the like. The method comprises the following steps: acquiring identification information of a candidate first main body in an image; performing subject recall based on the identification information of the first subject, determining a recalled second subject in the first subject and a recommendation query associated with the second subject, and sending the recommendation query to the client; and performing multiple rounds of dialogues with the client based on the recommendation query, obtaining a dialogue result of each round of dialogue returned by the cloud big model, and sending the dialogue result to the client. According to the method and the device, multiple rounds of dialogues can be performed with the client based on the recommendation query, and the dialogue result of each round of dialogue returned by the cloud big model is obtained, so that the efficiency in the user dialogue process is improved, the accuracy of the dialogue result is improved, and the user experience is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to technical fields such as deep learning, natural language processing and large models, and in particular to an information processing method, device, electronic device, storage medium and computer program product. Background Art

[0002] At present, with the continuous development of artificial intelligence technology, large models have advantages such as good generalization and have been widely used in fields such as intelligent interaction. Users have gradually begun to interact with large models through voice, text, and images. However, in related technologies, it is often impossible to conduct multiple rounds of dialogues on images, and the accuracy of the returned dialogue results is low, resulting in a poor user experience. Summary of the invention

[0003] The present disclosure proposes an information processing method, an apparatus, an electronic device, a storage medium and a computer program product.

[0004] According to a first aspect of the present disclosure, an information processing method is proposed, comprising: obtaining identification information of a candidate first subject in an image; performing subject recall based on the identification information of the first subject, determining a recalled second subject in the first subject and a recommended query associated with the second subject, and sending the recommended query to a client; conducting multiple rounds of conversations with the client based on the recommended query, obtaining conversation results of each round of conversations returned by a large cloud model, and sending the conversation results to the client.

[0005] According to a second aspect of the present disclosure, an information processing method is proposed, including: receiving an image, identifying the image, and obtaining identification information of a candidate first subject in the image; sending the identification information of the first subject to a front end, and receiving a recommendation query fed back by the front end, wherein the recommendation query is a recommendation query associated with a second subject recalled from the first subject based on the identification information; conducting multiple rounds of dialogue with the front end based on the recommendation query; and receiving dialogue results of each round of dialogue output by a large cloud-based model fed back by the front end.

[0006] According to a third aspect of the present disclosure, an information processing device is proposed, comprising: a first acquisition module, used to acquire identification information of a candidate first subject in an image; a determination module, used to perform subject recall based on the identification information of the first subject, determine a recalled second subject in the first subject and a recommended query associated with the second subject, and send the recommended query to a client; a second acquisition module, used to conduct multiple rounds of dialogue with the client based on the recommended query, acquire dialogue results of each round of dialogue returned by a large cloud model, and send the dialogue results to the client.

[0007] According to a fourth aspect of the present disclosure, an information processing device is proposed, comprising a recognition module, configured to receive an image, recognize the image, and obtain recognition information of a candidate first subject in the image; a receiving module, configured to send the recognition information of the first subject to a front end, and receive a recommendation query fed back by the front end, wherein the recommendation query is a recommendation query associated with a second subject recalled from the first subject based on the recognition information; a dialogue module, configured to conduct multiple rounds of dialogues with the front end based on the recommendation query; and a receiving module, configured to receive dialogue results of each round of dialogue output by a large cloud-based model fed back by the front end.

[0008] According to the fifth aspect of the present disclosure, an electronic device is proposed, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the information processing method proposed in the first aspect above, or execute the information processing method proposed in the second aspect above.

[0009] According to the sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is proposed, wherein the computer instructions are used to enable the computer to execute the information processing method proposed in the first aspect above, or to execute the information processing method proposed in the second aspect above.

[0010] According to a seventh aspect of the present disclosure, a computer program product is proposed, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the information processing method proposed in the first aspect, or implements the information processing method proposed in the second aspect.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0013] Figure 1 A flowchart of an information processing method according to an embodiment of the present disclosure;

[0014] Figure 2 A schematic diagram of a subject mapping rule according to an embodiment of the present disclosure;

[0015] Figure 3 A flowchart of an information processing method according to another embodiment of the present disclosure;

[0016] Figure 4 A flowchart of an information processing method according to another embodiment of the present disclosure;

[0017] Figure 5 A flowchart of an information processing method according to an embodiment of the present disclosure;

[0018] Figure 6 A schematic diagram of a display card according to an embodiment of the present disclosure;

[0019] Figure 7 is a schematic diagram of a display card according to another embodiment of the present disclosure;

[0020] Figure 8 A flowchart of an information processing method according to an embodiment of the present disclosure;

[0021] Fig. 9 It is a structural schematic diagram of an information processing device according to an embodiment of the present disclosure;

[0022] Fig.10 It is a structural schematic diagram of an information processing device according to an embodiment of the present disclosure;

[0023] Fig.11 A schematic block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0025] Artificial Intelligence (AI) is a technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. At present, AI technology has the advantages of high automation, high precision and low cost, and has been widely used.

[0026] Deep Learning (DL) is a new research direction in the field of Machine Learning (ML). It is a science that studies the inherent laws and representation levels of sample data, enabling machines to have analytical learning capabilities like humans and to recognize data such as text, images, and sounds. It is widely used in speech and image recognition.

[0027] Natural Language Processing (NLP) is an important field in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between humans and computers using natural language. Natural Language Processing is a science that integrates linguistics, computer science, and mathematics.

[0028] A large model refers to a machine learning model with a large parameter scale and complexity. It requires a large amount of computing resources and storage space to train and store, and often requires distributed computing and special hardware acceleration technology. Large models have stronger generalization and expression capabilities.

[0029] Figure 1 FIG. 1 is a flow chart of an information processing method according to an embodiment of the present disclosure. Figure 1 As shown, the method includes:

[0030] S101, obtaining identification information of a first subject candidate in an image.

[0031] It should be noted that the execution subject of the information processing method of the embodiment of the present disclosure may be a hardware device with data information processing capabilities and / or the necessary software required to drive the hardware device to work. Optionally, the execution subject may include a workstation, a server, a computer, a user terminal and other intelligent devices. Among them, the user terminal includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, an intelligent home appliance, a vehicle-mounted terminal, etc.

[0032] The image may be any image to be processed.

[0033] It should be noted that the present disclosure does not limit the method for acquiring images, which can be selected according to actual conditions.

[0034] Optionally, the image can be acquired by performing real-time acquisition through an image acquisition device in the client according to an image acquisition instruction of the user.

[0035] Optionally, an image may be selected from an image storage repository in the client through an image selection instruction of a user to obtain the image.

[0036] It should be noted that after the image is acquired, the image may be recognized based on an image recognition model to obtain recognition information of the candidate first subject in the image.

[0037] The identification information of the first subject may include classification information and location information of the first subject.

[0038] It should be noted that the present disclosure may pre-set a classification table, for example, the classification table may include multiple classification information such as "text", "daily necessities", "clothing accessories", etc.

[0039] In the embodiment of the present disclosure, after the client obtains the identification information of the candidate first subject in the image, the identification information of the first subject can be sent to the front end. Correspondingly, the front end can receive the identification information of the candidate first subject in the image sent by the client to obtain the identification information (coarse classification information) of the candidate first subject in the image.

[0040] S102: recall a subject based on the identification information of the first subject, determine a second subject recalled from the first subject and a recommended query associated with the second subject, and send the recommended query to the client.

[0041] It should be noted that after the identification information of the first subject is obtained, subject recall can be performed based on the identification information of the first subject.

[0042] In the disclosed embodiment, the classification information and location information of the first subject may be determined from the identification information of the first subject, and the second subject and the recommended query associated with the second subject may be determined from the first subject based on the classification information and location information of the first subject.

[0043] Optionally, the area of ​​the first subject may be determined according to the location information, the areas of the first subject may be sorted in descending order, the second subject may be determined from the first subject according to the sorting result, and subject rule mapping may be performed according to the classification information of the second subject to obtain a recommended query associated with the second subject.

[0044] For example, you can pre-build subject mapping rules, such as Figure 2 As shown, when the classification information of the subject is "text", the corresponding subject mapping rule is set to text extraction, text translation, image translation, image + text translation; when the classification information of the subject is "people / animals / plants", etc., the corresponding subject mapping rule is set to guess word recognition; when there is no classification information of the subject, the corresponding subject mapping rule is set to visual question answering (Visual question answering, abbreviated as VQA).

[0045] For example, if the classification information of the second subject is "plant", by performing subject mapping rule mapping, that is, guessing word recognition, the recommended query associated with the second subject is obtained.

[0046] In the embodiment of the present disclosure, after the recommended query associated with the second subject is obtained, the recommended query is sent to the client. Accordingly, the client receives the recommended query fed back by the front end and can display the recommended query on the user interaction page of the client.

[0047] S103, conduct multiple rounds of dialogue with the client based on the recommended query, obtain the dialogue results of each round of dialogue returned by the cloud-based large model, and send the dialogue results to the client.

[0048] In an embodiment of the present disclosure, for the i-th round of conversation, a user query sent by the client can be received, wherein the user query is related to the recommendation query, conversation information of the i-th round is determined based on the user query, and the conversation information of the i-th round is sent to the cloud-based big model. The cloud-based big model can conduct a conversation based on the i-th round of conversation information, receive a conversation result returned by the cloud-based big model that matches the i-th round of conversation information, and send the conversation result to the client.

[0049] The information processing method proposed in the present disclosure obtains the identification information of the candidate first subject in the image, performs subject recall based on the identification information of the first subject, determines the recalled second subject in the first subject and the recommended query associated with the second subject, sends the recommended query to the client, conducts multiple rounds of dialogues with the client based on the recommended query, obtains the dialogue results of each round of dialogue returned by the cloud-based large model, and sends the dialogue results to the client. The present disclosure can conduct multiple rounds of dialogues with the client based on the recommended query and obtains the dialogue results of each round of dialogue returned by the cloud-based large model, thereby improving the efficiency of the user dialogue process, improving the accuracy of the dialogue results, and enhancing the user experience.

[0050] Figure 3 It is a flowchart of an information processing method according to the second embodiment of the present disclosure.

[0051] like Figure 3 As shown, in Figure 1 Based on the illustrated embodiment, the pattern generating method of the embodiment of the present disclosure may specifically include the following steps:

[0052] S301, obtaining identification information of a first candidate subject in an image.

[0053] For the relevant contents of step S301, please refer to the above embodiment, which will not be repeated here.

[0054] In the above embodiment, step S102 of “recalling a subject based on the identification information of the first subject, and determining a recalled second subject in the first subject and a recommended query associated with the second subject” may specifically include the following steps S302 and S303 .

[0055] S302: Determine classification information and location information of the first subject from identification information of the first subject.

[0056] In the embodiment of the present disclosure, after obtaining the identification information of the first subject, the classification information (coarse classification information) of the first subject, such as "green plants", "text", etc., can be determined from the identification information of the first subject, and the location information (coordinate information) of the first subject can be determined.

[0057] S303, according to the classification information and location information of the first subject, determine the second subject and the recommended query associated with the second subject from the first subject, perform subject rule mapping according to the classification information of the second subject, and obtain the recommended query associated with the second subject.

[0058] In the embodiment of the present disclosure, the area of ​​the first body may be determined according to the position information, and the second body may be determined from the first body based on the area of ​​the first body.

[0059] Optionally, the coordinate point set of the first body can be determined according to the position information of the first body, and the boundary of the first body can be determined based on the coordinate point set of the first body, and the area of ​​the figure enclosed by the boundary of the first body can be calculated, that is, the area of ​​the first body.

[0060] In the embodiment of the present disclosure, after the area of ​​the first body is acquired, the second body may be determined from the first body based on the area of ​​the first body.

[0061] It should be noted that the present disclosure does not limit the specific method of determining the second body from the first body based on the area of ​​the first body, and it can be selected according to actual conditions.

[0062] Optionally, the areas of the first bodies may be sorted in descending order, and the first bodies ranked in the top N positions are determined as the second bodies, where N is a positive integer.

[0063] It should be noted that the present disclosure does not limit the setting of N. For example, N may be 3, or for another example, N may be 4.

[0064] For example, when N is 3, if the area of ​​the first body A is S1, the area of ​​the first body B is S2, the area of ​​the first body C is S3, and the area of ​​the first body D is S4, and S1>S3>S4>S2, then the second bodies are body A, body C, and body D.

[0065] Optionally, an area threshold S corresponding to the first body may be preset. a , the area is greater than or equal to the area threshold S a The first subject is determined as the second subject, wherein the present disclosure has an area threshold S a There is no limitation on the settings, and they can be set according to the actual situation.

[0066] For example, for the area threshold Sa , if the area of ​​the first body A is S1, the area of ​​the first body B is S2, the area of ​​the first body C is S3, the area of ​​the first body D is S4, and S1>S a , S2≥S a 、S3<S a 、S4≥S a , then the second subject is subject A, subject B and subject D.

[0067] In the disclosed embodiment, after determining the second subject, the candidate category corresponding to the second subject and the recognition probability (similarity) of the candidate category can be obtained from the classification information of the second subject, the subject mapping rule corresponding to the candidate category can be determined, and the candidate query corresponding to the candidate category can be generated according to the subject mapping rule, the candidate queries can be sorted based on the recognition probability of the candidate category, and according to the sorting result, the recommended query associated with the second subject can be determined from the candidate queries.

[0068] It should be noted that the present disclosure does not limit the specific method of determining the recommended query associated with the second subject from the candidate queries according to the ranking results, and the method can be selected according to actual conditions.

[0069] Optionally, the candidate queries may be sorted in descending order based on the recognition probabilities of the candidate categories, and the candidate query ranked first in the sorting result may be determined as the recommended query.

[0070] For example, if the second subject is subject A, the recognition probability of candidate category 1 of subject A is P1, the recognition probability of candidate category 2 is P2, the recognition probability of candidate category 3 is P3, and P1>P3>P2, then the candidate query corresponding to candidate category 1 is used as the recommended query associated with subject A.

[0071] For example, if the second subject is subject B, the recognition probability of candidate category 1 for subject B is P4, the recognition probability of candidate category 2 is P5, the recognition probability of candidate category 3 is P6, and P6>P4>P5, then the candidate query corresponding to candidate category 3 is used as the recommended query associated with subject B.

[0072] S304: Send a recommendation query to the client.

[0073] In the embodiment of the present disclosure, after the front end obtains the recommended query, the recommended query can be sent to the client.

[0074] In the above embodiment, step S103 "conducting multiple rounds of dialogues with the client based on the recommendation query, obtaining the dialogue results of each round of dialogues returned by the cloud-based large model, and sending the dialogue results to the client" may specifically include the following steps S205 and S207.

[0075] S305 : For the i-th round of dialogue, receiving a user query sent by the client, wherein the user query is related to a recommendation query.

[0076] It should be noted that the user query is the search information entered by the user on the client, for example: the search information entered by the user on the mobile phone, the search information entered by the user on the computer, etc.

[0077] In the embodiment of the present disclosure, multiple rounds of user queries belonging to the same target subject sent by the client may be received, wherein the user queries sent each time are different and are related to the recommendation queries of the same target subject.

[0078] S306: Determine the conversation information of the i-th round based on the user query, and send the conversation information of the i-th round to the cloud big model.

[0079] As a possible implementation, Figure 4 As shown, based on the above embodiment, the specific process of determining the conversation information of the i-th round based on the user query includes the following steps:

[0080] S401, performing intent recognition on a user query to determine a target subject and a target category, wherein the target subject is at least one of the second subjects.

[0081] It should be noted that after obtaining the user query, the user query can be identified based on the natural language processing (NLP) technology and the optical character recognition (OCR) technology to determine the target subject and target category.

[0082] S402, generating the i-th round of dialogue information according to the target subject, target category and user query.

[0083] In the disclosed embodiment, after the target subject, target category and user query are acquired, the dialogue information of the i-th round may be generated according to the target subject, target category and user query.

[0084] Optionally, if it is identified that the target subject of the i-th round of dialogue is the same as the target subject of the historical dialogue, the historical dialogue results of the historical dialogue are determined, the target subject, target type, user query and historical dialogue results are combined to generate dialogue information of the i-th round of dialogue.

[0085] It should be noted that if it is identified that the target subject of the i-th round of dialogue is the same as the target subject of the historical dialogue, the dialogue information of the i-th round of dialogue can be enriched by combining the historical dialogue results to improve the accuracy of generating the dialogue information of the i-th round of dialogue.

[0086] Optionally, if it is identified that the target subject of the i-th round of dialogue is different from the target subject of the historical dialogue, the target subject, the target type and the user query are combined to generate dialogue information of the i-th round of dialogue.

[0087] It should be noted that if it is identified that the target subject of the i-th round of dialogue is different from the target subject of the historical dialogue, the target subject, target type and user query can be combined to generate dialogue information for the i-th round of dialogue. This solves the problem in the prior art that when conducting dialogues with different subjects in an image, it is necessary to conduct dialogues based on multiple images, resulting in a very cumbersome dialogue process and low efficiency in obtaining dialogue results.

[0088] S307, receiving the conversation result matched with the conversation information of the i-th round returned by the cloud-based large model, and sending the conversation result to the client.

[0089] The step S103 "sending the conversation result to the client" in the above embodiment may specifically include the following step S208.

[0090] S308, rendering the conversation result, generating a display card, and sending the display card to the client.

[0091] In the disclosed embodiment, the target subject of the user's query and the queried target type of the target subject can be determined based on the conversation results, the conversation sub-results of each target type can be extracted from the conversation information, and the card can be rendered based on the target subject and the conversation sub-results of the target type to obtain a display card.

[0092] Optionally, the recognition probability of the target type of the target subject can be determined, the display order of the target type's dialogue sub-results can be determined, sub-cards of the display card can be generated based on the display order and the target type's dialogue sub-results, and the sub-cards can be combined to obtain a display card.

[0093] For example, the recognition probabilities of the target types of the target subjects may be sorted in descending order to obtain a sorting result of the recognition probabilities of the target types of the target subjects, and the order in the sorting result is used as the display order of the dialog sub-results of the target types.

[0094] In the embodiment of the present disclosure, after the display order is obtained, sub-cards of the display card can be generated based on the display order and the dialog sub-results of the target type, and the sub-cards can be combined to obtain the display card.

[0095] In summary, the information processing method proposed in the present disclosure obtains the identification information of the candidate first subject in the image, obtains the identification information of the candidate first subject in the image, determines the second subject from the first subject and the recommended query associated with the second subject according to the classification information and position information of the first subject, performs subject rule mapping according to the classification information of the second subject, obtains the recommended query associated with the second subject, sends the recommended query to the client, receives the user query sent by the client for the i-th round of dialogue, wherein the user query is related to the recommended query, determines the dialogue information of the i-th round based on the user query, and sends the recommended query to the cloud large model. The model sends the conversation information of the i-th round, receives the conversation result matched with the conversation information of the i-th round returned by the cloud-based big model, sends the conversation result to the client, renders the conversation information, generates a display card, and sends the display card to the client. Therefore, the present invention reduces the error rate of the conversation question and answer results in the conversation question and answer process through the collaboration between the front-end and the cloud-based big model of the client, improves the accuracy of the conversation question and answer, and provides multi-modal recognition capabilities, which can meet the needs of users to ask questions about images, and conduct multiple rounds of interactive questions and answers on images, and can answer different subjects in the image through the model, thereby ensuring the efficiency of the conversation question and answer and improving the user experience.

[0096] Figure 5 FIG. 1 is a flow chart of an information processing method according to an embodiment of the present disclosure. Figure 5 As shown, the method includes:

[0097] S501, receiving an image, and identifying the image to obtain identification information of a candidate first subject in the image.

[0098] It should be noted that the execution subject of the information processing method of the embodiment of the present disclosure may be a hardware device with data information processing capabilities and / or the necessary software required to drive the hardware device to work. Optionally, the execution subject may include a workstation, a server, a computer, a user terminal and other intelligent devices. Among them, the user terminal includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, an intelligent home appliance, a vehicle-mounted terminal, etc.

[0099] The image may be any image to be processed.

[0100] For example, the client can obtain camera permissions through the user's permission instructions and select images based on the user's operation instructions.

[0101] In the disclosed embodiment, before recognizing an image and obtaining identification information of a candidate first subject in the image, a target function to be called by the image can be determined from multiple candidate functions displayed by the client, and an image recognition model associated with the target function can be entered.

[0102] It should be noted that the multiple functions displayed by the client include but are not limited to image interpretation assistant, photo search and other functions.

[0103] In the disclosed embodiment, after receiving the image, the image may be preprocessed to meet the requirements of a software development kit (SDK) built into the client, that is, to meet the requirements of an image recognition model associated with a target function.

[0104] For example, with respect to the resolution of the image, the pixels of the shortest side of the image may be set to be greater than 512 pixels, and with respect to the size of the image, the size of the image may be set to be within 600K.

[0105] It should be noted that if the target function to be called by the image is an image interpretation assistant, that is, to achieve understanding of the image, the image after the preprocessing operation can be identified based on the image recognition model associated with the target function to obtain the identification information of the candidate first subject in the image, wherein the identification information of the first subject includes the classification information and position information of the first subject.

[0106] It should be noted that the present disclosure may pre-set multiple categories according to actual conditions, so as to obtain classification information of the candidate first subject in the image based on the image recognition model associated with the target function.

[0107] For example, multiple classification categories may include "code", "text", "body parts", "green plants", "upper body", "lower body", "transportation", "daily necessities", "clothing accessories", "face", "cultural relics and collectibles", "animals", "graphic images", "toys", "digital appliances", "tobacco and alcohol", "whole picture", "food", "building" and "book images", etc.

[0108] It should be noted that the client may pre-set a coordinate reference window of the subject to obtain the position information (coordinate information) of the candidate first subject in the image based on the image recognition model associated with the target function.

[0109] S502, sending identification information of the first subject to the front end, and receiving a recommended query fed back by the front end, where the recommended query is a recommended query associated with the second subject recalled from the first subject based on the identification information.

[0110] In an embodiment of the present disclosure, after obtaining the identification information of the first subject, the identification information of the first subject can be sent to the front end. Accordingly, the front end can receive the identification information of the first subject and perform subject recall based on the identification information of the first subject, determine the recalled second subject in the first subject and the recommended query associated with the second subject, and send the recommended query to the client. Accordingly, the client can receive the recommended query fed back by the front end.

[0111] S503, conduct multiple rounds of dialogue with the front end based on the recommended query.

[0112] In the disclosed embodiment, user queries related to the recommendation query are sent to the front end in multiple rounds, wherein different user queries are used to generate different dialog information.

[0113] Among them, user queries sent in multiple rounds are associated with the same target subject; or, two user queries sent adjacently are associated with different target subjects.

[0114] In the disclosed embodiment, for the i-th round of dialogue, the i-th round of dialogue information is determined based on the user query, and the i-th round of dialogue information is sent to the cloud-based big model, and multiple rounds of dialogue are conducted with the front end based on the recommended query.

[0115] S504, receiving the dialogue results of each round of dialogue output by the cloud-based large model fed back by the front end.

[0116] In the disclosed embodiment, the front end receives the conversation result matched with the conversation information of the i-th round returned by the cloud-based large model, and sends the conversation result to the client.

[0117] Optionally, the front end can render the dialogue results, generate a display card, and send the display card to the client. Correspondingly, the client can receive the display card fed back by the front end and display it on the interactive interface.

[0118] For example, Figure 6 As shown, from the multiple candidate functions displayed by the client, Figure 6 In (a), the target function (image analysis assistant function) to be called by the image is determined, and the image recognition model associated with the target function is entered. By recognizing the image, the recognition information of the candidate first subject in the image is obtained, and the subject is recalled based on the recognition information of the first subject. The recalled second subject (fruit) in the first subject is determined, and the candidate categories corresponding to the second subject, namely litchi, bayberry and strawberry litchi (branch) are obtained. The recognition probability (similarity) of litchi is 95%, the recognition probability (similarity) of bayberry is 71%, and the recognition probability of strawberry litchi (branch) is not shown. Figure 6In (b), candidate queries corresponding to the candidate categories are generated according to the subject mapping rules, and recommended queries are determined from the candidate queries, such as: "What do you want to ask about this picture?", "What is this?", "Give it a circle of friends copy?", "Describe this picture?", etc. Figure 6 In (c), the target subject is Lizhi. The display card is generated by rendering the dialogue results of each round of dialogue returned by the cloud-based large model, and the display card is displayed on the client's interactive interface. Figure 6 In (d), the target subject is Yang Mei. Display cards are generated by rendering the dialogue results of each round of dialogue returned by the cloud-based large model, and the display cards are displayed on the client's interactive interface.

[0119] For example, Figure 7 As shown, from multiple candidate functions displayed by the client, Figure 7 (a) determines the target function (image analysis assistant function) to be called in the image, and enters the image recognition model associated with the target function. By recognizing the image, the recognition information of the candidate first subject in the image is obtained, and the subject is recalled based on the recognition information of the first subject, and the recalled second subject (text) in the first subject is determined. Figure 7 In (b), the candidate category (English text) corresponding to the second subject is obtained, and the candidate query corresponding to the candidate category is generated according to the subject mapping rule, and the recommended query is determined from the candidate query, such as: "What do you want to ask about this picture", "Extract text and translate it into Chinese", "Change the text in the picture to Chinese with one click", "Describe this picture", etc. Figure 7 (c) Through the conversation information "extract text and translate into Chinese", the conversation result of the conversation returned by the cloud-based large model is obtained, and the conversation result of the conversation returned by the cloud-based large model is rendered to generate a display card, and the display card is displayed on the client's interactive interface. Figure 7 (d) The dialogue result of the dialogue returned by the cloud-based large model is obtained through the dialogue information "Change the text in the image to Chinese with one click, and return the translation result", and the dialogue result of the dialogue returned by the cloud-based large model is rendered to generate a display card, and the display card is displayed on the client's interactive interface.

[0120] The information processing method proposed in the present disclosure receives an image, identifies the image, obtains identification information of a candidate first subject in the image, sends the identification information of the first subject to a front end, receives a recommendation query fed back by the front end, the recommendation query is a recommendation query associated with a second subject recalled from the first subject based on the identification information, conducts multiple rounds of dialogues with the front end based on the recommendation query, and receives dialogue results of each round of dialogue output by a cloud-based large model fed back by the front end. Thus, by obtaining the identification information of the candidate first subject in the image and conducting multiple rounds of dialogues with the front end based on the recommendation query, the efficiency of the user dialogue process is improved, the accuracy of the dialogue results is improved, and the user experience is improved.

[0121] The specific process of the information processing method proposed in the present disclosure is explained below.

[0122] For example, Figure 8 As shown, the client (interaction layer) recognizes the image, obtains the identification information of the candidate first subject in the image, and sends the identification information of the first subject to the front end (business layer). The front end performs subject recall based on the identification information of the first subject, that is, determines the area of ​​the first subject according to the location information of the first subject, determines the second subject from the first subject based on the area of ​​the first subject (subject recall), obtains the recommended query associated with the second subject, and sends the recommended query to the client. For the i-th round of dialogue, the user query sent by the client is received, and the dialogue information of the i-th round is determined based on the user query. The front end sends a query to the cloud large model. The front end receives the conversation result matched with the conversation information of the i-th round returned by the cloud big model, renders the conversation result, generates a display card, and sends the display card to the client. In the multi-round question-and-answer process, if it is identified that the target subject of the i-th round of conversation is the same as the target subject of the historical conversation, the target subject, target type, user query and historical conversation result are combined to generate the conversation information of the i-th round of conversation; if it is identified that the target subject of the i-th round of conversation is different from the target subject of the historical conversation, the target subject, target type and user query are combined to generate the conversation information of the i-th round of conversation.

[0123] In summary, the information processing method proposed in the present invention reduces the error rate of the dialogue question and answer results during the dialogue question and answer process through the collaboration of the front-end and the cloud-based large model of the client, improves the accuracy of the dialogue question and answer, generates candidate queries corresponding to the candidate categories according to the subject mapping rules, and sorts the candidate queries based on the recognition probability of the candidate categories, and determines the recommended query associated with the second subject from the candidate queries according to the sorting results, discards the full amount of data, prevents pollution of user interaction, and conducts multiple rounds of interactive question and answer for the image, which can answer different subjects in the image through the model, ensuring the efficiency of the dialogue question and answer and improving the user experience.

[0124] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0125] According to an embodiment of the present disclosure, the present disclosure also provides an information processing device for implementing the above-mentioned information processing method.

[0126] Fig. 9 is a block diagram of an information processing device according to an embodiment of the present disclosure.

[0127] like Fig. 9 As shown, the information processing device 900 includes: a first acquisition module 901, a determination module 902 and a second acquisition module 903.

[0128] A first acquisition module 901 is used to acquire identification information of a candidate first subject in an image;

[0129] A determination module 902 is configured to perform subject recall based on the identification information of the first subject, determine a second subject recalled from the first subject and a recommended query associated with the second subject, and send the recommended query to a client;

[0130] The second acquisition module 903 is used to conduct multiple rounds of dialogues with the client based on the recommendation query, obtain the dialogue results of each round of dialogue returned by the cloud-based large model, and send the dialogue results to the client.

[0131] In one embodiment of the present disclosure, the second acquisition module 903 is used to: for the i-th round of dialogue, receive a user query sent by the client, wherein the user query is related to the recommendation query; determine the dialogue information of the i-th round based on the user query, and send the dialogue information of the i-th round to the cloud-based big model; receive a dialogue result returned by the cloud-based big model that matches the dialogue information of the i-th round, and send the dialogue result to the client.

[0132] In one embodiment of the present disclosure, the determination module 902 is used to: determine the classification information and location information of the first subject from the identification information of the first subject; and determine the second subject and the recommended query associated with the second subject from the first subject based on the classification information and location information of the first subject.

[0133] In one embodiment of the present disclosure, the determination module 902 is used to: determine the area of ​​the first subject according to the location information; determine the second subject from the first subject based on the area of ​​the first subject; perform subject rule mapping according to the classification information of the second subject to obtain a recommended query associated with the second subject.

[0134] In one embodiment of the present disclosure, the determination module 902 is used to: obtain the candidate category corresponding to the second subject and the recognition probability of the candidate category from the classification information of the second subject; determine the subject mapping rule corresponding to the candidate category, and generate a candidate query corresponding to the candidate category according to the subject mapping rule; sort the candidate queries based on the recognition probability of the candidate category, and determine the recommended query associated with the second subject from the candidate queries according to the sorting result.

[0135] In one embodiment of the present disclosure, the second acquisition module 903 is used to: perform intent recognition on the user query to determine a target subject and a target category, wherein the target subject is at least one of the second subjects; and generate the conversation information of the i-th round according to the target subject, the target category and the user query.

[0136] In one embodiment of the present disclosure, the device 900 is used to: receive user queries belonging to the same target entity sent by the client in multiple rounds, wherein the user queries sent each time are different and are related to the recommendation queries of the same target entity.

[0137] In one embodiment of the present disclosure, the second acquisition module 903 is used to: if it is identified that the target subject of the i-th round of dialogue is the same as the target subject of the historical dialogue, determine the historical dialogue result of the historical dialogue; combine the target subject, the target type, the user query and the historical dialogue result to generate the dialogue information of the i-th round of dialogue.

[0138] In one embodiment of the present disclosure, the second acquisition module 903 is used to: if it is identified that the target subject of the i-th round of dialogue is different from the target subject of the historical dialogue, combine the target subject, the target type and the user query to generate the dialogue information of the i-th round of dialogue.

[0139] In one embodiment of the present disclosure, the second acquisition module 903 is used to render the conversation result, generate a display card, and send the display card to the client.

[0140] In one embodiment of the present disclosure, the second acquisition module 903 is used to: determine the target subject of the user's query and the queried target type of the target subject based on the conversation information; extract the conversation sub-results of each target type from the conversation information; and render a card based on the target subject and the conversation sub-results of the target type to obtain the display card.

[0141] In one embodiment of the present disclosure, the second acquisition module 903 is used to: determine the recognition probability of the target type of the target subject, and determine the display order of the dialogue sub-results of the target type; based on the display order and the dialogue sub-results of the target type, generate sub-cards of the display card, and combine the sub-cards to obtain the display card.

[0142] The information processing device proposed in the present disclosure obtains the identification information of the candidate first subject in the image, performs subject recall based on the identification information of the first subject, determines the recalled second subject in the first subject and the recommended query associated with the second subject, sends the recommended query to the client, conducts multiple rounds of dialogues with the client based on the recommended query, obtains the dialogue results of each round of dialogue returned by the cloud-based large model, and sends the dialogue results to the client. The present disclosure can conduct multiple rounds of dialogues with the client based on the recommended query and obtain the dialogue results of each round of dialogue returned by the cloud-based large model, thereby improving the efficiency of the user dialogue process, improving the accuracy of the dialogue results, and enhancing the user experience.

[0143] According to an embodiment of the present disclosure, the present disclosure also provides an information processing device for implementing the above-mentioned information processing method.

[0144] Fig.10 is a block diagram of an information processing device according to an embodiment of the present disclosure.

[0145] like Fig.10 As shown, the information processing device 1000 includes: an identification module 1001, a first receiving module 1002, a dialogue module 1003 and a second receiving module 1004.

[0146] The recognition module 1001 is used to receive an image, recognize the image, and obtain recognition information of a candidate first subject in the image;

[0147] A first receiving module 1002 is configured to send the identification information of the first subject to a front end, and receive a recommended query fed back by the front end, wherein the recommended query is a recommended query associated with a second subject recalled from the first subject based on the identification information;

[0148] A dialogue module 1003, configured to conduct multiple rounds of dialogue with the front end based on the recommendation query;

[0149] The second receiving module 1004 is used to receive the dialogue results of each round of dialogue output by the cloud-based large model fed back by the front end.

[0150] In one embodiment of the present disclosure, the dialogue module 1003 is used to: send user queries related to the recommendation query to the front end in multiple rounds, wherein different user queries are used to generate different dialogue information.

[0151] In one embodiment of the present disclosure, the user queries sent in multiple rounds are associated with the same target subject; or, two user queries sent adjacently are associated with different target subjects.

[0152] In one embodiment of the present disclosure, the device 1000 is further used to: determine the target function to be called by the image from the displayed multiple candidate functions, and enter the image recognition model associated with the target function.

[0153] In one embodiment of the present disclosure, the device 1000 is further used to: receive a display card of the front-end feedback and display it on an interactive interface, wherein the display card is rendered by the dialogue information.

[0154] The information processing device proposed in the present disclosure receives an image, identifies the image, obtains identification information of a candidate first subject in the image, sends the identification information of the first subject to a front end, receives a recommendation query fed back by the front end, the recommendation query is a recommendation query associated with a second subject recalled from the first subject based on the identification information, conducts multiple rounds of dialogues with the front end based on the recommendation query, and receives dialogue results of each round of dialogue output by a cloud-based large model fed back by the front end. Thus, by obtaining the identification information of the candidate first subject in the image and conducting multiple rounds of dialogues with the front end based on the recommendation query, the efficiency of the user dialogue process is improved, the accuracy of the dialogue results is improved, and the user experience is improved.

[0155] According to an embodiment of the present disclosure, the present disclosure also proposes an electronic device, a readable storage medium, and a computer program product.

[0156] Fig.11 A schematic block diagram of an example electronic device 1100 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0157] like Fig.11 As shown, the device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0158] A number of components in the device 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1106, such as various types of displays, speakers, etc.; a storage unit 1108, such as a disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0159] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as information processing methods. For example, in some embodiments, the information processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1100 via ROM 1102 and / or communication unit 1109. When the computer program is loaded into RAM 1103 and executed by the computing unit 1101, one or more steps of the information processing method described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to perform the information processing method in any other appropriate manner (e.g., by means of firmware).

[0160] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0161] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be presented to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0162] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0163] To propose interactions with a user account, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user account; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user account can propose input to the computer. Other types of devices may also be used to propose interactions with the user account; for example, feedback proposed to the user account may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user account may be received in any form (including acoustic input, voice input, or tactile input).

[0164] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user account computer with a graphical user account interface or a web browser through which a user account can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0165] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0166] According to an embodiment of the present disclosure, the present disclosure further provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the steps of the information processing method described in the above embodiment of the present disclosure are implemented.

[0167] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0168] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An information processing method, wherein: The method comprises: Obtaining identification information of a candidate first subject in the image; Recalling a subject based on the identification information of the first subject, determining a second subject in the first subject that is recalled and a recommended query associated with the second subject, and sending the recommended query to a client; Conduct multiple rounds of dialogues with the client based on the recommendation query, obtain dialogue results of each round of dialogues returned by the cloud-based large model, and send the dialogue results to the client.

2. The method according to claim 1, wherein: The performing multiple rounds of dialogues with the client based on the recommendation query, obtaining dialogue results of each round of dialogues returned by the cloud-based large model, and sending the dialogue results to the client, includes: For the i-th round of dialogue, receiving a user query sent by the client, wherein the user query is related to the recommendation query; Determine the conversation information of the i-th round based on the user query, and send the conversation information of the i-th round to the cloud-based big model; Receive a conversation result returned by the cloud-based large model that matches the conversation information of the i-th round, and send the conversation result to the client.

3. The method according to claim 1, wherein: The recalling of subjects based on the identification information of the first subject and determining the recalled second subject in the first subject and the recommended query associated with the second subject include: determining classification information and location information of the first subject from the identification information of the first subject; According to the classification information and the location information of the first subject, the second subject and the recommended query associated with the second subject are determined from the first subject.

4. The method according to claim 3, wherein: The determining, from the first subject according to the classification information and the location information of the first subject, the second subject and the recommended query associated with the second subject, comprises: Determining the area of ​​the first body according to the position information; determining the second body from the first body based on the area of ​​the first body; Subject rule mapping is performed according to the classification information of the second subject to obtain a recommended query associated with the second subject.

5. The method according to claim 3, wherein: The performing subject rule mapping according to the classification information of the second subject to obtain a recommended query associated with the second subject includes: acquiring, from the classification information of the second subject, a candidate category corresponding to the second subject and a recognition probability of the candidate category; Determine a subject mapping rule corresponding to the candidate category, and generate a candidate query corresponding to the candidate category according to the subject mapping rule; The candidate queries are sorted based on the recognition probabilities of the candidate categories, and a recommended query associated with the second subject is determined from the candidate queries according to the sorting result.

6. The method according to claim 2, wherein: The determining the conversation information of the i-th round based on the user query includes: Performing intent recognition on the user query to determine a target subject and a target category, wherein the target subject is at least one of the second subjects; The dialogue information of the i-th round is generated according to the target subject, the target category and the user query.

7. The method according to claim 2, wherein: The method further comprises: The user queries belonging to the same target subject are received from the client in multiple rounds, wherein the user queries sent each time are different and are related to the recommendation queries of the same target subject.

8. The method according to claim 6, wherein: The generating the dialog information according to the target subject, the target category and the user query includes: If it is identified that the target subject of the i-th round of dialogue is the same as the target subject of the historical dialogue, determining the historical dialogue result of the historical dialogue; The target subject, the target type, the user query and the historical dialogue results are combined to generate dialogue information of the i-th dialogue round.

9. The method according to claim 6, wherein: The generating the dialog information according to the target subject, the target category and the user query includes: If it is identified that the target subject of the i-th round of dialogue is different from the target subject of the historical dialogue, the target subject, the target type and the user query are combined to generate dialogue information of the i-th round of dialogue.

10. The method according to any one of claims 1 to 9, wherein: The sending the conversation result to the client comprises: The conversation result is rendered to generate a display card, and the display card is sent to the client.

11. The method according to claim 10, wherein: The rendering of the conversation result to generate a display card includes: Determining, according to the dialog information, a target subject queried by the user and a target type of the target subject being queried; extracting a dialog sub-result of each target type from the dialog information; Card rendering is performed based on the target subject and the target type of dialog sub-result to obtain the display card.

12. The method according to claim 11, wherein: The card rendering based on the dialog sub-result of the target subject and the target type to obtain the display card includes: Determine the recognition probability of the target type of the target subject, and determine the display order of the dialog sub-results of the target type; Based on the display order and the dialog sub-results of the target type, sub-cards of the display card are generated, and the sub-cards are combined to obtain the display card.

13. An information processing method, wherein: The method comprises: Receiving an image, and identifying the image to obtain identification information of a candidate first subject in the image; Sending the identification information of the first subject to the front end, and receiving a recommended query fed back by the front end, wherein the recommended query is a recommended query associated with the second subject recalled from the first subject based on the identification information; Conduct multiple rounds of dialogue with the front end based on the recommendation query; Receive the dialogue results of each round of dialogue output by the cloud-based big model fed back by the front end.

14. The method according to claim 13, wherein: The performing multiple rounds of dialogue with the front end based on the recommendation query includes: Sending user queries related to the recommendation query to the front end in multiple rounds, wherein different user queries are used to generate different dialogue information.

15. The method according to claim 14, wherein: The user queries sent in multiple rounds are associated with the same target subject; or, two user queries sent adjacently are associated with different target subjects.

16. The method according to any one of claims 13 to 15, wherein: Before identifying the image and obtaining identification information of the candidate first subject in the image, the method further includes: From the displayed multiple candidate functions, determine the target function to be called by the image, and enter the image recognition model associated with the target function.

17. The method according to any one of claims 13 to 15, wherein: The method further comprises: A display card of the front-end feedback is received and displayed on an interactive interface, wherein the display card is rendered by the dialogue information.

18. An information processing device, wherein: The device comprises: A first acquisition module, used to acquire identification information of a candidate first subject in an image; a determination module, configured to perform subject recall based on the identification information of the first subject, determine a second subject recalled from the first subject and a recommended query associated with the second subject, and send the recommended query to a client; The second acquisition module is used to conduct multiple rounds of dialogues with the client based on the recommendation query, obtain the dialogue results of each round of dialogue returned by the cloud-based large model, and send the dialogue results to the client.

19. The device according to claim 18, wherein: The second acquisition module is used to: For the i-th round of dialogue, receiving a user query sent by the client, wherein the user query is related to the recommendation query; Determine the conversation information of the i-th round based on the user query, and send the conversation information of the i-th round to the cloud-based big model; Receive a conversation result returned by the cloud-based large model that matches the conversation information of the i-th round, and send the conversation result to the client.

20. The device according to claim 18, wherein The determining module is used to: determining classification information and location information of the first subject from the identification information of the first subject; According to the classification information and the location information of the first subject, the second subject and the recommended query associated with the second subject are determined from the first subject.

21. The device according to claim 20, wherein: The determining module is used to: Determining the area of ​​the first body according to the position information; determining the second body from the first body based on the area of ​​the first body; Subject rule mapping is performed according to the classification information of the second subject to obtain a recommended query associated with the second subject.

22. The device according to claim 20, wherein: The determining module is used to: acquiring, from the classification information of the second subject, a candidate category corresponding to the second subject and a recognition probability of the candidate category; Determine a subject mapping rule corresponding to the candidate category, and generate a candidate query corresponding to the candidate category according to the subject mapping rule; The candidate queries are sorted based on the recognition probabilities of the candidate categories, and a recommended query associated with the second subject is determined from the candidate queries according to the sorting result.

23. The device according to claim 19, wherein: The second acquisition module is used to: Performing intent recognition on the user query to determine a target subject and a target category, wherein the target subject is at least one of the second subjects; The dialogue information of the i-th round is generated according to the target subject, the target category and the user query.

24. The device according to claim 19, wherein The device is also used for: The user queries belonging to the same target subject are received from the client in multiple rounds, wherein the user queries sent each time are different and are related to the recommendation queries of the same target subject.

25. The device according to claim 23, wherein: The second acquisition module is used to: If it is identified that the target subject of the i-th round of dialogue is the same as the target subject of the historical dialogue, determining the historical dialogue result of the historical dialogue; The target subject, the target type, the user query and the historical dialogue results are combined to generate dialogue information of the i-th dialogue round.

26. The device according to claim 23, wherein The second acquisition module is used to: If it is identified that the target subject of the i-th round of dialogue is different from the target subject of the historical dialogue, the target subject, the target type and the user query are combined to generate dialogue information of the i-th round of dialogue.

27. The device according to any one of claims 18 to 26, wherein: The second acquisition module is used to: The conversation result is rendered to generate a display card, and the display card is sent to the client.

28. The device according to claim 27, wherein The second acquisition module is used to: Determining, according to the dialogue result, a target subject queried by the user and a target type of the target subject being queried; extracting a dialog sub-result of each target type from the dialog information; Card rendering is performed based on the target subject and the target type of dialog sub-result to obtain the display card.

29. The device according to claim 28, wherein The second acquisition module is used to: Determine the recognition probability of the target type of the target subject, and determine the display order of the dialog sub-results of the target type; Based on the display order and the dialog sub-results of the target type, sub-cards of the display card are generated, and the sub-cards are combined to obtain the display card.

30. An information processing device, wherein: The device comprises: A recognition module, used to receive an image, recognize the image, and obtain recognition information of a candidate first subject in the image; A receiving module, configured to send the identification information of the first subject to a front end, and receive a recommended query fed back by the front end, wherein the recommended query is a recommended query associated with a second subject recalled from the first subject based on the identification information; A dialogue module, used for conducting multiple rounds of dialogue with the front end based on the recommendation query; The receiving module is used to receive the dialogue results of each round of dialogue output by the cloud-based large model fed back by the front end.

31. The device according to claim 30, wherein The dialogue module is used to: Sending user queries related to the recommendation query to the front end in multiple rounds, wherein different user queries are used to generate different dialogue information.

32. The device according to claim 31, wherein The user queries sent in multiple rounds are associated with the same target subject; or, two user queries sent adjacently are associated with different target subjects.

33. The device according to any one of claims 30 to 32, wherein: The device is also used for: From the displayed multiple candidate functions, determine the target function to be called by the image, and enter the image recognition model associated with the target function.

34. The device according to any one of claims 30 to 32, wherein: The device is also used for: A display card of the front-end feedback is received and displayed on an interactive interface, wherein the display card is rendered by the dialogue information.

35. An electronic device, characterized in that: including a processor and a memory; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the method according to any one of claims 1-12 or claims 13-17.

36. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 12 or claims 13 to 17 is implemented.

37. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1-12 or claims 13-17.

Citation Information

Patent Citations

  • Intelligent robot dialogue method and system based on visual scenario

    CN107016046A

  • Conversation element recommendation method and device, electronic equipment and medium

    CN111385188A

  • Display device, server and media asset content recommendation method

    CN115695844A

  • Image processing method and related device

    CN116700575A

  • Image processing method and device based on artificial intelligence, equipment and storage medium

    CN116975336A

Cited By

  • Image processing method and device based on artificial intelligence and electronic equipment

    CN120783094A

  • Time sequence card generation method and device based on large language model

    CN121212089A

  • A time sequence card generation method and device based on a large language model

    CN121212089B