Photo processing method and device, computer equipment and readable storage medium

Abstract

The invention relates to a photo processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: receiving a photo retrieval instruction sent by a terminal; the photo retrieval instruction carries an account identifier; identifying first intention information corresponding to the photo retrieval instruction through a multi-modal large model; converting the photo retrieval instruction into a corresponding first computer instruction according to the first intention information; according to the first computer instruction, screening out a target photo meeting the first intention information from cloud storage photos corresponding to the account identifier; and returning the target photo to the terminal. By adopting the method, the photo retrieval efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a photo processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] In order to expand the storage space of the terminal, the user usually uploads a large number of photos taken in the terminal to the server for cloud storage, and then retrieves the relevant photos from the server when it is necessary to display specific photos.

[0003] In traditional technology, when retrieving relevant photos from a server, it is usually done manually by the user, that is, manually retrieving the required photos from the server; however, there are a lot of photos stored in the server, and manual retrieval is a cumbersome process and takes a lot of time, resulting in low photo retrieval efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a photo processing method, device, computer equipment, computer-readable storage medium and computer program product that can improve photo retrieval efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a photo processing method, which is applied to a server and includes:

[0006] A photo retrieval instruction sent by a receiving terminal; the photo retrieval instruction carries an account identifier;

[0007] Identifying, by means of a multimodal large model, first intent information corresponding to the photo retrieval instruction;

[0008] converting the photo retrieval instruction into a corresponding first computer instruction according to the first intention information;

[0009] According to the first computer instruction, filter out target photos that meet the first intention information from the cloud storage photos corresponding to the account identifier;

[0010] The target photo is returned to the terminal.

[0011] In one embodiment, the step of selecting target photos that meet the first intent information from the cloud storage photos corresponding to the account identifier according to the first computer instruction includes:

[0012] Determine, according to the first computer instruction, a target photo tag associated with the first intention information and a target photo attribute associated with the first intention information;

[0013] Filtering candidate photos whose corresponding photo tags also include the target photo tag from the cloud storage photos corresponding to the account identifier in the file management system; the cloud storage photos are stored in the file management system, and each of the cloud storage photos carries the corresponding photo tag;

[0014] From the candidate photos, a photo corresponding to the attribute of the target photo is screened out as the target photo.

[0015] In one embodiment, before receiving the photo retrieval instruction sent by the terminal, the method further includes:

[0016] Using the multimodal large model, content recognition processing is performed on the cloud storage photos corresponding to the account identifier in the file management system to obtain content information of the cloud storage photos;

[0017] Determine a photo tag of the cloud storage photo according to the content information of the cloud storage photo.

[0018] In one embodiment, after selecting the target photos that meet the first intention information, the method further includes:

[0019] Sorting the target photos to obtain sorted target photos, and determining first feedback information of the photo retrieval instruction according to the target photos; the first feedback information at least includes first reply information of the photo retrieval instruction and first guidance information associated with the first reply information;

[0020] The step of returning the target photo to the terminal includes:

[0021] The sorted target photos and the first feedback information are returned to the terminal; the terminal is used to display thumbnails of the sorted target photos, and to display the first feedback information in a display manner that matches the photo retrieval instruction, and to perform corresponding photo playback processing in response to the photo playback instruction for the first feedback information.

[0022] In one embodiment, sorting the target photos to obtain sorted target photos includes:

[0023] Determining an arrangement order of photos corresponding to the first intent information;

[0024] The target photos are sorted according to the photo arrangement order to obtain the sorted target photos.

[0025] In one of the embodiments, the terminal is deployed with the multimodal large model;

[0026] After returning the target photo to the terminal, the method further includes:

[0027] receiving a second computer instruction sent by the terminal; the terminal is used to identify, through the multimodal macromodel, second intention information corresponding to the photo editing instruction for the selected photo among the target photos, and convert the photo editing instruction into the corresponding second computer instruction according to the second intention information;

[0028] According to the second computer instruction, the selected photo is edited to obtain an edited photo, and according to the edited photo, second feedback information of the photo editing instruction is determined; the second feedback information at least includes second reply information of the photo editing instruction and second guidance information associated with the second reply information;

[0029] The edited photo and the second feedback information are returned to the terminal; the terminal is used to display the edited photo and the second feedback information in a display manner matching the photo editing instruction.

[0030] In a second aspect, the present application further provides a photo processing device, which is applied to a server and includes:

[0031] An instruction receiving module, used for receiving a photo retrieval instruction sent by a terminal; the photo retrieval instruction carries an account identifier;

[0032] An intention recognition module, used to recognize first intention information corresponding to the photo retrieval instruction through a multimodal large model;

[0033] an instruction conversion module, configured to convert the photo retrieval instruction into a corresponding first computer instruction according to the first intention information;

[0034] a photo screening module, configured to screen target photos that meet the first intention information from the cloud storage photos corresponding to the account identifier according to the first computer instruction;

[0035] The photo returning module is used to return the target photo to the terminal.

[0036] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0037] A photo retrieval instruction sent by a receiving terminal; the photo retrieval instruction carries an account identifier;

[0038] Identifying, by means of a multimodal large model, first intent information corresponding to the photo retrieval instruction;

[0039] converting the photo retrieval instruction into a corresponding first computer instruction according to the first intention information;

[0040] According to the first computer instruction, filter out target photos that meet the first intention information from the cloud storage photos corresponding to the account identifier;

[0041] The target photo is returned to the terminal.

[0042] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0043] A photo retrieval instruction sent by a receiving terminal; the photo retrieval instruction carries an account identifier;

[0044] Identifying, by means of a multimodal large model, first intent information corresponding to the photo retrieval instruction;

[0045] converting the photo retrieval instruction into a corresponding first computer instruction according to the first intention information;

[0046] According to the first computer instruction, filter out target photos that meet the first intention information from the cloud storage photos corresponding to the account identifier;

[0047] The target photo is returned to the terminal.

[0048] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0049] A photo retrieval instruction sent by a receiving terminal; the photo retrieval instruction carries an account identifier;

[0050] Identifying, by means of a multimodal large model, first intent information corresponding to the photo retrieval instruction;

[0051] converting the photo retrieval instruction into a corresponding first computer instruction according to the first intention information;

[0052] According to the first computer instruction, filter out target photos that meet the first intention information from the cloud storage photos corresponding to the account identifier;

[0053] The target photo is returned to the terminal.

[0054] The above-mentioned photo processing method, device, computer equipment, computer-readable storage medium and computer program product, by receiving the photo retrieval instruction sent by the terminal, and identifying the first intent information corresponding to the photo retrieval instruction through the multimodal large model; the photo retrieval instruction carries the account identification; then according to the first intent information, the photo retrieval instruction is converted into the corresponding first computer instruction, and finally according to the first computer instruction, the target photos that meet the first intent information are screened out from the cloud storage photos corresponding to the account identification, and the target photos are returned to the terminal; in this way, when retrieving the target photo, the first intent information of the photo retrieval instruction sent by the terminal is identified through the multimodal large model and converted into the first computer instruction, and then according to the first computer instruction, the target photos that meet the first intent information are automatically retrieved from the cloud storage photos, thereby achieving the purpose of automatic retrieval of photos without manual retrieval, thereby simplifying the photo retrieval process and improving the photo retrieval efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A diagram showing an application environment of a photo processing method in an embodiment;

[0057] Figure 2 is a schematic diagram of a process of a photo processing method in one embodiment;

[0058] Figure 3 is a schematic diagram of a process of photo editing steps in one embodiment;

[0059] Figure 4 is a schematic flow chart of a photo processing method in another embodiment;

[0060] Figure 5 A schematic diagram of a process of retrieving cloud storage photos in one embodiment;

[0061] Figure 6 A schematic diagram of a flow chart of a method for editing cloud storage photos in one embodiment;

[0062] Figure 7 is a schematic diagram of the structure of a local device in one embodiment;

[0063] Figure 8 is a structural block diagram of a photo processing device in one embodiment;

[0064] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0066] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0067] In order to expand storage space or achieve portability, users will use cloud storage to save a large number of photos or videos they have taken; when they need to display content, they will use mobile terminals such as mobile phones or electronic photo frames to log in to the cloud storage account, manually search for target photos or videos, and then perform demonstrations; these devices often only support the display of photos according to the logic set by the user, or static, or sequential, or random playback. Therefore, a method and device that can support users to intelligently display or edit content after retrieving photos from cloud storage albums is needed to reduce the difficulty of users managing increasingly large electronic albums; based on this, the present application proposes a photo processing method.

[0068] The photo processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown in FIG. 1 , the terminal 102 communicates with the server 104 via a network. Specifically, refer to Figure 1 , the terminal 102 collects the photo retrieval instruction carrying the account identifier, and sends the photo retrieval instruction to the server 104. The server 104 identifies the first intent information corresponding to the received photo retrieval instruction through the multimodal large model; according to the first intent information, the photo retrieval instruction is converted into the corresponding first computer instruction; according to the first computer instruction, the target photo that meets the first intent information is screened from the cloud storage photos corresponding to the account identifier, and the target photo is returned to the terminal 102. The terminal 102 displays the received target photo on the interface. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablet computers and Internet of Things devices, and the Internet of Things devices can be smart TVs, electronic photo frames, etc. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services, such as a cloud storage server.

[0069] It should be noted that the photo processing method provided in this application is mainly explained using photo processing as an example, and is also applicable to video processing. That is, the method for retrieving or editing cloud-stored photos provided in this application is also applicable to retrieving or editing cloud-stored videos.

[0070] In an exemplary embodiment, Figure 2 As shown, a photo processing method is provided, which is applied to Figure 1 The server in the example is used to illustrate, including the following steps S201 to S205. Among them:

[0071] Step S201, receiving a photo retrieval instruction sent by a terminal; the photo retrieval instruction carries an account identifier.

[0072] The terminal may refer to an electronic photo frame, a television, a tablet computer, a handheld terminal (such as a smart phone), etc.

[0073] Among them, photo retrieval instructions refer to multimodal instructions for retrieving cloud-stored photos issued by users in the form of natural languages ​​such as voice, touch, body movements, and text, specifically voice instructions, touch instructions, body movement instructions, text instructions, etc.; for example, the user inputs the voice "When was the last time I went to Huangshan", inputs the text "What are the photos of hiking", and makes gestures to indicate the search for scenic cloud-stored photos, etc.

[0074] Among them, the account identifier refers to the user's unique identification information, such as user account, identity information, etc.

[0075] Exemplarily, the terminal collects a photo retrieval instruction issued by a user that carries the user's account identification, and sends the photo retrieval instruction to a corresponding server, such as a cloud storage server.

[0076] For example, see Figure 5 , the user issues a photo retrieval command through the terminal in the form of natural language such as voice, touch, body movement, text, etc.; the terminal collects the photo retrieval command issued by the user through the various natural language collection modules set up, and sends the photo retrieval command to the server. For example, the terminal uses an acoustic module including a microphone array to collect the user's voice command, uses a touch screen to collect the user's input text or action command, uses a remote control or other control device to collect the user's input text or action command, and collects the user's body movement command through an action collection module.

[0077] Step S202: Identify the first intent information corresponding to the photo retrieval instruction through the multimodal large model.

[0078] Among them, the multimodal big model refers to a big model that can recognize and understand photo retrieval instructions and convert them into computer instructions. In actual scenarios, the multimodal big model refers to a new generation of artificial intelligence system with multimodal interaction capabilities such as speech recognition and synthesis, image understanding, and body movement recognition.

[0079] Among them, the first intent information refers to the user intent information corresponding to the photo retrieval instruction; for example, assuming the photo retrieval instruction is "When was the last time I went to Huangshan", then the first intent information refers to "find the photos taken the last time I went to Huangshan".

[0080] It should be noted that the way in which a multimodal large model understands user intent information includes at least the following: (1) Statistical-based methods: For example, the language model in a multimodal large model can be trained using an annotated corpus to learn the association and probability distribution between words; in this method, each word in a sentence is regarded as an implicit state, and the goal of the model is to find the best word division for a given sentence to maximize the probability of the entire sentence. (2) Neural network-based methods: For example, the language model in a multimodal large model can use deep learning techniques, such as recurrent neural networks (Recurrent Neural Networks) or Transformer models, taking each character in a sentence as input and predicting the position of the word segmentation by learning the relationship between characters; this method usually requires a large amount of annotated data for training, but after the model is trained, it can segment sentences of any length.

[0081] Exemplarily, the server preprocesses the photo retrieval instruction, such as noise reduction, to obtain a preprocessed photo retrieval instruction, and then inputs the preprocessed photo retrieval instruction into the multimodal large model, and uses the multimodal large model to identify and understand the preprocessed photo retrieval instruction to obtain intent information corresponding to the preprocessed photo retrieval instruction as the first intent information; for example, the server identifies the first intent information corresponding to the photo retrieval instruction through a statistical-based method or a neural network-based method.

[0082] For example, the server will perform simple processing such as noise reduction on user commands (i.e., photo retrieval commands) input in natural languages ​​such as voice, text, and body movements, and then send them to the multimodal large model; the multimodal large model will perform a series of recognition and understanding processing on the user commands to obtain the user's intention information.

[0083] Step S203: converting the photo retrieval instruction into a corresponding first computer instruction according to the first intention information.

[0084] The first computer instruction refers to a computer instruction corresponding to the photo retrieval instruction, which is mainly used to implement the first intention information. For example, if the photo retrieval instruction is "When was the last time I went to Huangshan", the first computer instruction is "Retrieve the photos I took when I went to Huangshan last time".

[0085] Exemplarily, the server converts the photo retrieval instruction according to the first intention information to obtain a first computer instruction corresponding to the photo retrieval instruction. For example, the server converts the user instruction into a corresponding computer instruction after understanding the user's intention.

[0086] Furthermore, the server may also query the correspondence between the intent information and the computer instruction based on the first intent information, obtain the computer instruction corresponding to the first intent information, and use the computer instruction as the first computer instruction corresponding to the photo retrieval instruction.

[0087] Step S204: According to the first computer instruction, target photos that meet the first intention information are screened out from the cloud storage photos corresponding to the account identifier.

[0088] Among them, the file management system in the server stores cloud storage photos corresponding to the account identification, such as photos taken by the user.

[0089] The target photo refers to a photo that satisfies the first intent information; for example, if the first intent information is “find photos taken during the last trip to Huangshan”, then the target photo is “photos taken during the last trip to Huangshan”.

[0090] Exemplarily, the server identifies cloud storage photos corresponding to the account identifier from the file management system according to the first computer instruction, and then filters out photos matching the first intent information from the cloud storage photos corresponding to the account identifier as target photos.

[0091] For example, according to the first computer instruction, the server screens the cloud storage photos corresponding to the account identifier multiple times based on pre-classification, photo attributes, photo time and other conditions to obtain target photos that meet the first intent information.

[0092] Step S205, returning the target photo to the terminal.

[0093] Exemplarily, the server returns the retrieved target photos to the terminal; the terminal displays thumbnails of the target photos on the terminal interface in the order of shooting time, such as thumbnails of photos taken by the user during his last trip to Huangshan.

[0094] Furthermore, the server can also send the first reply information of the photo retrieval instruction and the first guidance information associated with the first reply information to the terminal; the terminal displays the first reply information and the first guidance information while displaying the thumbnail of the target photo. For example, the last time you visited Huangshan was May 1, 2024. Do you need to display these photos?

[0095] In the above-mentioned photo processing method, a photo retrieval instruction sent by a terminal is received, and the first intent information corresponding to the photo retrieval instruction is identified through a multimodal large model; the photo retrieval instruction carries an account identifier; then, according to the first intent information, the photo retrieval instruction is converted into a corresponding first computer instruction, and finally, according to the first computer instruction, the target photos that meet the first intent information are screened out from the cloud storage photos corresponding to the account identifier, and the target photos are returned to the terminal; in this way, when retrieving the target photo, the first intent information of the photo retrieval instruction sent by the terminal is identified through the multimodal large model and converted into the first computer instruction, and then, according to the first computer instruction, the target photos that meet the first intent information are automatically retrieved from the cloud storage photos, thereby achieving the purpose of automatic photo retrieval without manual retrieval, thereby simplifying the photo retrieval process and improving the photo retrieval efficiency.

[0096] In an exemplary embodiment, the above step S204, according to the first computer instruction, filters out target photos that meet the first intent information from the cloud storage photos corresponding to the account identifier, and specifically includes the following contents: according to the first computer instruction, determine the target photo tag associated with the first intent information and the target photo attributes associated with the first intent information; from the cloud storage photos corresponding to the account identifier in the file management system, filter out candidate photos that have the corresponding photo tags and the target photo tags; the file management system stores cloud storage photos, and each cloud storage photo carries a corresponding photo tag; from the candidate photos, filter out photos corresponding to the target photo attributes as target photos.

[0097] The file management system stores cloud storage photos corresponding to different account identifiers, and each cloud storage photo carries a corresponding photo tag and photo attributes.

[0098] Among them, the target photo tag refers to the photo tag associated with the first intention information, such as "myself", "travel", "hiking", "outdoors", "pine tree", etc.

[0099] The target photo attributes refer to the photo attributes associated with the first intent information, such as shooting time, shooting location, etc.

[0100] Among them, candidate photos refer to cloud storage photos corresponding to the account ID, whose corresponding photo tags also contain the target photo tags, such as cloud storage photos that also contain photo tags such as "myself", "travel", "hiking", "outdoors", and "pine trees".

[0101] The target photo refers to a photo among the candidate photos that corresponds to the attributes of the target photo, such as a photo that satisfies the shooting location.

[0102] Exemplarily, the server determines, based on the first computer instruction, a photo tag associated with the first intent information as a target photo tag, and determines a photo attribute associated with the first intent information as a target photo attribute; then, from the file management system, the server identifies the cloud storage photos corresponding to the account ID, and from the cloud storage photos corresponding to the account ID, filters out cloud storage photos that have the corresponding photo tag and contain the target photo tag as candidate photos; finally, from the candidate photos, filters out photos corresponding to the target photo attributes as target photos.

[0103] For example, a user asks a voice question to achieve the purpose of image retrieval: "When was the last time I went to Huangshan?" After recognizing the user's intention, the multimodal large model first looks for photos that contain labels such as "myself", "travel", "hiking", "outdoors", and "pine trees"; then combines attribute information such as the shooting location of the photos to further narrow the screening range of photos; finally, these photos are sorted by shooting time, and the sorted photos are obtained as target photos, that is, photos whose labels, shooting time, and shooting location all meet the first intention information.

[0104] In this embodiment, the target photo tag associated with the first intent information and the target photo attribute associated with the first intent information are first determined according to the first computer instruction, and then candidate photos having the corresponding photo tag and the target photo tag are screened out from the cloud storage photos corresponding to the account identifier in the file management system, and finally the target photo corresponding to the target photo attribute is screened out from the candidate photos; in this way, the target photo is obtained by multiple screenings, which can improve the retrieval accuracy of the target photo, and at the same time, there is no need for manual retrieval, which is conducive to improving the photo retrieval efficiency.

[0105] In an exemplary embodiment, the above step S201, before receiving the photo retrieval instruction sent by the terminal, also includes the following content: using a multimodal large model, content recognition processing is performed on the cloud storage photos corresponding to the account identifier in the file management system to obtain content information of the cloud storage photos; based on the content information of the cloud storage photos, the photo tags of the cloud storage photos are determined.

[0106] The content information of the cloud storage photos is used to indicate the content contained in the cloud storage photos, such as people, scenery, etc.

[0107] Among them, the photo tags of the cloud storage photos are used to represent the photo categories of the cloud storage photos, such as people, landscapes, animals, etc.

[0108] Exemplarily, the server inputs the cloud storage photo corresponding to the account ID into the multimodal big model, performs content recognition processing on the cloud storage photo through the multimodal big model, and obtains the content information of the cloud storage photo; then the photo tag corresponding to the content information of the cloud storage photo is used as the photo tag of the cloud storage photo.

[0109] For example, after a user logs in to a cloud storage account and enables the intelligent retrieval and editing functions, the multimodal large model, with the assistance of the file management system in the cloud storage server, will traverse the photos and videos under the user account located in the cloud storage server, and annotate them with data labels after understanding their content. The data labels include at least categories such as people, dynamics, landscapes, places, and files.

[0110] In this embodiment, content recognition processing is performed in advance on the cloud storage photos corresponding to the account identifier in the file management system through a multimodal large model to obtain content information of the cloud storage photos, and then photo tags of the cloud storage photos are determined based on the content information of the cloud storage photos. This is beneficial for subsequent rapid retrieval of target photos that meet the first intent information based on the predetermined photo tags of the cloud storage photos, and is beneficial for further improving photo retrieval efficiency.

[0111] In an exemplary embodiment, after filtering out the target photos that meet the first intention information, the above step S204 further includes the following contents: sorting the target photos to obtain sorted target photos, and determining the first feedback information of the photo retrieval instruction based on the target photos; the first feedback information at least includes the first reply information of the photo retrieval instruction, and the first guidance information associated with the first reply information. Then, the above step S205, returning the target photos to the terminal, specifically includes the following contents: returning the sorted target photos and the first feedback information to the terminal; the terminal is used to display thumbnails of the sorted target photos, and display the first feedback information in a display mode that matches the photo retrieval instruction, and perform corresponding photo playback processing in response to the photo playback instruction for the first feedback information.

[0112] The first feedback information refers to the feedback information for the photo retrieval instruction, specifically including the first reply information and the first guidance information.

[0113] Among them, the first reply information refers to the reply information to the photo retrieval instruction. For example, if the photo retrieval instruction is "When was the last time I went to Huangshan", then the first reply information is "The last time you went to Huangshan was May 1, 2024."

[0114] Among them, the first guidance information refers to the guidance information associated with the first reply information. For example, if the first reply information is "The last time you visited Huangshan was on May 1, 2024", then the first guidance information is "Do you need to show these photos?"

[0115] Among them, if the photo retrieval instruction is in voice form, the first feedback information is displayed in voice form, and if the photo retrieval instruction is in other forms, the first feedback information is displayed in text form. For example, the terminal (i.e., the local device) will select a method to guide the user according to the interactive convenience of the user's environment. If the user inputs the instruction in voice, the terminal uses voice to guide the user to perform the operation; if the user inputs the instruction in other ways, the terminal uses text prompts to guide the user to complete the operation.

[0116] The photo play command refers to the play command for the target photo, such as carousel, random selection, full screen play of a photo selected by the user, etc. It can be set to play in sequence when the user does not specify the play order, and follow the user's instructions when there is a clear opinion such as "open the xth photo".

[0117] Exemplarily, the server sorts the target photos according to the photo arrangement order corresponding to the first intent information to obtain sorted target photos; at the same time, based on the retrieved target photos, the server generates the first reply information of the photo retrieval instruction and the first guide information associated with the first reply information, and combines the first reply information with the first guide information to obtain the first feedback information of the photo retrieval instruction, and finally returns the sorted target photos and the first feedback information to the terminal. The terminal displays thumbnails of the sorted target photos on the terminal interface, and displays the first feedback information in a display mode (such as voice or text) that matches the photo retrieval instruction, and performs corresponding photo playback processing in response to the photo playback instruction for the first feedback information, such as playing the target photos in sequence, or opening the third photo, etc.

[0118] For example, the server obtains the user's target photos according to computer instructions, pre-classification of labels, photo attributes, photo time and other multi-step screening; then, according to the user's intention, the user's target photos are arranged in the corresponding order, and the user is guided to complete the visual operation in a way that matches the user's environment. For example, after the large model recognizes that the user wants to find the photos taken during the most recent trip to Huangshan, it displays the thumbnails of these photos on the screen after completing the screening and sorting steps, and guides the user's next action by voice, which can be displayed in a carousel, randomly selected, and full-screen playback of a photo selected by the user.

[0119] For example, after recognizing the user's intention, the multimodal large model first finds photos with labels such as "me", "travel", "climbing", "outdoors", and "pine trees"; then combines the attribute information such as the photo shooting location to further narrow the photo screening range; finally, these photos are sorted according to the shooting time, and the thumbnails are displayed on the screen while answering the user's question with voice, "Based on the content of the photos and the shooting location, your last visit to Huangshan was on xx / xx / xx", and actively asks "Do you need to show these photos?", and then displays the photos according to the user's further voice instructions. If the user's voice answer is yes and there is no clear playback order, the order will be rotated; if there is a clear opinion such as "open the xth photo", follow the user's instructions.

[0120] In this embodiment, the sorted target photos and the first feedback information of the target photos are returned to the terminal at the same time, so that the terminal can display the sorted target photos, display the first feedback information in a display manner that matches the photo retrieval instruction, and perform corresponding photo playback processing in response to the photo playback instruction for the first feedback information, which is conducive to increasing the interactivity in the photo retrieval process.

[0121] In an exemplary embodiment, sorting the target photos to obtain sorted target photos specifically includes the following contents: determining the photo arrangement order corresponding to the first intent information; sorting the target photos according to the photo arrangement order to obtain sorted target photos.

[0122] The photo arrangement order corresponding to the first intent information refers to the photo arrangement order that matches the time feature words in the first intent information, which is used to indicate that the target photos are displayed according to the user's intent. For example, if the user uses the feature words closest to the current time point, such as "last time", "recently", or "just now", they will be displayed in reverse order; if the user uses the feature words farthest from the current time point, such as "first time", "at the beginning", or "first time", they will be displayed in order.

[0123] Exemplarily, the server identifies the time feature words in the first intent information, obtains the photo arrangement order corresponding to the time feature words, uses the photo arrangement order corresponding to the first intent information, and then sorts the target photos according to the photo arrangement order to obtain sorted target photos.

[0124] For example, if the user uses feature words such as "last time", "recently", "just now" etc. that represent the time closest to the current point in time, the target photos will be sorted in reverse order; if the user uses feature words such as "first time", "beginning", "first time" etc. that represent the time farthest from the current point in time, the target photos will be sorted in order.

[0125] In this embodiment, the target photos are sorted according to the photo arrangement order corresponding to the first intent information to obtain sorted target photos. This can make the sorted target photos more satisfy the first intent information without the need for further screening by the user, which is conducive to further improving the photo retrieval efficiency.

[0126] In an exemplary embodiment, the terminal is deployed with a multimodal large model; then, Figure 3 As shown, the above step S205, after returning the target photo to the terminal, further includes a photo editing step, specifically including the following steps S301 to S303. Among them:

[0127] Step S301, receiving a second computer instruction sent by the terminal; the terminal is used to identify second intention information corresponding to the photo editing instruction for the selected photo in the target photo through the multimodal large model, and convert the photo editing instruction into a corresponding second computer instruction according to the second intention information.

[0128] Step S302, according to the second computer instruction, edit the selected photo to obtain the edited photo, and determine the second feedback information of the photo editing instruction based on the edited photo; the second feedback information at least includes the second reply information of the photo editing instruction and the second guidance information associated with the second reply information.

[0129] Step S303, returning the edited photo and the second feedback information to the terminal; the terminal is used to display the edited photo, and display the second feedback information in a display manner matching the photo editing instruction.

[0130] The selected photo refers to the photo selected by the user from the target photos, such as the photo number.

[0131] Among them, photo editing instructions refer to multimodal instructions for editing cloud-stored photos issued by users in the form of natural languages ​​such as voice, touch, body movement, and text, specifically referring to voice instructions, touch instructions, body movement instructions, text instructions, etc.; for example, the user inputs the voice "delete the person next to him", inputs the text "adjust the photo to gray", makes gestures to indicate the rotation of cloud-stored photos, etc.

[0132] It should be noted that the photo editing processing of this application is not limited to traditional operations such as cropping, rotating, and color adjustment, but also includes operations such as conversion between two-dimensional and three-dimensional images. Specifically, when the local device is configured with a 2D (two-dimensional) display screen, the editing operation supports conversion of three-dimensional content into two-dimensional images or video displays; when the local device is configured with a (naked eye) 3D (three-dimensional) display screen, the editing operation supports conversion of two-dimensional content into three-dimensional images or video displays.

[0133] The second intent information refers to the user intent information corresponding to the photo editing instruction; for example, assuming the photo editing instruction is "delete the person next to him", then the second intent information refers to "delete a person in the photo".

[0134] The second computer instruction refers to a computer instruction corresponding to the photo editing instruction, and is mainly used to implement the second intention information. For example, if the photo editing instruction is "delete the person next to you", the second computer instruction is an operation such as pixel erasing and overwriting.

[0135] The edited photo refers to a photo obtained after editing the selected photo according to the second computer instruction, such as a rotated photo, a color-adjusted photo, etc.

[0136] The second feedback information refers to feedback information regarding the photo editing instruction, and specifically includes second reply information and second guidance information.

[0137] The second reply information refers to the reply information to the photo editing instruction. For example, if the photo editing instruction is "delete the person next to him", then the second reply information is "editing completed".

[0138] The second guidance information refers to the guidance information associated with the second reply information. For example, if the second reply information is “Editing completed”, then the second guidance information is “See how the effect is”.

[0139] Among them, if the photo editing instruction is in voice form, the second feedback information is displayed in voice form; if the photo editing instruction is in other forms, the second feedback information is displayed in text form.

[0140] Exemplarily, the terminal collects the photo editing instructions (such as rotation instructions) issued by the user for the selected photo in the target photo, and preprocesses the photo editing instructions, such as noise reduction processing, to obtain the preprocessed photo editing instructions, and then inputs the preprocessed photo editing instructions into the multimodal large model, and the preprocessed photo editing instructions are recognized and understood by the multimodal large model to obtain the intention information corresponding to the preprocessed photo editing instructions as the second intention information, and then according to the second intention information, the photo editing instructions are converted accordingly to obtain the second computer instructions corresponding to the photo editing instructions, and finally the second computer instructions are sent to the corresponding server. The server performs corresponding editing processing on the selected photo according to the second computer instruction to obtain the edited photo (such as the rotated photo), and then generates the second reply information of the photo editing instruction and the second guidance information associated with the second reply information according to the edited photo, and combines the second reply information with the second guidance information to obtain the second feedback information of the photo editing instruction, and finally returns the edited photo and the second feedback information to the terminal. The terminal displays the edited photo on the terminal interface, and displays the second feedback information in a display mode (such as voice or text) that matches the photo editing instruction, such as "Editing is completed, how is the effect?"

[0141] For example, the user points to a direction and says "delete the person next to me"; the terminal (i.e., the local device) performs voice noise reduction and collection, and then uses the large model to perform voice recognition and natural language understanding to learn that the user's intention is to delete part of the content and fill it in naturally; at the same time, through image content recognition (such as the person himself, group photo, etc.) and the direction of the user's finger, the operation target is selected, and if necessary, the target is framed out for secondary confirmation by the user; the large model converts the user's intention into computer instructions such as pixel erasing, pixel copying and overwriting, hue and sharpness adjustment, and other operation instructions. With the assistance of the file management system in the server, the picture is edited and the edited effect is displayed to guide the user to confirm or take the next editing operation. For example, if the user is not satisfied with the effect, he needs to edit the photo again.

[0142] In this embodiment, when editing a target photo, the multimodal large model is used to identify the second intention information of the photo editing instruction for the selected photo in the target photo, and convert it into a second computer instruction. Then, according to the second computer instruction, the selected photo is edited, so as to achieve the purpose of automatically editing the photo without manual editing, thereby simplifying the photo editing process and improving the photo editing efficiency. At the same time, the edited photo and the second feedback information of the edited photo are returned to the terminal, so that the terminal can display the edited photo, and the second feedback information is displayed in a display mode matching the photo editing instruction, which is conducive to increasing the interactivity in the photo editing process.

[0143] In an exemplary embodiment, Figure 4 As shown, another photo processing method is provided, which is applied to Figure 1 Taking the server in as an example, the method includes the following steps S401 to S413.

[0144] in:

[0145] Step S401, using a multimodal large model, performs content recognition processing on the cloud storage photos corresponding to the account identifier in the file management system to obtain content information of the cloud storage photos; and determines the photo tags of the cloud storage photos based on the content information of the cloud storage photos.

[0146] Step S402, receiving a photo retrieval instruction sent by a terminal; the photo retrieval instruction carries an account identifier.

[0147] Step S403: Identify the first intent information corresponding to the photo retrieval instruction through the multimodal large model.

[0148] Step S404: convert the photo retrieval instruction into a corresponding first computer instruction according to the first intention information.

[0149] Step S405: determining, according to the first computer instruction, a target photo tag associated with the first intent information and a target photo attribute associated with the first intent information.

[0150] Step S406, filter out candidate photos with corresponding photo tags and target photo tags from the cloud storage photos corresponding to the account identifier in the file management system; the file management system stores cloud storage photos, and each cloud storage photo carries a corresponding photo tag.

[0151] Step S407, selecting a photo corresponding to the attributes of the target photo from the candidate photos as the target photo.

[0152] Step S408, determining the photo arrangement order corresponding to the first intent information; sorting the target photos according to the photo arrangement order to obtain sorted target photos.

[0153] Step S409, determining first feedback information of the photo retrieval instruction based on the target photo; the first feedback information at least includes first reply information of the photo retrieval instruction and first guidance information associated with the first reply information.

[0154] Step S410, returns the sorted target photos and the first feedback information to the terminal; the terminal is used to display thumbnails of the sorted target photos, and display the first feedback information in a display manner matching the photo retrieval instruction, and perform corresponding photo playback processing in response to the photo playback instruction for the first feedback information.

[0155] Step S411, receiving a second computer instruction sent by the terminal; the terminal is used to identify second intention information corresponding to the photo editing instruction for the selected photo in the target photo through the multimodal large model, and convert the photo editing instruction into a corresponding second computer instruction according to the second intention information.

[0156] Step S412, according to the second computer instruction, edit the selected photo to obtain the edited photo, and determine the second feedback information of the photo editing instruction based on the edited photo; the second feedback information at least includes the second reply information of the photo editing instruction and the second guidance information associated with the second reply information.

[0157] Step S413, returning the edited photo and the second feedback information to the terminal; the terminal is used to display the edited photo, and display the second feedback information in a display manner matching the photo editing instruction.

[0158] In the above-mentioned photo processing method, when retrieving the target photo, the first intention information of the photo retrieval instruction sent by the terminal is identified through the multimodal large model, and converted into a first computer instruction, and then the target photo that meets the first intention information is automatically retrieved from the cloud storage photos according to the first computer instruction, thereby achieving the purpose of automatic photo retrieval without manual retrieval, thereby simplifying the photo retrieval process and improving the photo retrieval efficiency. At the same time, the purpose of automatic photo editing is achieved without manual editing, thereby simplifying the photo editing process and improving the photo editing efficiency.

[0159] In order to more clearly illustrate the photo processing method provided by the embodiment of the present application, the photo processing method is specifically described below with a specific embodiment. In an exemplary embodiment, the present application proposes a method and device for retrieving or editing cloud-stored photos. Users can use natural languages ​​such as voice, text, and body movements to input instructions to perform operations such as retrieving or editing photos or video content in cloud-stored albums on a local device, which greatly reduces the difficulty for users to manage increasingly large electronic albums.

[0160] like Figure 5 As shown, the present application proposes a method for retrieving cloud-stored photos. The integrated multimodal large model will fully understand the content of the pictures or videos stored in the cloud storage and pre-classify them; after the local device collects and recognizes the instructions input by the user in natural language, the multimodal large model understands the user's intentions based on statistical or neural network reasoning, and then guides the user to perform visual operations such as content demonstration or editing while retrieving the content of the photo or video. This method mainly includes two steps: pre-understanding and classification of picture content, and completing retrieval or editing operations in response to user instructions:

[0161] Step 1: Image pre-understanding and classification

[0162] With the assistance of the file management system in the cloud storage server, the multimodal large model will traverse the photos and videos under the user account located in the cloud storage server, and annotate them with data labels after understanding their content. The data labels include at least categories such as people, dynamics, landscapes, places, and files.

[0163] Step 2: Complete the search or edit operation in response to the user's instructions

[0164] 1) User issues an instruction

[0165] Users can use local devices to issue instructions to retrieve or edit cloud-stored photos in the form of natural languages ​​such as voice, touch, body movements, and text.

[0166] 2) Instruction Collection

[0167] A variety of natural language acquisition modules are arranged on the local device to collect multi-modal instructions from users.

[0168] 3) Command recognition and intent understanding

[0169] The multimodal large model recognizes natural languages ​​such as voice, text, and body movements input by users, understands the user's intentions, and converts user instructions into computer instructions.

[0170] 4) Guide users to complete search operations

[0171] The multimodal large model transmits computer instructions to the file management system of the cloud storage server and performs corresponding operations on the cloud storage photos or videos; it also visually displays the user's operation process on the screen and guides the user to complete subsequent operations with voice or text.

[0172] It should be noted that Figure 5 In the process, the multimodal large model runs on the cloud storage server. At this time, the local device is only used to collect the user's multi-modal natural language instructions and send them to the cloud storage server; receive and display the photo or video data from the cloud storage server and the user's operation feedback.

[0173] like Figure 6As shown, the present application also proposes a method for editing cloud-stored photos. The customized simplified multimodal large model runs on the local device. At this time, after collecting the user's multi-modal natural language instructions, the local device directly recognizes and understands the user's intentions and converts them into computer instructions, and then passes these computer instructions including call clues (i.e. tag content, such as photo tags, which photo, etc.), editing instructions (such as image zooming, pixel erasing, moving, etc. The computer language corresponding to the editing actions) is passed to the file management system of the cloud storage server; photos or videos stored on the cloud storage server, editing operation feedback (such as "Editing is completed, how do you see the effect?") are received and corresponding demonstrations are performed.

[0174] It should be noted that the method designed in this application can not only be used to retrieve or edit photos or videos stored by users on cloud storage servers, but can also be used to access electronic photo albums logged in to the user account and located on multiple terminals such as mobile phones, PADs (tablet computers), digital cameras and other mobile terminals and local devices; when accessing multiple terminal albums, the user's operation authorization should be obtained.

[0175] It should be noted that the method for retrieving or editing user photos or videos designed in the present application performs the retrieval and editing operations on the original storage medium of the photos or videos, and does not copy the content to the storage medium of the local device, which can reduce the hardware resource requirements of the local device.

[0176] like Figure 7 As shown, the present application also proposes a local device, which should include a display screen and a touch screen, an acoustic module, a remote control module, a motion acquisition module and a communication module as well as necessary processors, power modules, storage modules, etc.

[0177] The display screen is used to display interactive content such as the user's photos, videos, and operating status during the user's operation; this application does not limit the implementation method of the display screen, and it can be a 2D or 3D display screen based on technical principles such as LED (Light Emitting Diode), LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode Display).

[0178] The acoustic module includes acoustic devices and circuits such as microphone arrays and speakers to realize user voice command input and voice feedback of user operations.

[0179] The touch screen provides a second input method besides voice, which can realize text input and other operations by body touch; this application does not limit the implementation method of the touch screen, which can be capacitive, resistive or infrared, etc.

[0180] The remote control module implements a second input method besides voice. Text input and other operations can be implemented using a wired or wireless remote control. The remote control can be a button remote control, mouse, keyboard, etc.

[0181] The local device may also include a third input method besides voice, that is, using a camera or an infrared transceiver array, or an ultrasonic transceiver array to form a motion acquisition module to realize the user's body movement input.

[0182] The communication module enables the local device to communicate with the user's cloud storage server or other terminal in a wired or wireless manner; this application does not limit the wireless communication method, which can be 4G and other cellular network wireless communication, BT (Bluetooth), WiFi and other methods.

[0183] The power module includes an AC (Alternating Current) / DC (Direct Current) conversion circuit and other necessary voltage conversion circuits to provide appropriate power to other hardware modules; the power module may also include a rechargeable battery to enable mobile use of the local device.

[0184] The storage module includes devices such as RAM (Random Access Memory) and Flash (flash memory), which are used to store data and programs involved in the operation of the local device, as well as photos, videos and other content saved locally by users.

[0185] The processor is the core of the local device hardware architecture and is used for voice and image-related data processing, touch, remote control, body movement recognition, and controlling external communications of the product.

[0186] It should be noted that the present application does not limit the product form of the local device, which may be an electronic photo frame, a television, a tablet computer, a handheld terminal, and the like.

[0187] It should be noted that the local device has the functions of a general electronic photo frame, such as storage, sequential / random playback of photos or videos, timed power on and off, etc. Moreover, the local device also has the general functions of an intelligent assistant, such as information Q&A, control of intelligent devices, and access to Internet media resources.

[0188] The above embodiments can achieve the following technical effects: (1) provide a method and device for collecting natural language instructions such as voice, text and body movements; (2) provide a method for intelligently retrieving and editing photos or videos in original storage media such as cloud storage through a local device; (3) provide a method for displaying photos in the order corresponding to the user's intention, and a method for selecting an appropriate guidance method according to the user environment to realize visual operation.

[0189] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0190] Based on the same inventive concept, the embodiment of the present application also provides a photo processing device for implementing the photo processing method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more photo processing device embodiments provided below can refer to the limitations on the photo processing method above, and will not be repeated here.

[0191] In an exemplary embodiment, Figure 8 As shown, a photo processing device is provided, which is applied to a server, and includes: an instruction receiving module 810, an intention recognition module 820, an instruction conversion module 830, a photo screening module 840 and a photo returning module 850, wherein:

[0192] The instruction receiving module 810 is used to receive a photo retrieval instruction sent by a terminal; the photo retrieval instruction carries an account identifier.

[0193] The intention recognition module 820 is used to identify the first intention information corresponding to the photo retrieval instruction through the multimodal large model.

[0194] The instruction conversion module 830 is used to convert the photo retrieval instruction into a corresponding first computer instruction according to the first intention information.

[0195] The photo screening module 840 is used to screen target photos that meet the first intention information from the cloud storage photos corresponding to the account identifier according to the first computer instruction.

[0196] The photo returning module 850 is used to return the target photo to the terminal.

[0197] In an exemplary embodiment, the photo screening module 840 is also used to determine, according to the first computer instruction, a target photo tag associated with the first intent information and a target photo attribute associated with the first intent information; screen out candidate photos having corresponding photo tags and containing target photo tags from the cloud storage photos corresponding to the account identifier in the file management system; the file management system stores cloud storage photos, each cloud storage photo carrying a corresponding photo tag; and screen out photos corresponding to the target photo attributes from the candidate photos as target photos.

[0198] In an exemplary embodiment, the photo processing device also includes a label determination module, which is used to perform content recognition processing on the cloud storage photos corresponding to the account identifier in the file management system through a multimodal large model to obtain content information of the cloud storage photos; and determine the photo tags of the cloud storage photos based on the content information of the cloud storage photos.

[0199] In an exemplary embodiment, the photo processing device further includes a photo sorting module, which is used to sort the target photos to obtain sorted target photos, and determine first feedback information of the photo retrieval instruction according to the target photos; the first feedback information at least includes first reply information of the photo retrieval instruction, and first guidance information associated with the first reply information;

[0200] The photo return module 850 is also used to return the sorted target photos and the first feedback information to the terminal; the terminal is used to display thumbnails of the sorted target photos, and to display the first feedback information in a display manner that matches the photo retrieval instruction, and to perform corresponding photo playback processing in response to the photo playback instruction for the first feedback information.

[0201] In an exemplary embodiment, the photo sorting module is further used to determine the photo arrangement order corresponding to the first intent information; sort the target photos according to the photo arrangement order to obtain the sorted target photos.

[0202] In an exemplary embodiment, a terminal is deployed with a multimodal large model; the photo processing device also includes a photo editing module for receiving a second computer instruction sent by the terminal; the terminal is used to identify the second intention information corresponding to the photo editing instruction for the selected photo in the target photo through the multimodal large model, and convert the photo editing instruction into a corresponding second computer instruction according to the second intention information; according to the second computer instruction, the selected photo is edited to obtain an edited photo, and according to the edited photo, second feedback information of the photo editing instruction is determined; the second feedback information at least includes second reply information of the photo editing instruction, and second guidance information associated with the second reply information; the edited photo and the second feedback information are returned to the terminal; the terminal is used to display the edited photo, and display the second feedback information in a display method matching the photo editing instruction.

[0203] Each module in the above-mentioned photo processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0204] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as photos. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a photo processing method is implemented.

[0205] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0206] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0207] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0208] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0209] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0210] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0211] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A photo processing method, characterized in that: Applied to a server, the method comprises: A photo retrieval instruction sent by a receiving terminal; the photo retrieval instruction carries an account identifier; Identifying, by means of a multimodal large model, first intent information corresponding to the photo retrieval instruction; converting the photo retrieval instruction into a corresponding first computer instruction according to the first intention information; According to the first computer instruction, filter out target photos that meet the first intention information from the cloud storage photos corresponding to the account identifier; The target photo is returned to the terminal.

2. The method according to claim 1, characterized in that The step of screening, according to the first computer instruction, target photos that meet the first intention information from the cloud storage photos corresponding to the account identifier includes: Determine, according to the first computer instruction, a target photo tag associated with the first intention information and a target photo attribute associated with the first intention information; Filtering candidate photos whose corresponding photo tags also include the target photo tag from the cloud storage photos corresponding to the account identifier in the file management system; the cloud storage photos are stored in the file management system, and each of the cloud storage photos carries the corresponding photo tag; From the candidate photos, a photo corresponding to the attribute of the target photo is screened out as the target photo.

3. The method according to claim 2, characterized in that Before receiving the photo retrieval instruction sent by the terminal, it also includes: Using the multimodal large model, content recognition processing is performed on the cloud storage photos corresponding to the account identifier in the file management system to obtain content information of the cloud storage photos; Determine a photo tag of the cloud storage photo according to the content information of the cloud storage photo.

4. The method according to claim 1, characterized in that: After selecting the target photos that meet the first intention information, the method further includes: Sorting the target photos to obtain sorted target photos, and determining first feedback information of the photo retrieval instruction according to the target photos; the first feedback information at least includes first reply information of the photo retrieval instruction and first guidance information associated with the first reply information; The step of returning the target photo to the terminal includes: The sorted target photos and the first feedback information are returned to the terminal; the terminal is used to display thumbnails of the sorted target photos, and to display the first feedback information in a display manner that matches the photo retrieval instruction, and to perform corresponding photo playback processing in response to the photo playback instruction for the first feedback information.

5. The method according to claim 4, characterized in that The step of sorting the target photos to obtain sorted target photos includes: Determining an arrangement order of photos corresponding to the first intent information; The target photos are sorted according to the photo arrangement order to obtain the sorted target photos.

6. The method according to any one of claims 1 to 5, characterized in that: The terminal is deployed with the multimodal large model; After returning the target photo to the terminal, the method further includes: receiving a second computer instruction sent by the terminal; the terminal is used to identify, through the multimodal macromodel, second intention information corresponding to the photo editing instruction for the selected photo among the target photos, and convert the photo editing instruction into the corresponding second computer instruction according to the second intention information; According to the second computer instruction, the selected photo is edited to obtain an edited photo, and according to the edited photo, second feedback information of the photo editing instruction is determined; the second feedback information at least includes second reply information of the photo editing instruction and second guidance information associated with the second reply information; The edited photo and the second feedback information are returned to the terminal; the terminal is used to display the edited photo and the second feedback information in a display manner matching the photo editing instruction.

7. A photo processing device, characterized in that: Applied to a server, the device comprises: An instruction receiving module, used for receiving a photo retrieval instruction sent by a terminal; the photo retrieval instruction carries an account identifier; An intention recognition module, used to recognize first intention information corresponding to the photo retrieval instruction through a multimodal large model; an instruction conversion module, configured to convert the photo retrieval instruction into a corresponding first computer instruction according to the first intention information; a photo screening module, configured to screen target photos that meet the first intention information from the cloud storage photos corresponding to the account identifier according to the first computer instruction; The photo returning module is used to return the target photo to the terminal.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images