Image processing method, device, medium, electronic and product based on large model

Abstract

The present disclosure provides a large model-based image processing method, device, medium, electronic and product, relating to the technical field of image processing, which comprises: acquiring a target image and a target prompt word, the target prompt word being obtained by performing multi-round iterative optimization on an original prompt word according to a target large model and multiple optimization samples; wherein the optimization sample comprises a sample image and a specified processing result corresponding to the sample image; in the optimization process, the target large model is used to process the sample image and the original prompt word to obtain a predicted processing result, and an optimization summary is obtained according to historical data, the predicted processing result and the specified processing result, and the original prompt word is optimized according to the optimization summary; and a target processing result is obtained through the target large model according to the target image and the target prompt word. The automatic optimization of the prompt word can be realized, the accuracy of the obtained target prompt word can be improved, and a more accurate target processing result can be obtained.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, in particular, to a large model-based image processing method and device, medium, electronic product and product. BACKGROUND

[0002] Image processing is one of the core tasks in the field of computer vision, and is widely used in scenarios such as advertisement logo recognition and image abstract feature labeling.

[0003] In the related art, a special model can be trained through a large number of manually annotated samples to perform image processing. Although this method is effective, it has inherent defects such as high annotation cost and limited model generalization ability. For multi-modal large models, a prompt-based method such as a thinking chain provides a new paradigm for zero-sample or few-sample image processing. However, the prompt words in such multi-modal large models need to be manually optimized, and the optimization effect is poor, which is poor for the final performance improvement of the multi-modal large model. SUMMARY

[0004] This summary is provided to introduce a selection of concepts that are further described below in the detailed description. This summary is not intended to identify key or essential features of the claimed technology, nor is it intended to be used to limit the scope of the claimed technology.

[0005] In a first aspect, the present disclosure provides a large model-based image processing method, comprising:

[0006] obtaining a target image and a target prompt word, the target prompt word being obtained by performing multi-round iterative optimization on an original prompt word according to a target large model and a plurality of optimization samples; wherein the optimization samples include a sample image and a specified processing result corresponding to the sample image, in the optimization process, the target large model is used to process the sample image and the original prompt word to obtain a predicted processing result, and an optimization summary is obtained according to historical data, the predicted processing result and the specified processing result, and the original prompt word is optimized according to the optimization summary, the historical data includes the predicted processing result, the specified processing result and the optimization summary corresponding to each round of iterative optimization before the current round of iterative optimization;

[0007] obtaining a target processing result through the target large model according to the target image and the target prompt word.

[0008] In a second aspect, the present disclosure provides a large model-based image processing device, comprising:

[0009] An obtaining module is configured to obtain a target image and a target prompt word, the target prompt word being obtained by performing multi-round iterative optimization on an original prompt word according to a target large model and a plurality of optimization samples; wherein the optimization samples include a sample image and a specified processing result corresponding to the sample image, in an optimization process, the target large model is used to process the sample image and the original prompt word to obtain a predicted processing result, and an optimization summary is obtained according to historical data, the predicted processing result and the specified processing result, and the original prompt word is optimized according to the optimization summary, and the historical data includes the predicted processing result, the specified processing result and the optimization summary corresponding to each round of iterative optimization before the present round of iterative optimization;

[0010] A processing module is configured to obtain a target processing result by the target large model according to the target image and the target prompt word.

[0011] In a third aspect, the present disclosure provides a computer readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method of the first aspect.

[0012] In a fourth aspect, the present disclosure provides an electronic device, comprising:

[0013] A storage device having a computer program stored thereon;

[0014] A processing device for executing the computer program in the storage device to implement the steps of the method of the first aspect.

[0015] In a fifth aspect, the present disclosure provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method of the first aspect.

[0016] Based on the above technical scheme, the target image and the target prompt word are obtained, the target prompt word is obtained by iteratively optimizing the original prompt word according to the target large model and multiple optimization samples; wherein the optimization sample includes a sample image and a specified processing result corresponding to the sample image, in the optimization process, the target large model is used to process the sample image and the original prompt word to obtain a predicted processing result, and an optimization summary is obtained according to historical data, the predicted processing result and the specified processing result, and the original prompt word is optimized according to the optimization summary, the historical data includes the predicted processing result, the specified processing result and the optimization summary corresponding to each round of iterative optimization before the current round of iterative optimization; and the target processing result is obtained by the target large model according to the target image and the target prompt word. The sample image and the original prompt word can be processed to obtain the predicted processing result based on the target large model in advance, and the optimization summary is obtained according to the historical data, the predicted processing result and the specified processing result, and the original prompt word is optimized according to the optimization summary, so as to realize the automatic optimization of the prompt word, and the historical data includes the predicted processing result, the specified processing result and the optimization summary corresponding to each round of iterative optimization before the current round of iterative optimization. Through careful analysis based on the historical data, long-term performance degradation caused by short-term optimization can be effectively prevented, the accuracy of the obtained target prompt word can be improved, and the target large model can process the target image more accurately with the assistance of the target prompt word to obtain more accurate target processing result.

[0017] Other features and advantages of the present disclosure will be described in detail in the following detailed description section. BRIEF DESCRIPTION OF DRAWINGS

[0018] The above and other features, advantages and aspects of embodiments of the present disclosure will become more apparent by describing in detail some embodiments with reference to the attached drawings. The same or similar elements are denoted by the same or similar reference numerals throughout the drawings. It is to be understood that the drawings are schematic, and the original and elements are not necessarily drawn to scale. In the drawings:

[0019] Figure 1 is a flowchart of a large model-based image processing method according to some embodiments.

[0020] Figure 2 is a flowchart of obtaining a target prompt word according to some embodiments.

[0021] Figure 3 is an architecture diagram of obtaining a target prompt word according to some embodiments.

[0022] Figure 4 is a structural schematic diagram of a large model-based image processing device according to some embodiments.

[0023] Figure 5is a structural schematic diagram of an electronic device according to some embodiments. DETAILED DESCRIPTION

[0024] Embodiments of the present disclosure will be described in more detail with reference to the drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, but rather the embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are only for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0025] It should be understood that each step described in the method embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0026] The term "comprising" and variations thereof as used herein are open-ended, that is, "comprising but not limited to." The term "based on" is "based, at least in part, on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments." Related terms are defined in the following description.

[0027] It should be noted that the "first", "second", and the like concepts mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0028] It should be noted that the modification of "one", "multiple" mentioned in the present disclosure is illustrative and not limiting, and those skilled in the art should understand that unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0029] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0030] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the type, use range, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the consent of the user should be obtained in a proper manner according to relevant laws and regulations.

[0031] For example, in response to receiving an active request of a user, a prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed by the user will require obtaining and using personal information of the user. Thus, the user can autonomously select whether to provide personal information to the software or hardware, such as an electronic device, an application program, a server or a storage medium, performing the operation of the technical solution of the present disclosure according to the prompt information.

[0032] As an optional but non-limiting implementation, in response to receiving an active request of a user, the prompt information can be sent to the user in the form of a pop-up window, and the prompt information can be presented in the form of text in the pop-up window. In addition, the pop-up window can also carry selection controls for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0033] It can be understood that the above notification and obtaining of user consent process is only illustrative and does not limit the implementation of the present disclosure, and other ways that meet the relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0034] At the same time, it can be understood that the data (including but not limited to the data itself, the acquisition or use of the data) involved in the technical solution should comply with the requirements of the relevant laws and regulations and the relevant provisions.

[0035] Figure 1 is a flowchart of a large model-based image processing method according to some embodiments, as shown in Figure 1 The present embodiment provides a large model-based image processing method, which can be specifically executed by a large model-based image processing device. As shown in Figure 1 The method can include the following steps.

[0036] In step S110, a target image and a target prompt word are obtained, and the target prompt word is obtained by performing multi-round iterative optimization on an original prompt word according to a target large model and a plurality of optimization samples; wherein the optimization sample includes a sample image and a specified processing result corresponding to the sample image, in the optimization process, the target large model is used to process the sample image and the original prompt word to obtain a predicted processing result, and an optimization summary is obtained according to historical data, the predicted processing result and the specified processing result, and the original prompt word is optimized according to the optimization summary, and the historical data includes the predicted processing result, the specified processing result and the optimization summary corresponding to each round of iterative optimization before the present round of iterative optimization.

[0037] In this embodiment, the large model-based image processing method can be applied to the following scenarios, including a specific character recognition task in an image, an image rendering effect classification and evaluation task, and an image erasing failure recognition task. The target image can be an image to be processed. The target large model can be a multi-modal large model with image processing capability. The target large model can process the image in combination with a prompt word to obtain a desired processing result.

[0038] When processing the target prompt word using the target large model, the original prompt word can be automatically optimized by the target large model and a plurality of optimization samples to obtain a more accurate target prompt word. The optimization sample can include a sample image and a specified processing result corresponding to the sample image. The specified processing result corresponding to the sample image is an accurate processing result obtained by processing the sample image. The sample image can be labeled to obtain the specified processing result.

[0039] In the process of optimizing the original prompt word to obtain the target prompt word, the sample image and the original prompt word can be processed by the target large model to obtain a predicted processing result. An optimization summary can be obtained according to historical data, the predicted processing result, and the specified processing result. The original prompt word can be optimized according to the optimization summary to obtain the target prompt word. The historical data includes the predicted processing result, the specified processing result, and the optimization summary corresponding to each round of iterative optimization before the current round of iterative optimization.

[0040] In step S120, a target processing result is obtained by the target large model according to the target image and the target prompt word.

[0041] In this embodiment, after the target prompt word is optimized, the target image and the target prompt word can be input into the target large model. The target large model processes the target image based on the target prompt word, thereby outputting a more accurate target processing result by the target large model.

[0042] Through the above method, the sample image and the original prompt word can be processed by the target large model to obtain a predicted processing result in advance. An optimization summary can be obtained according to historical data, the predicted processing result, and the specified processing result. The original prompt word can be optimized according to the optimization summary. Thus, the automatic optimization of the prompt word can be realized. The historical data includes the predicted processing result, the specified processing result, and the optimization summary corresponding to each round of iterative optimization before the current round of iterative optimization. Through careful analysis of the historical data, long-term performance degradation caused by short-term optimization can be effectively prevented. The accuracy of the obtained target prompt word can be improved. With the assistance of the target prompt word, the target large model can process the target image more accurately to obtain a more accurate target processing result.

[0043] In some possible implementation manners, the target large model comprises a teacher large model and a student large model, and in the optimization process, the student large model is used to process the sample image and the original prompt word to obtain a predicted processing result, and the teacher large model is used to obtain an optimized summary according to historical data, the predicted processing result and a specified processing result, and to optimize the original prompt word according to the optimized summary.

[0044] According to the target image and the target prompt word, a target processing result is obtained through the target large model, comprising:

[0045] According to the target image and the target prompt word, a target processing result is obtained through a student large model in the target large model.

[0046] In the embodiment, the target large model can comprise a teacher large model and a student large model, wherein the student large model is an executor of an image processing task, and its base model can be GPT4o, which has strong image recognition capability and can obtain a predicted processing result based on a sample image and an original prompt word according to the guidance of the original prompt word. The original prompt word can be an initial original prompt word, and if there are multiple rounds of iterative optimization in the optimization process, the original prompt word can be an original prompt word after optimization of the teacher large model. The teacher large model is the brain and guide of the system, and its base model can be Gemini-2.5-Pro, which not only has image recognition capability but also has strong logical reasoning capability. The teacher large model can receive a sample image and a specified processing result corresponding to the sample image, and feedback information of the student large model, i.e., a predicted processing result, to accurately evaluate the prediction result of the student large model, and further optimize the original prompt word to obtain a target prompt word. The teacher large model can be bidirectionally synchronized with a memory module, and the teacher large model can accumulate and utilize past experience to make better decisions, i.e., obtain a more accurate target prompt word according to historical data, so as to obtain a more accurate target processing result.

[0047] In some possible implementation manners, the target prompt word can be obtained through the following steps:

[0048] An original prompt word and multiple optimization samples are obtained, the original prompt word is iteratively optimized through the target large model and the multiple optimization samples, and when a preset stop condition is met, the iterative optimization is stopped, and the original prompt word after the last optimization is determined as the target prompt word.

[0049] In the embodiment, the original prompt can be iteratively optimized by the target large model based on multiple optimization samples, so as to further improve the accuracy of the obtained target prompt. The preset stopping condition can be that the original prompt no longer changes, or the number of iterations reaches a preset number. Each iteration training can optimize the current original prompt, and when the preset stopping condition is met, the iteration optimization of the original prompt can be stopped, and the original prompt after the last optimization is determined as the target prompt. By referring to the traditional deep model training process, a multi-round iteration training mechanism is designed, which can fully utilize the supervised samples, that is, the optimization samples, to deeply learn the business knowledge and obtain a more accurate target prompt.

[0050] Figure 2 FIG. 1 is a flowchart of optimizing a target prompt according to some embodiments, which can include the following steps: Figure 2

[0051] In step S210, the optimization sample is obtained.

[0052] In step S220, it is determined whether the preset stopping condition is met. If yes, step S250 is performed, and if no, step S230 is performed.

[0053] In step S230, the optimization sample processing is performed based on the teacher large model and the student large model.

[0054] In step S240, the original prompt is optimized.

[0055] In step S250, the original prompt after the last optimization is determined as the target prompt.

[0056] Figure 3 FIG. 2 is an architecture diagram of optimizing a target prompt according to some embodiments, as shown in the figure, in some possible implementations, the original prompt is iteratively optimized by the target large model and multiple optimization samples, including: Figure 3

[0057] For any round of iterative optimization, the multiple optimization samples are randomly divided into multiple batches of optimization samples, and the original prompt is iteratively optimized by the target large model and the multiple batches of optimization samples.

[0058] In the embodiment, the entire optimization process can include multiple rounds of iterative optimization. In order to reduce the number of optimization samples, the multiple optimization samples of each round of iterative optimization can be the same. In each round of iterative optimization, the multiple optimization samples can be shuffled, and the multiple optimization samples can be randomly divided into multiple batches of optimization samples, and then the optimization samples can be input into the target large model through the multiple batches, so as to iteratively optimize the original prompt by the target large model. ​​

[0059] In some possible implementation manners, the target large model can include a teacher large model and a student large model.

[0060] Through the target large model and the plurality of optimization samples, the original prompt word is iteratively optimized in multiple rounds, including:

[0061] For any round of iterative optimization, the current original prompt word and the sample image corresponding to the current round of iterative optimization are input into the student large model to obtain a predicted processing result corresponding to the current round of iterative optimization; the historical data, the predicted processing result corresponding to the current round of iterative optimization, and the specified processing result are input into the teacher large model to obtain an optimization summary corresponding to the current round of iterative optimization; and the current original prompt word is optimized according to the optimization summary corresponding to the current round of iterative optimization.

[0062] In the embodiment, the target large model can include a teacher large model and a student large model, and the current original prompt word can be the original prompt word after the original prompt word is optimized in the last round of iterative optimization. For any round of iterative optimization, the current original prompt word and the sample image corresponding to the current round of iterative optimization can be input into the student large model, the student large model processes the sample image based on the original prompt word to obtain a predicted processing result corresponding to the current round of iterative optimization. Then, the historical data corresponding to the current round of iterative optimization can be obtained, and the historical data, the predicted processing result corresponding to the current round of iterative optimization, and the specified processing result can be input into the teacher large model. The teacher large model evaluates the processing capability of the student large model under the current original prompt word through the gap between the predicted processing result and the specified processing result, and obtains an optimization summary corresponding to the current round of iterative optimization in combination with the historical data, and then optimizes the current original prompt word according to the optimization summary corresponding to the current round of iterative optimization, so as to obtain a more accurate original prompt word. The teacher large model can be bidirectionally synchronized with a memory module, and the teacher large model can accumulate and utilize past experience to make a better decision, that is, according to the historical data and the predicted processing result corresponding to the current round of iterative optimization and the specified processing result, a more accurate optimization summary is obtained, so that a more accurate target prompt word can be obtained through optimization. Through the collaborative workflow between the student large model and the teacher large model, dynamic adjustment and optimization of the original prompt word are realized.

[0063] The whole process forms an intelligent closed loop of "execution-feedback-optimization", mainly including the following two steps:

[0064] The first step: prompt word driven image processing:

[0065] The student large model can execute an image processing task according to the current original prompt word and the sample image, and then output a predicted processing result. To ensure the stability of the prediction behavior, the temperature of the student large model is usually set to 0, that is, the student large model remains stable and unchanged.

[0066] Second step: real-time optimization of the original prompt word:

[0067] The prediction processing result of the student large model is fed back to the teacher large model as feedback. At this time, the teacher large model starts its core internal decision-making process to finely adjust the current original prompt word in order to complete the optimization of the current original prompt word.

[0068] In one possible implementation, the historical data and the prediction processing result and the specified processing result corresponding to the current iteration optimization are input into the teacher large model to obtain an optimization summary corresponding to the current iteration optimization, including:

[0069] The teacher large model obtains an original summary corresponding to the current iteration optimization according to the prediction processing result and the specified processing result corresponding to the current iteration optimization, and the original summary includes error cause analysis and modification details; and the teacher large model obtains an optimization summary corresponding to the current iteration optimization according to the original summary and the historical data.

[0070] In this embodiment, the teacher large model can compare the prediction processing result and the specified processing result according to the prediction processing result and the specified processing result corresponding to the current iteration optimization, determine the gap between the two, evaluate the performance of the student large model in image processing under the current original prompt word, and reflect on the possible defects of the current original prompt word, thereby obtaining an original summary corresponding to the current iteration optimization, which can include error cause analysis of the prediction processing result and modification details for the current original prompt word. Before optimizing the current original prompt word, historical data can be obtained for further adjustment of the original summary to obtain a more accurate optimization summary corresponding to the current iteration optimization.

[0071] In some possible implementations, the teacher large model obtains an optimization summary corresponding to the current iteration optimization according to the original summary and the historical data, including:

[0072] The teacher large model analyzes the original summary and the historical data to obtain an analysis result, and adjusts the original summary according to the analysis result to obtain an optimization summary corresponding to the current iteration optimization.

[0073] In the embodiment, the teacher large model can further analyze the historical data on the basis of the original summary, and can think of multiple preset questions, which can include: whether such errors have occurred in history? Can the modification of the current original prompt achieve the expected effect? Will the modification of the current original prompt introduce new problems or cause image processing performance degradation? Thus, an analysis result is obtained. Through careful analysis based on the historical data, the teacher large model can effectively prevent long-term performance degradation caused by short-term optimization. Finally, the original summary can be adjusted according to the analysis result to obtain an optimized summary corresponding to the current round of iterative optimization, so that the final and more robust modification of the current original prompt can be made based on the optimized summary to obtain a more accurate target prompt.

[0074] In some possible embodiments, the method further includes:

[0075] For any round of iterative optimization, the predicted processing result, the specified processing result and the optimized summary corresponding to the round of iterative optimization are stored in the memory module; and for any round of iterative optimization, the historical data is obtained from the memory module.

[0076] In the embodiment, for any round of iterative optimization, after the current original prompt is optimized, the complete teaching event of the current round of iterative optimization is stored in the memory module as a snapshot, that is, the predicted processing result, the specified processing result and the optimized summary corresponding to the round of iterative optimization are stored in the memory module, so as to accumulate and deposit knowledge, and provide more abundant experience for future guidance, so as to obtain a more accurate optimized summary based on the historical data.

[0077] Figure 4 is a structural schematic diagram of a large model-based image processing device according to some embodiments. As Figure 4 shown, the disclosure embodiment provides a large model-based image processing device 400, which comprises:

[0078] The acquisition module 401 is configured to acquire a target image and a target prompt, wherein the target prompt is obtained by multiple rounds of iterative optimization of an original prompt by a target large model and multiple optimization samples; wherein the optimization samples include a sample image and a specified processing result corresponding to the sample image, in the optimization process, the target large model is used to process the sample image and the original prompt to obtain a predicted processing result, and an optimized summary is obtained according to historical data, the predicted processing result and the specified processing result, and the original prompt is optimized according to the optimized summary, and the historical data includes the predicted processing result, the specified processing result and the optimized summary corresponding to each round of iterative optimization before the current round of iterative optimization;

[0079] The processing module 402 is configured to obtain a target processing result by the target large model according to the target image and the target prompt word.

[0080] In some possible implementation manners, the target large model includes a teacher large model and a student large model, and in the optimization process, the student large model is used to process the sample image and the original prompt word to obtain a predicted processing result, the teacher large model is used to obtain an optimized summary according to the historical data, the predicted processing result and the specified processing result, and the original prompt word is optimized according to the optimized summary.

[0081] The processing module 402 is configured to:

[0082] obtain the target processing result by the student large model in the target large model according to the target image and the target prompt word.

[0083] In some possible implementation manners, the obtaining module 401 is configured to:

[0084] obtain the original prompt word and the plurality of optimization samples.

[0085] perform multi-round iterative optimization on the original prompt word by the target large model and the plurality of optimization samples.

[0086] In a case where a preset stop condition is met, stop the iterative optimization, and determine the original prompt word after the last optimization as the target prompt word.

[0087] In some possible implementation manners, the obtaining module 401 is configured to:

[0088] for any round of iterative optimization, randomly divide the plurality of optimization samples into a plurality of batches of optimization samples;

[0089] perform iterative optimization on the original prompt word by the target large model and the plurality of batches of optimization samples.

[0090] In some possible implementation manners, the target large model includes a teacher large model and a student large model.

[0091] The obtaining module 401 is configured to:

[0092] for any round of iterative optimization, input the current original prompt word and the sample image corresponding to the current round of iterative optimization into the student large model to obtain a predicted processing result corresponding to the current round of iterative optimization.

[0093] input the historical data, the prediction processing result corresponding to the current round of iterative optimization, and the specified processing result into the teacher large model to obtain an optimized summary corresponding to the current round of iterative optimization.

[0094] According to the optimized summary corresponding to the current round of iterative optimization, the original prompt word is optimized.

[0095] In some possible implementation manners, the obtaining module 401 is configured to:

[0096] The teacher large model obtains an original summary corresponding to the current round of iterative optimization according to the prediction processing result and the specified processing result corresponding to the current round of iterative optimization, and the original summary includes error cause analysis and modification details.

[0097] The teacher large model obtains an optimized summary corresponding to the current round of iterative optimization according to the original summary and the historical data.

[0098] In some possible implementation manners, the obtaining module 401 is configured to:

[0099] The teacher large model obtains an analysis result by performing analysis based on the original summary and the historical data.

[0100] According to the analysis result, the original summary is adjusted to obtain an optimized summary corresponding to the current round of iterative optimization.

[0101] In some possible implementation manners, the obtaining module 401 is configured to:

[0102] For any round of iterative optimization, the prediction processing result, the specified processing result, and the optimized summary corresponding to the round of iterative optimization are stored in a memory module.

[0103] For any round of iterative optimization, the historical data is obtained from the memory module.

[0104] The function logic performed by each functional module in the above-described image processing apparatus 400 based on a large model has been described in detail in the method part, and will not be described here again.

[0105] Reference will be made below to Figure 5 which shows a structural schematic diagram of an electronic device (for example, a terminal device or a server) 500 suitable for use to implement embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure can include, but is not limited to, a mobile terminal such as a head-mounted device, a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a vehicle-mounted terminal (for example, a vehicle-mounted navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. Figure 5The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0106] like Figure 5 As shown, electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. RAM 503 also stores various programs and data required for the operation of electronic device 500. Processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0107] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0108] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0109] It is noted that the aforementioned computer-readable medium of the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example and without limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a computer-readable program code transmitted by a computer-readable medium or a carrier wave in a baseband or as part of a carrier wave. Such a propagated computer-readable signal medium can take many forms, including but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. The computer-readable signal medium can also be any computer-readable medium that is not a computer-readable storage medium and that can be used to carry or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained in the computer-readable medium can be transmitted by any suitable medium, including but not limited to, wire, cable, RF (radio frequency), or the like, or any suitable combination of the foregoing.

[0110] In some embodiments, the quality assurance system and the business system can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communications (e.g., a communications network) of any form or medium (e.g., wireline, wireless, etc.). Examples of communications networks include local area networks ("LANs"), wide area networks ("WANs"), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

[0111] The aforementioned computer-readable medium can be included in the aforementioned electronic device; or can exist separately from the electronic device and can be accessed via the electronic device.

[0112] The computer readable medium described above carries one or more programs, when the one or more programs are executed by the electronic device, cause the electronic device to: obtain a target image and a target prompt word, the target prompt word being obtained by performing multi-round iterative optimization on an original prompt word according to a target large model and a plurality of optimization samples; wherein the optimization samples include a sample image and a specified processing result corresponding to the sample image, in the optimization process, the target large model is used to process the sample image and the original prompt word to obtain a predicted processing result, and an optimization summary is obtained according to historical data, the predicted processing result and the specified processing result, and the original prompt word is optimized according to the optimization summary, the historical data includes the predicted processing result, the specified processing result and the optimization summary corresponding to each round of iterative optimization before the current round of iterative optimization; and obtain a target processing result by the target large model according to the target image and the target prompt word.

[0113] Computer program code for carrying out operations of the present disclosure can be written in any of one or more programming languages or combinations of languages including object or visual programming languages such as Java, Smalltalk, C++ or conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0114] The flow and block diagrams in the drawings show architectural, functional, and operational aspects of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may be executed in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0115] The modules involved in the embodiments of the present disclosure can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0116] The functions described above in the present disclosure can be performed at least in part by one or more hardware logic components. For example, non-limiting example types of hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0117] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable storage media can include, without limitation, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media can include, but are not limited to, one or more lines of electrical wire, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0118] The above description is merely the preferred embodiments of the present disclosure and the explanation of the principles of the applied technology. It should be understood by those skilled in the art that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or their equivalent features without departing from the disclosed concept. For example, the above technical features can be replaced with the technical features disclosed in the present disclosure (but not limited to) having similar functions to form technical solutions.

[0119] Moreover, while operations have been depicted in a particular order, this should not be understood as requiring such an order nor limiting it to only those operations shown and described. One of ordinary skill in the art will recognize that many of the operations can be performed in a differing order, or be performed concurrently, that some operations can be performed in any order or omitted, and that some operations can be performed in parallel. Similarly, while several specific implementation details have been included herein, these should not be taken as limitations on the scope of the present disclosure. Certain features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination. Moreover, although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended definitions is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the subject matter defined in the appended definitions.

[0120] While the subject matter has been described above in the general context of "structural features" and "methodological acts" that can be performed by an apparatus, it is to be understood that the subject matter defined in the appended definitions can be implemented in any manner found to be within the scope of the appended definitions. For example, the subject matter described above can be implemented in hardware, software, firmware, or any combination thereof. As such, the subject matter can take many different forms that are not expressly mentioned above.

Claims

1. A large model-based image processing method, characterized in that, The method comprises: obtaining a target image and a target prompt word, the target prompt word being obtained by performing multi-round iterative optimization on an original prompt word according to a target large model and a plurality of optimization samples; wherein the optimization samples comprise a sample image and a specified processing result corresponding to the sample image; in the optimization process, the target large model is used to process the sample image and the original prompt word to obtain a predicted processing result, and according to the gap between the predicted processing result and the specified processing result corresponding to the current round of iterative optimization, the defects of the current original prompt word are reflected, so as to obtain an original summary corresponding to the current round of iterative optimization, and on the basis of the original summary, historical data and a plurality of preset problems are considered to obtain an optimization summary, and the original prompt word is optimized according to the optimization summary; the historical data comprises the predicted processing result, the specified processing result and the optimization summary corresponding to each round of iterative optimization before the current round of iterative optimization, the specified processing result corresponding to the sample image is an accurate processing result obtained by processing the sample image, and the original summary comprises error cause analysis and modification details; obtaining a target processing result by the target large model according to the target image and the target prompt word.

2. The large model-based image processing method of claim 1, wherein, The target large model comprises a teacher large model and a student large model; in the optimization process, the student large model is used to process the sample image and the original prompt word to obtain a predicted processing result, and the teacher large model is used to obtain an optimization summary according to the historical data, the predicted processing result and the specified processing result, and optimize the original prompt word according to the optimization summary; The target processing result is obtained by the student large model in the target large model according to the target image and the target prompt word. The target prompt word is obtained by the following steps:

3. The large model-based image processing method of claim 1, wherein, obtaining the original prompt word and the plurality of optimization samples; performing multi-round iterative optimization on the original prompt word by the target large model and the plurality of optimization samples; stopping the iterative optimization and determining the original prompt word after the last optimization as the target prompt word when a preset stop condition is met.

4. The large model-based image processing method according to claim 3, wherein The multi-round iterative optimization on the original prompt word by the target large model and the plurality of optimization samples comprises: for any round of iterative optimization, the plurality of optimization samples are randomly divided into a plurality of batches of optimization samples; the iterative optimization on the original prompt word by the target large model and the plurality of batches of optimization samples. The target large model comprises a teacher large model and a student large model; 5. The large model-based image processing method according to claim 3, characterized in that, The multi-round iterative optimization on the original prompt word by the target large model and the plurality of optimization samples comprises: for any round of iterative optimization, the current original prompt word and the sample image corresponding to the current round of iterative optimization are input into the student large model to obtain a predicted processing result corresponding to the current round of iterative optimization; ​ input the historical data, the prediction processing result corresponding to the current round of iterative optimization, and the specified processing result into the teacher large model to obtain an optimized summary corresponding to the current round of iterative optimization; According to the optimized summary corresponding to the current round of iterative optimization, the current original prompt word is optimized.

6. The large model-based image processing method of claim 5, characterized in that, the inputting of the historical data, the prediction processing result corresponding to the current round of iterative optimization, and the specified processing result into the teacher large model to obtain an optimized summary corresponding to the current round of iterative optimization comprises: the teacher large model obtains an original summary corresponding to the current round of iterative optimization according to the prediction processing result and the specified processing result corresponding to the current round of iterative optimization; the teacher large model obtains an optimized summary corresponding to the current round of iterative optimization according to the original summary and the historical data.

7. The large model-based image processing method of claim 6, characterized in that, the obtaining of the optimized summary corresponding to the current round of iterative optimization by the teacher large model according to the original summary and the historical data comprises: the teacher large model analyzes the original summary and the historical data to obtain an analysis result; the original summary is adjusted according to the analysis result to obtain an optimized summary corresponding to the current round of iterative optimization.

8. The large model-based image processing method according to any one of claims 1-7, characterized in that, The method further comprises: for any round of iterative optimization, the prediction processing result, the specified processing result, and the optimized summary corresponding to the round of iterative optimization are stored in a memory module; for any round of iterative optimization, the historical data is obtained from the memory module.

9. A large model-based image processing apparatus, characterized by comprising: comprises: an acquisition module configured to acquire a target image and a target prompt word, the target prompt word being obtained by multiple rounds of iterative optimization of an original prompt word by a target large model and multiple optimization samples; wherein the optimization samples comprise a sample image and a specified processing result corresponding to the sample image, in the optimization process, the target large model is used to process the sample image and the original prompt word to obtain a prediction processing result, and according to the gap between the prediction processing result and the specified processing result corresponding to the current round of iterative optimization, the defects existing in the current original prompt word are reflected, thereby obtaining an original summary corresponding to the current round of iterative optimization, and on the basis of the original summary, historical data is combined and multiple preset problems are considered to obtain an optimized summary, and the original prompt word is optimized according to the optimized summary, the historical data comprises the prediction processing result, the specified processing result, and the optimized summary corresponding to each round of iterative optimization before the current round of iterative optimization, the specified processing result corresponding to the sample image is an accurate processing result obtained by processing the sample image, and the original summary comprises error cause analysis and modification details; a processing module configured to obtain a target processing result by the target large model according to the target image and the target prompt word.

10. A computer readable medium having stored thereon a computer program, characterized in that, The computer program is executed by a processing device to implement the steps of the method of any one of claims 1-8.

11. An electronic device, comprising: comprises: a storage device having a computer program stored thereon; processing means for executing the computer program in the storage means to implement the steps of the method of any of claims 1-8.

12. A computer program product comprising a computer program, characterized in that, The computer program which, when executed by the processor, carries out the steps of the method of any of claims 1-8.

Images