Picture material deconstruction method and related device

Abstract

The invention discloses a picture material deconstruction method and a related device, and relates to the technical field of picture processing, the picture material deconstruction method comprises the following steps: obtaining target data, the target data comprising a picture material; performing label analysis on the picture material by adopting one or more label analysis strategies in label analysis strategies respectively corresponding to a plurality of preset label sets to obtain label analysis results of the picture material on labels contained in the corresponding label sets, the plurality of label sets being obtained by dividing a plurality of predefined labels according to categories, and the label analysis results being obtained by dividing the plurality of labels according to categories; the label analysis strategy corresponding to any label set can analyze labels in the corresponding label set for the picture materials; and formatting the obtained label analysis result to obtain label information of the picture material. The picture material deconstruction method disclosed by the invention has relatively high efficiency, and relatively accurate and comprehensive label information can be obtained.

Description

Technical Field

[0001] This application relates to the technical field of picture material processing, and in particular, to a method for deconstructing picture materials and related devices. Background Art

[0002] In many application scenarios (such as digital content management, media analysis, precise advertising placement, etc.), it is necessary to label picture materials.

[0003] Currently, most methods for labeling picture materials are manual deconstruction methods, that is, manually deconstructing picture materials to determine the label information of the picture materials.

[0004] However, the manual deconstruction method is time-consuming and laborious, and moreover, it is easily affected by subjective factors, making it difficult to ensure the accuracy of label information. Summary of the Invention

[0005] In view of this, this application provides a method for deconstructing picture materials and related devices to solve the problems of the existing manual deconstruction method being time-consuming and laborious and difficult to ensure the accuracy of label information. The technical solutions are as follows:

[0006] The first aspect of this application provides a method for deconstructing picture materials, including:

[0007] Obtain target data, where the target data includes picture materials;

[0008] Adopt one or more of the label analysis strategies corresponding to a plurality of preset label sets to perform label analysis on the picture materials, and obtain label analysis results of the picture materials on the labels included in the corresponding label sets. Among them, the plurality of label sets are obtained by classifying a predefined plurality of labels, and the label analysis strategy corresponding to any label set can analyze the labels in the corresponding label set for the picture materials;

[0009] Perform formatting processing on the obtained label analysis results to obtain the label information of the picture materials.

[0010] In a possible implementation manner, the target data further includes specified labels;

[0011] The step of adopting one or more of the label analysis strategies corresponding to a plurality of preset label sets to perform label analysis on the picture materials and obtain label analysis results of the picture materials on the labels included in the corresponding label sets includes:

[0012] Determine the label set to which the specified label belongs from the plurality of label sets;

[0013] Using the label analysis strategy corresponding to the label set to which the specified label belongs, perform label analysis on the picture material to obtain the label analysis result of the picture material on the labels included in the corresponding label set.

[0014] In a possible implementation, the label analysis strategies respectively corresponding to the multiple label sets include a label analysis strategy based on a large model and a supplementary analysis strategy;

[0015] Among them, the supplementary analysis strategy includes one or more of the following label analysis strategies: a label analysis strategy based on object recognition, a label analysis strategy based on text recognition, and a label analysis strategy combined with a knowledge database.

[0016] In a possible implementation, using the label analysis strategy based on a large model to perform label analysis on the picture material to obtain the label analysis result of the picture material on the labels included in the corresponding label set includes:

[0017] Using a large model to analyze the labels included in the first label set for the picture material to obtain the label analysis result of the picture material on the labels included in the first label set, where the first label set is the label set corresponding to the label analysis strategy based on a large model.

[0018] In a possible implementation, using the label analysis strategy based on object recognition to perform label analysis on the picture material to obtain the label analysis result of the picture material on the labels included in the corresponding label set includes: inputting the picture material into a pre-constructed object recognition model to obtain the label analysis result of the picture material on the labels included in the second label set output by the object recognition model, where the second label set is the label set corresponding to the label analysis strategy based on object recognition.

[0019] In a possible implementation, using the label analysis strategy based on text recognition to perform label analysis on the picture material to obtain the label analysis result of the picture material on the labels included in the corresponding label set includes:

[0020] Inputting the picture material into a pre-constructed text recognition model to obtain the text recognition result of the picture material;

[0021] Using a large model to analyze the labels related to the text content in the third label set for the text recognition result of the picture material to obtain the label analysis result of the picture material on the labels related to the text content.

[0022] In a possible implementation, the label analysis strategy combined with the knowledge database is adopted to perform label analysis on the picture material, and the label analysis result of the picture material on the labels included in the corresponding label set is obtained, including:

[0023] Obtain knowledge data related to the picture material from the knowledge database;

[0024] Using a large model and referring to the knowledge data related to the picture material, analyze the labels included in the fourth label set for the picture material to obtain the label analysis result of the picture material on the labels included in the fourth label set, where the fourth label set is the label set corresponding to the label analysis strategy combined with the knowledge database.

[0025] In a possible implementation, the formatting process of the obtained label analysis result to obtain the label information of the picture material includes:

[0026] Using a large model, format the obtained label analysis result according to preset formatting requirements to obtain the label information of the picture material.

[0027] The second aspect of the present application provides a picture material deconstruction device, including: a data acquisition module, a label analysis module, and a formatting process module;

[0028] The data acquisition module is used to acquire target data, where the target data includes picture materials;

[0029] The label analysis module is used to adopt one or more label analysis strategies corresponding to a plurality of preset label sets respectively to perform label analysis on the picture material, and obtain the label analysis result of the picture material on the labels included in the corresponding label set, where the plurality of label sets are obtained by classifying a plurality of predefined labels, and the label analysis strategy corresponding to any label set can analyze the labels in the corresponding label set for the picture material;

[0030] The formatting process module is used to format the obtained label analysis result to obtain the label information of the picture material.

[0031] The third aspect of the present application provides an electronic device, including at least one processor and a memory connected to the processor, where:

[0032] The memory is used to store computer programs;

[0033] The processor is used to execute the computer program so that the electronic device can implement the steps of any one of the above-mentioned picture material deconstruction methods.

[0034] In a fourth aspect of the present application, a computer storage medium is provided. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of any one of the above-mentioned picture material deconstruction methods.

[0035] In a fifth aspect of the present application, a computer program product is provided, including computer-readable instructions. When the computer-readable instructions run on an electronic device, the electronic device can implement the steps of any one of the above-mentioned picture material deconstruction methods.

[0036] By means of the above technical solutions, for the picture material deconstruction method provided by the present application, after obtaining the picture material, one or more of the label analysis strategies corresponding to a plurality of preset label sets can be used to perform label analysis on the picture material, so as to obtain the label analysis results of the picture material on the labels included in the corresponding label set. After obtaining the label analysis results, the obtained label analysis results can be formatted, and finally the label information of the picture material can be obtained. The picture material deconstruction method provided by the present application can automatically deconstruct the picture material. Compared with the manual deconstruction method, the deconstruction time is greatly shortened and the deconstruction efficiency is greatly improved. Since the deconstruction process does not require manual participation, the influence of subjective human factors is avoided, and the accuracy of the deconstruction result can be ensured. In addition, the picture material deconstruction method provided by the present application can use a variety of label analysis strategies to perform label analysis on the picture material. Since each label analysis strategy can analyze a type of label, multiple types of labels can be obtained through the picture material deconstruction method provided by the present application. It can be seen that the picture material deconstruction method provided by the present application can deconstruct the picture material more comprehensively. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0038] Figure 1 It is a schematic diagram of a system architecture related to the present application;

[0039] Figure 2 It is a schematic diagram of a hardware structure of a terminal provided by an embodiment of the present application;

[0040] Figure 3 It is a schematic diagram of a hardware structure of a server provided by an embodiment of the present application;

[0041] Figure 4Schematic flowchart of the picture material deconstruction method provided by the embodiments of the present application;

[0042] Figure 5 Schematic diagram of the picture material deconstruction process provided by the embodiments of the present application;

[0043] Figure 6 Schematic diagram of the structure of the picture material deconstruction device provided by the embodiments of the present application. Detailed implementation manners

[0044] The embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. The terms used in the embodiments of the present application are only used to explain the specific embodiments of the present application, rather than intended to limit the present application.

[0045] The embodiments of the present application will be described below with reference to the accompanying drawings. Those skilled in the art know that with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0046] The terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances, which is only a way of distinguishing objects with the same attributes when describing the embodiments of the present application. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion, so that a process, method, system, product or device including a series of units does not have to be limited to those units, but may include other units not clearly listed or inherent to these process, method, product or device.

[0047] In a possible implementation manner, as Figure 1 shown, the system architecture involved in the present application may include a terminal 101 and a server 102. The terminal 101 can interact with the server 102 through a network (wired network or wireless network). Among them, the server 102 may include one or more servers ( Figure 1 illustrated by including one server as an example). The terminal can obtain picture materials, transmit the picture materials to the server through the network, and the server uses the picture material deconstruction method provided by the present application to deconstruct the picture materials to obtain the label information of the picture materials.

[0048] In another possible implementation manner, the system architecture involved in the present application may include a terminal. The terminal has strong data processing capabilities. The terminal can obtain picture materials and then use the picture material deconstruction method provided by the present application to deconstruct the picture materials to obtain the label information of the picture materials.

[0049] Next, the product form of the above terminal will be described.

[0050] The above terminal can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, a robot, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The embodiments of the present application do not impose any restrictions on this.

[0051] Figure 2 FIG. shows a schematic diagram of an optional hardware structure of the terminal.

[0052] Reference Figure 2 As shown, the terminal may include a radio frequency unit 210, a memory 220, an input unit 230, a display unit 240, a camera 250 (optional), an audio circuit 260 (optional), a speaker 261 (optional), a microphone 262 (optional), a headphone jack 263 (optional), a processor 270, an external interface 280, a power supply 290, and other components. Those skilled in the art can understand that Figure 2 This is merely an example of the terminal and does not constitute a limitation on the terminal. It may include more or fewer components than shown in the figure, or combine certain components, or different components.

[0053] The input unit 230 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the terminal. Specifically, the input unit 230 may include a touch screen 231 (optional) and / or other input devices 232. The touch screen 231 can collect touch operations of the user on or near it (such as operations of the user using a finger, a joint, a stylus, or any suitable object on or near the touch screen), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the touch action of the user on the touch screen, convert the touch action into a touch signal and send it to the processor 270, and can receive commands sent by the processor 270 and execute them; the touch signal at least includes contact coordinate information. The touch screen 231 can provide an input interface and an output interface between the terminal and the user. In addition, various types such as resistive, capacitive, infrared, and surface acoustic waves can be used to implement the touch screen. In addition to the touch screen 231, the input unit 230 may further include other input devices. Specifically, the other input devices 232 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0054] The display unit 240 can be used to display information input by the user or information provided to the user, various menus of the terminal, an interactive interface, file display, and / or the playback of any multimedia file.

[0055] The memory 220 can be used to store instructions and data. The memory 220 may mainly include a storage instruction area and a storage data area. The storage data area can store various data, such as multimedia files, texts, etc.; the storage instruction area can store software units such as an operating system, applications, instructions required for at least one function, or their subsets or extended sets. It may also include a non-volatile random access memory; it provides for the processor 270 to manage the hardware, software, and data resources in the computing processing device, support control software and applications. It is also used for the storage of multimedia files, as well as the storage of running programs and applications.

[0056] The processor 270 is the control center of the terminal. It uses various interfaces and lines to connect all parts of the entire terminal. By running or executing the instructions stored in the memory 220 and calling the data stored in the memory 220, it executes various functions of the terminal and processes data, thereby controlling the terminal as a whole. Optionally, the processor 270 may include one or more processing units; preferably, the processor 270 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above modem processor may not be integrated into the processor 270 either. In some embodiments, the processor and the memory can be implemented on a single chip. In some embodiments, they can also be separately implemented on independent chips. The processor 270 can also be used to generate corresponding operation control signals, send them to corresponding components of the computing processing device, read and process data in the software, especially read and process the data and programs in the memory 220, so that each function module therein executes corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0057] Among them, the memory 220 can be used to store software code related to the picture material deconstruction method. The processor 270 can execute the software code in the memory 220 or can also schedule other units (such as the above input unit 230 and display unit 240) to implement corresponding functions.

[0058] The radio frequency unit 210 (optional) can be used for receiving and transmitting information or signals during a call. For example, after receiving the downlink information from the base station, it is sent to the processor 270 for processing; in addition, the uplink data is sent to the base station. Generally, the radio frequency unit 210 includes, but is not limited to, antennas, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the radio frequency unit 210 can also communicate with network devices and other devices through wireless communication. This wireless communication can use any communication standard or protocol, including but not limited to the Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0059] Among them, in the embodiments of the present application, the radio frequency unit 210 can send data to other devices and can also receive data sent by other devices. It should be understood that the radio frequency unit 210 is optional and can be replaced by other communication interfaces, such as a network interface.

[0060] The terminal further includes a power supply 290 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 270 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system.

[0061] The terminal further includes an external interface 280, which can be a standard Micro USB interface or a multi-pin connector, and can be used to connect the terminal to other devices for communication or to connect a charger to charge the terminal.

[0062] Although not shown, the terminal may further include a flash, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which will not be elaborated here.

[0063] Next, the product form of the above server will be described.

[0064] Figure 3 A schematic structural diagram of the above server is provided, as Figure 3As shown, the server may include a bus 301, a processor 302, a communication interface 303, and a memory 304. The processor 302, the memory 304, and the communication interface 303 communicate with each other via the bus 301.

[0065] The bus 301 may be a peripheral component interconnect (PCI) bus, an extended industry standard architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 3 only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0066] The processor 302 can be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0067] The memory 304 may include volatile memory, such as random access memory (RAM). The memory 304 may also include non-volatile memory, such as read-only memory (ROM), flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0068] The memory 304 can be used to store software code related to the method for deconstructing picture materials. The processor 302 can call the software code stored in the memory 304 or schedule other units to implement corresponding functions.

[0069] The processors in the above-mentioned terminal and server (such as processor 270 and processor 302) can be hardware circuits (such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), general-purpose processor, digital signal processor (DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with the function of executing instructions, such as CPU, DSP, etc., or a hardware system without the function of executing instructions, such as ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without the function of executing instructions and hardware systems with the function of executing instructions.

[0070] In the process of implementing this case, the inventors of this case found that in addition to the manual deconstruction method, there are currently a picture material deconstruction method based on computer vision algorithms and a picture material deconstruction method based on deep learning models. Among them, the picture material deconstruction method based on computer vision algorithms analyzes the picture material based on methods such as feature point matching, edge detection, and color histogram analysis to obtain the label information of the picture material. The picture material deconstruction method based on deep learning models predicts the label information of the image through the trained model. The inventors of this case studied the above two deconstruction methods and found that the above two deconstruction methods can only produce some simple labels and cannot meet the needs of some complex scenarios.

[0071] In view of the problems existing in the current picture material deconstruction method, the inventors of this case conducted research. Through continuous research, a picture material deconstruction method with better effects was finally proposed. Next, the picture material deconstruction method provided in this application will be introduced through the following embodiments.

[0072] Please refer to Figure 4 , which shows a schematic flowchart of the picture material deconstruction method provided in the embodiment of this application. The picture material deconstruction method can include:

[0073] Step S401: Obtain target data.

[0074] In one possible implementation, the target data can include picture materials.

[0075] In another possible implementation, the target data can include picture materials and specified labels (users can specify which labels to analyze for the picture materials).

[0076] After obtaining the picture materials, the picture materials can be preprocessed, for example, adjusting the size, unifying the format, etc. Then, the subsequent analysis and processing can be carried out on the preprocessed picture materials.

[0077] Step S402: Use one or more of the label analysis strategies corresponding to a plurality of preset label sets to perform label analysis on the picture materials, and obtain the label analysis results of the picture materials on the labels included in the corresponding label sets.

[0078] Among them, the multiple label sets are obtained by classifying a plurality of predefined labels. That is, each label set is a category of labels, and the label analysis strategy corresponding to any label set can analyze the labels in the corresponding label set for the picture materials. That is, each label analysis strategy can analyze a category of labels for the picture materials.

[0079] As mentioned above, the target data can only include picture materials, that is, the user does not specify which labels to analyze for the picture materials. In this case, the label analysis strategies corresponding to a plurality of preset label sets can be used to perform label analysis on the picture materials, that is, all the label analysis strategies are used to perform label analysis on the picture materials.

[0080] As mentioned above, in addition to picture materials, the target data can also include specified labels, that is, the user specifies which labels to analyze for the picture materials. If the target data includes specified labels, the label set to which the specified labels belong can be determined from the multiple label sets first. Then, the label analysis strategy corresponding to the label set to which the specified labels belong is used to perform label analysis on the picture materials to obtain the label analysis results of the picture materials on the specified labels.

[0081] It should be noted that when the target data includes specified labels, in addition to using the above processing method, the label analysis strategies corresponding to the multiple label sets (that is, all the label analysis strategies) can also be used to perform label analysis on the picture materials. After obtaining the label analysis results obtained by using all the label analysis strategies, the label analysis results of the picture materials on the specified labels can be obtained from the obtained label analysis results.

[0082] Step S403: Perform formatting processing on the obtained label analysis results to obtain the label information of the picture materials.

[0083] After obtaining all the label analysis results, all the label analysis results can be formatted to obtain label information in a unified format as the label information of the picture materials.

[0084] The picture material deconstruction method provided by the embodiment of the present application, after obtaining the picture material, can adopt one or more of the label analysis strategies corresponding to a plurality of preset label sets to perform label analysis on the picture material, so as to obtain the label analysis result of the picture material on the labels included in the corresponding label set. After obtaining the label analysis result, the obtained label analysis result can be formatted, and finally the label information of the picture material is obtained. The picture material deconstruction method provided by the embodiment of the present application can automatically deconstruct the picture material. Compared with the manual deconstruction method, the deconstruction time is greatly shortened and the deconstruction efficiency is greatly improved. Since the deconstruction process does not require manual participation, the influence of subjective factors is avoided, and the accuracy of the deconstruction result can be ensured. In addition, the picture material deconstruction method provided by the embodiment of the present application can adopt a variety of label analysis strategies to perform label analysis on the picture material. Since each label analysis strategy can analyze a certain type of label, multiple types of labels can be obtained through the picture material deconstruction method provided by the embodiment of the present application. It can be seen that the picture material deconstruction method provided by the embodiment of the present application can deconstruct the picture material more comprehensively.

[0085] In another embodiment of the present application, the specific implementation process of "step S402: adopt one or more of the label analysis strategies corresponding to a plurality of preset label sets to perform label analysis on the picture material, and obtain the label analysis result of the picture material on the labels included in the corresponding label set" in the above embodiment is introduced.

[0086] In order to realize picture deconstruction, the present application pre-defines a plurality of labels (which specific labels are defined depends on the actual application scenario), and presets a variety of label analysis strategies. Specifically, the variety of label analysis strategies may include a label analysis strategy based on a large model, and also include a supplementary analysis strategy. Considering that the large model has poor analysis effects on some labels, in this embodiment, on the basis of the label analysis strategy based on the large model, a supplementary analysis strategy is introduced to make up for the deficiencies of the label analysis strategy based on the large model.

[0087] Among them, the supplementary analysis strategy may include a label analysis strategy based on object recognition, a label analysis strategy based on text recognition, and a label analysis strategy combining a knowledge database. Of course, this embodiment is not limited thereto, and the supplementary analysis strategy may also include any one or any two of the above three label analysis strategies. In order to obtain a more comprehensive deconstruction result, the supplementary analysis strategy preferably includes the above three label analysis strategies.

[0088] This embodiment takes the variety of label analysis strategies including a label analysis strategy based on a large model, a label analysis strategy based on object recognition, a label analysis strategy based on text recognition, and a label analysis strategy combining a knowledge database as an example to introduce the subsequent content.

[0089] In a possible implementation, according to the analysis effects of four tag analysis strategies (the large model-based tag analysis strategy, the object recognition-based tag analysis strategy, the text recognition-based tag analysis strategy, and the tag analysis strategy combined with a knowledge database) on predefined tags, the predefined tags can be divided into four categories to obtain four tag sets, namely: the first tag set corresponding to the large model-based tag analysis strategy, the second tag set corresponding to the object recognition-based tag analysis strategy, the third tag set corresponding to the text recognition-based tag analysis strategy, and the fourth tag set corresponding to the tag analysis strategy combined with a knowledge database. The large model-based tag analysis strategy has a good analysis effect on the tags in the first tag set (the tags in the first tag set can be tags for which the verified large model has a good analysis effect), the object recognition-based tag analysis strategy has a good analysis effect on the tags in the second tag set (the tags in the second tag set are object class tags), the text recognition-based tag analysis strategy has a good analysis effect on the tags in the third tag set (the tags in the third tag set are text class tags), and the tag analysis strategy combined with a knowledge database has a good analysis effect on the tags in the fourth tag set (the tags in the fourth tag set are tags related to specific scenarios).

[0090] Exemplarily, the first tag set may include tags such as object category (real person / cartoon character / animal), person gender, person expression, picture hue, picture style, scene, and environmental atmosphere. The second tag set may include tags such as the number of objects (such as people), object location, object proportion, product logo, and product location. The third tag set may include tags such as text content, text location, context, and appeal. The fourth tag set may include tags such as star figures, advertising slogans, and legal provisions. It should be noted that the tags included in each of the above tag sets are only examples, and this embodiment does not limit that each tag set includes the tags listed above.

[0091] Next, the processes of performing tag analysis on picture materials using the large model-based tag analysis strategy, the object recognition-based tag analysis strategy, the text recognition-based tag analysis strategy, and the tag analysis strategy combined with a knowledge database will be introduced separately.

[0092] The process of performing tag analysis on picture materials using the large model-based tag analysis strategy to obtain the tag analysis results of the picture materials on the tags included in the corresponding tag set may include: using the large model to analyze the tags included in the first tag set for the picture materials (such as Figure 5As shown in the figure, the tag analysis result of the picture material on the tags included in the first tag set is obtained.

[0093] Specifically, using the large model to analyze the tags included in the first tag set for the picture material, the process of obtaining the tag analysis result of the picture material on the tags included in the first tag set may include:

[0094] Step a1: Obtain the first prompt template (i.e., Prompt template).

[0095] Among them, the first prompt template includes a picture information slot and the tags included in the first tag set, and the first prompt template is used to prompt the large model to analyze the tags included in the first tag set for the picture information in the picture information slot.

[0096] Step a2: Fill the picture material into the picture information slot of the first prompt template to obtain the first prompt.

[0097] Step a3: Input the first prompt into the large model to obtain the tag analysis result of the picture material on the tags included in the first tag set output by the large model.

[0098] Input the first prompt into the large model, and the large model analyzes the tags included in the first tag set for the picture material and outputs the tag analysis result of the picture material on the tags included in the first tag set.

[0099] The process of adopting a tag analysis strategy based on object recognition to perform tag analysis on the picture material and obtaining the tag analysis result of the picture material on the tags included in the corresponding tag set may include: Inputting the picture material into a pre-constructed object recognition model to obtain the tag analysis result of the picture material on the tags included in the second tag set output by the object recognition model.

[0100] It should be noted that the object recognition model in this embodiment is a model specifically used for object recognition. Optionally, the object recognition model in this embodiment can adopt traditional object recognition models, such as YOLO, DeepLogo2, etc.

[0101] The process of adopting a tag analysis strategy based on text recognition to perform tag analysis on the picture material and obtaining the tag analysis result of the picture material on the tags included in the corresponding tag set may include:

[0102] Step b1: Input the picture material into a pre-constructed text recognition model to obtain the text recognition result of the picture material.

[0103] The text recognition model in this embodiment can be an OCR model.

[0104] Such as Figure 5As shown, the picture material is input into the text recognition model. The text recognition model recognizes the text in the picture material and outputs the text recognition result of the picture material (such as text content, text position).

[0105] Step b2: Use the large model to analyze the labels related to the text content in the third label set for the text recognition result of the picture material, and obtain the label analysis result of the picture material on the labels related to the text content.

[0106] Considering that the text recognition model usually can only obtain the text content and text position, in order to obtain more comprehensive text - type labels, as Figure 5 shown, after obtaining the text recognition result of the picture material in this embodiment, the large model is further used to analyze the labels related to the text content (such as context, appeal, etc. labels) for the text recognition result of the picture material (such as text content).

[0107] Specifically, the process of using the large model to analyze the labels related to the text content in the third label set for the text recognition result of the picture material and obtaining the label analysis result of the picture material on the labels related to the text content may include:

[0108] Step b21: Obtain the second prompt template.

[0109] Among them, the second prompt template includes a text information slot and the labels related to the text content included in the third label set (such as context, appeal, etc. labels). The second prompt template is used to prompt the large model to analyze the labels related to the text content for the text information in the text information slot.

[0110] Step b22: Fill the text recognition result of the picture material into the text information slot of the second prompt template to obtain the second prompt.

[0111] Step b23: Input the second prompt into the large model to obtain the label analysis result of the picture material on the labels related to the text content output by the large model.

[0112] Input the second prompt into the large model. The large model analyzes the labels related to the text content (such as context, appeal, etc. labels) for the text recognition result of the picture material (such as text content), and outputs the label analysis result of the picture material on the labels related to the text content.

[0113] Adopt a label analysis strategy combined with a knowledge database to conduct label analysis on the picture material, and obtain the label analysis result of the picture material on the labels included in the corresponding label set, including:

[0114] Step c1: Obtain the knowledge data related to the picture material from the knowledge database.

[0115] Specifically, the picture material can be vectorized to obtain a representation vector of the picture material, calculate the similarity between the representation vector of the picture material and the representation vector of each piece of knowledge data in the knowledge database, and determine the knowledge data related to the picture material according to the similarity between the representation vector of the picture material and the representation vector of each piece of knowledge data in the knowledge database.

[0116] Step c2: Use the large model, refer to the knowledge data related to the picture material, analyze the labels included in the fourth label set for the picture material, and obtain the label analysis result of the picture material on the labels included in the fourth label set.

[0117] Specifically, the process of using the large model, referring to the knowledge data related to the picture material, and analyzing the labels included in the fourth label set for the picture material to obtain the label analysis result of the picture material on the labels included in the fourth label set may include:

[0118] Step c21: Obtain the third prompt word template.

[0119] Among them, the third prompt word template includes a picture information slot, a reference information slot, and the labels included in the fourth label set. The third prompt word template is used to prompt the large model to refer to the information in the above reference information slot and analyze the labels included in the fourth label set for the picture information in the picture information slot.

[0120] Step c22: Fill the picture material into the picture information slot in the third prompt word template, and fill the knowledge data related to the picture material into the reference information slot in the third prompt word template to obtain the third prompt word.

[0121] Step c23: Input the third prompt word into the large model to obtain the label analysis result of the picture material on the labels included in the fourth label set output by the large model.

[0122] Input the third prompt word into the large model. The large model refers to the knowledge data related to the picture material, analyzes the labels in the fourth label set for the picture material, and outputs the label analysis result of the picture material on the labels included in the fourth label set.

[0123] It should be noted that the label analysis strategies based on the large model, the label analysis strategy based on object recognition, the label analysis strategy based on text recognition, and the label analysis strategy combining the knowledge database can be used to perform label analysis on the picture material in parallel to improve efficiency.

[0124] After performing label analysis on picture materials using the large model-based label analysis strategy, object recognition-based label analysis strategy, text recognition-based label analysis strategy, and knowledge database-integrated label analysis strategy, all obtained label analysis results can be formatted to obtain the final deconstruction result of the picture materials, that is, the label information of the picture materials. Thus, the labeling of the picture materials is completed.

[0125] In a possible implementation, as Figure 5 shown, a large model can be used to format the obtained label analysis results according to preset formatting requirements to obtain the label information of the picture materials.

[0126] Specifically, the process of using a large model to format the obtained label analysis results according to preset formatting requirements to obtain the label information of the picture materials can include:

[0127] Step d1: Obtain the fourth prompt word template.

[0128] Among them, the fourth prompt word template includes a label information slot and a formatting requirement information slot. The fourth prompt word template is used to prompt the large model to format the label information in the label information slot according to the formatting requirement information in the formatting requirement information slot.

[0129] Step d2: Fill the obtained label analysis results into the label information slot of the fourth prompt word template, and fill the preset formatting requirements into the formatting requirement information slot of the fourth prompt word template to obtain the fourth prompt word.

[0130] Step d3: Input the fourth prompt word into the large model to obtain the formatted label analysis result output by the large model as the label information of the picture materials.

[0131] Input the fourth prompt word into the large model. The large model formats the label analysis results according to the preset formatting requirements and outputs the formatted label analysis results.

[0132] The picture material deconstruction method provided by the embodiments of this application can deconstruct picture materials using the large model-based label analysis strategy, object recognition-based label analysis strategy, text recognition-based label analysis strategy, and knowledge database-integrated label analysis strategy. Since the deconstruction process is automatically completed without manual participation, it has a high degree of intelligence. Furthermore, it has a high deconstruction efficiency. Since multiple label analysis strategies are used to perform label analysis on picture materials, each label analysis strategy can analyze a type of label, so multi-dimensional comprehensive deconstruction of picture materials can be achieved. Since the large model-based label analysis strategy is used in combination with supplementary analysis strategies to deconstruct picture materials, it has a high deconstruction accuracy.

[0133] The above introduced a method for decomposing picture materials provided by the embodiments of the present application. Next, an apparatus for executing the above method for decomposing picture materials will be introduced.

[0134] Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of an apparatus for decomposing picture materials provided by the embodiments of the present application. As Figure 6 shown, the apparatus for decomposing picture materials may include: a data acquisition module 601, a tag analysis module 602, and a formatting processing module 603.

[0135] The data acquisition module 601 is configured to acquire target data, where the target data includes picture materials.

[0136] The tag analysis module 602 is configured to perform tag analysis on the picture materials by using one or more tag analysis strategies corresponding to a plurality of preset tag sets, so as to obtain tag analysis results of the picture materials on the tags included in the corresponding tag sets.

[0137] Among them, the plurality of tag sets are obtained by classifying a plurality of predefined tags, and the tag analysis strategy corresponding to any tag set can analyze the tags in the corresponding tag set for the picture materials.

[0138] The formatting processing module 603 is configured to perform formatting processing on the obtained tag analysis results to obtain the tag information of the picture materials.

[0139] In a possible implementation manner, the target data further includes specified tags.

[0140] When the tag analysis module 602 performs tag analysis on the picture materials by using one or more tag analysis strategies corresponding to a plurality of preset tag sets to obtain tag analysis results of the picture materials on the tags included in the corresponding tag sets, it is specifically configured to:

[0141] Determine the tag set to which the specified tag belongs from the plurality of tag sets;

[0142] Perform tag analysis on the picture materials by using the tag analysis strategy corresponding to the tag set to which the specified tag belongs to obtain tag analysis results of the picture materials on the tags included in the corresponding tag set.

[0143] In a possible implementation manner, the tag analysis strategies corresponding to the plurality of tag sets respectively include a tag analysis strategy based on a large model and a supplementary analysis strategy.

[0144] Among them, the supplementary analysis strategy includes one or more of the following tag analysis strategies: the tag analysis strategy based on object recognition, the tag analysis strategy based on text recognition, and the tag analysis strategy combined with a knowledge database.

[0145] In a possible implementation, the tag analysis module 602 includes: a large model analysis module.

[0146] The large model analysis module is used to perform tag analysis on the picture material by adopting a tag analysis strategy based on a large model, so as to obtain the tag analysis result of the picture material on the tags included in the corresponding tag set.

[0147] When the large model analysis module performs tag analysis on the picture material by adopting a tag analysis strategy based on a large model and obtains the tag analysis result of the picture material on the tags included in the corresponding tag set, it is specifically used for:

[0148] Using the large model to analyze the tags included in the first tag set for the picture material, so as to obtain the tag analysis result of the picture material on the tags included in the first tag set, where the first tag set is the tag set corresponding to the tag analysis strategy based on the large model.

[0149] In a possible implementation, the tag analysis module 602 includes: an object recognition supplementary module.

[0150] The object recognition supplementary module is used to perform tag analysis on the picture material by adopting a tag analysis strategy based on object recognition, so as to obtain the tag analysis result of the picture material on the tags included in the corresponding tag set.

[0151] When the object recognition supplementary module performs tag analysis on the picture material by adopting a tag analysis strategy based on object recognition and obtains the tag analysis result of the picture material on the tags included in the corresponding tag set, it is specifically used for:

[0152] Inputting the picture material into a pre-constructed object recognition model to obtain the tag analysis result of the picture material output by the object recognition model on the tags included in the second tag set, where the second tag set is the tag set corresponding to the tag analysis strategy based on object recognition.

[0153] In a possible implementation, the tag analysis module 602 includes: a text recognition supplementary module.

[0154] The text recognition supplementary module is used to perform tag analysis on the picture material by adopting a tag analysis strategy based on text recognition, so as to obtain the tag analysis result of the picture material on the tags included in the corresponding tag set.

[0155] When the text recognition supplementary module adopts a text recognition-based label analysis strategy to analyze the picture material and obtain the label analysis result of the picture material on the labels contained in the corresponding label set, it is specifically used for:

[0156] Input the picture material into a pre-constructed text recognition model to obtain the text recognition result of the picture material;

[0157] Use a large model to analyze the labels related to the text content in the third label set for the text recognition result of the picture material, and obtain the label analysis result of the picture material on the labels related to the text content.

[0158] In a possible implementation manner, the label analysis module 602 includes: a supplementary module that combines with the knowledge database.

[0159] The supplementary module that combines with the knowledge database is used to adopt a label analysis strategy that combines with the knowledge database to analyze the picture material and obtain the label analysis result of the picture material on the labels contained in the corresponding label set.

[0160] When the supplementary module that combines with the knowledge database adopts a label analysis strategy that combines with the knowledge database to analyze the picture material and obtain the label analysis result of the picture material on the labels contained in the corresponding label set, it is specifically used for:

[0161] Obtain the knowledge data related to the picture material from the knowledge database;

[0162] Use a large model, refer to the knowledge data related to the picture material, and analyze the labels included in the fourth label set for the picture material to obtain the label analysis result of the picture material on the labels contained in the fourth label set, where the fourth label set is the label set corresponding to the label analysis strategy that combines with the knowledge database.

[0163] In a possible implementation manner, when the formatting processing module 603 performs formatting processing on the obtained label analysis result to obtain the label information of the picture material, it is specifically used for:

[0164] Use a large model to perform formatting processing on the obtained label analysis result according to the preset formatting requirements to obtain the label information of the picture material.

[0165] The picture material deconstruction device provided by the embodiments of the present application can automatically deconstruct picture materials. Compared with the manual deconstruction method, the deconstruction time is greatly shortened and the deconstruction efficiency is greatly improved. Since the deconstruction process does not require manual participation, the influence of human subjective factors is avoided, and the accuracy of the deconstruction result can be ensured. In addition, the picture material deconstruction device provided by the embodiments of the present application can adopt a variety of tag analysis strategies to analyze the picture materials. Since each tag analysis strategy can analyze a type of tag, multiple types of tags can be obtained through the picture material deconstruction device provided by the embodiments of the present application. It can be seen that the picture material deconstruction device provided by the embodiments of the present application can deconstruct picture materials more comprehensively.

[0166] The embodiments of the present application also provide an electronic device, which may include: at least one processor, at least one communication interface, at least one memory, and at least one communication bus.

[0167] In the embodiments of the present application, the number of the processor, the communication interface, the memory, and the communication bus is at least one, and the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0168] The processor may be a central processing unit (CPU), or a specific integrated circuit (ASIC) (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application, etc.;

[0169] The memory may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory;

[0170] Among them, the memory stores a program, and the processor can call the program stored in the memory, and the program is used to implement the steps of the picture material deconstruction method provided in the above embodiments.

[0171] The embodiments of the present application also provide a computer storage medium, and the storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can be enabled to implement the steps of the picture material deconstruction method provided in the above embodiments.

[0172] The embodiments of the present application also provide a computer program product, including computer-readable instructions. When the computer-readable instructions run on an electronic device, the electronic device is enabled to implement the steps of the picture material deconstruction method provided in the above embodiments.

[0173] In addition, it should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the attached drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines.

[0174] Through the description of the above embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. Generally, functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or dedicated circuits. However, for this application, in more cases, software program implementation is a better implementation method. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disc of a computer, and includes several instructions to enable a computer device (which can be a personal computer, training device, or network device, etc.) to execute the methods described in various embodiments of this application.

[0175] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product.

[0176] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

Claims

1. A method for deconstructing picture materials, characterized in that Including: Obtain target data, where the target data includes picture materials; Use one or more of the label analysis strategies corresponding to a plurality of preset label sets to perform label analysis on the picture materials, and obtain label analysis results of the picture materials on the labels included in the corresponding label sets. Among them, the plurality of label sets are obtained by classifying a plurality of predefined labels, and the label analysis strategy corresponding to any label set can analyze the labels in the corresponding label set for the picture materials; Perform formatting processing on the obtained label analysis results to obtain the label information of the picture materials.

2. The picture material deconstruction method according to claim 1, wherein The target data further includes specified labels; The step of using one or more of the label analysis strategies corresponding to a plurality of preset label sets to perform label analysis on the picture materials and obtain label analysis results of the picture materials on the labels included in the corresponding label sets includes: Determine the label set to which the specified label belongs from the plurality of label sets; Use the label analysis strategy corresponding to the label set to which the specified label belongs to perform label analysis on the picture materials, and obtain label analysis results of the picture materials on the labels included in the corresponding label sets.

3. The method for deconstructing picture materials according to claim 1 or 2, characterized in that The label analysis strategies corresponding to the plurality of label sets respectively include a label analysis strategy based on a large model and a supplementary analysis strategy; Among them, the supplementary analysis strategy includes one or more of the following label analysis strategies: a label analysis strategy based on object recognition, a label analysis strategy based on text recognition, and a label analysis strategy combining a knowledge database.

4. The picture material deconstruction method according to claim 3, wherein Using the label analysis strategy based on the large model to perform label analysis on the picture materials and obtain label analysis results of the picture materials on the labels included in the corresponding label sets includes: Use the large model to analyze the labels included in the first label set for the picture materials, and obtain label analysis results of the picture materials on the labels included in the first label set, where the first label set is the label set corresponding to the label analysis strategy based on the large model.

5. The method for deconstructing picture materials according to claim 3, characterized in that, Using the label analysis strategy based on object recognition to perform label analysis on the picture materials and obtain label analysis results of the picture materials on the labels included in the corresponding label sets includes: inputting the picture materials into a pre-constructed object recognition model, and obtaining label analysis results of the picture materials on the labels included in the second label set output by the object recognition model, where the second label set is the label set corresponding to the label analysis strategy based on object recognition.

6. The picture material deconstruction method according to claim 3, characterized in that Using the label analysis strategy based on text recognition to perform label analysis on the picture materials and obtain label analysis results of the picture materials on the labels included in the corresponding label sets includes: Input the picture materials into a pre-constructed text recognition model to obtain the text recognition result of the picture materials; Use the large model to analyze the labels related to the text content in the third label set for the text recognition result of the picture materials, and obtain label analysis results of the picture materials on the labels related to the text content.

7. The method for deconstructing picture materials according to claim 3, wherein Using the tag analysis strategy combined with the knowledge database, perform tag analysis on the picture material to obtain the tag analysis result of the picture material on the tags included in the corresponding tag set, including: Obtain the knowledge data related to the picture material from the knowledge database; Using a large model, referring to the knowledge data related to the picture material, analyze the tags included in the fourth tag set for the picture material to obtain the tag analysis result of the picture material on the tags included in the fourth tag set, where the fourth tag set is the tag set corresponding to the tag analysis strategy combined with the knowledge database.

8. The method for deconstructing picture materials according to claim 1, wherein Format the obtained tag analysis result to obtain the tag information of the picture material, including: Using a large model, format the obtained tag analysis result according to the preset formatting requirements to obtain the tag information of the picture material.

9. An image material deconstruction device, characterized in that Including: A data acquisition module, a tag analysis module, and a formatting processing module; The data acquisition module is used to acquire target data, where the target data includes picture materials; The tag analysis module is used to perform tag analysis on the picture material by using one or more of the tag analysis strategies respectively corresponding to a plurality of preset tag sets, to obtain the tag analysis result of the picture material on the tags included in the corresponding tag set, where the plurality of tag sets are obtained by classifying a plurality of predefined tags, and the tag analysis strategy corresponding to any tag set can analyze the tags in the corresponding tag set for the picture material; The formatting processing module is used to format the obtained tag analysis result to obtain the tag information of the picture material.

10. An electronic device, characterized in that, Including at least one processor and a memory connected to the processor, where: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the steps of the picture material deconstruction method according to any one of claims 1 to 8.

11. A computer storage medium, characterized in that, The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of the picture material deconstruction method according to any one of claims 1 to 8.

12. A computer program product, characterized in that, Including computer-readable instructions, when the computer-readable instructions run on an electronic device, the electronic device implements the steps of the picture material deconstruction method according to any one of claims 1 to 8.

Images