Data generation method and device, electronic equipment and storage medium
Through the combination of image segmentation large model and enhanced model, the problems of insufficient authenticity and diversity and logical errors in the data generation method are solved, and high-quality data generation is realized, which is suitable for a variety of scenarios such as engineering construction image data generation.
Patent Information
- Application Number
- CN202410984123.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-15
AI Technical Summary
The existing data generation methods based on game scenarios have high technical barriers, insufficient authenticity and diversity of generated data, and there are problems such as overlapping goals, wrong locations and logical disorders.
The image segmentation model is trained through the image data set, the images to be processed are obtained for semantic segmentation and recognition, the enhanced model is added for overlap, position and logical judgment, and the image data is adjusted to achieve the authenticity and rationality of the data.
By integrating image segmentation, area recognition and enhancement model addition, intelligent processing and modification of images and video content is realized, the authenticity and logic of generated data are ensured, and the lack of data authenticity and diversity and overlapping goals are solved. The problem of wrong position.
Smart Images

Figure CN118968058A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision and generative model technology, and in particular to a data generation method, device, electronic device and storage medium. Background Art
[0002] In recent years, data generation technology is in a stage of rapid development, and its application fields are wide-ranging, including but not limited to natural language processing, image generation, audio synthesis, etc.
[0003] However, data generation methods based on game scenarios have the problem of high technical barriers, which is mainly due to the need to reverse-parse complex game codes, which is a time-consuming and challenging process, and the quality of the generated data is highly dependent on the picture quality of the game itself; whether it is a simulation environment or a game scenario, there is a general problem that the generated data does not conform to the real-world situation and lacks diversity, which results in the generated data being unable to effectively reflect the complexity and variability of the real world when used for training or testing.
[0004] In addition, the mapping scheme does not technically analyze the distribution characteristics of the original data in depth, but relies on random sampling and random position mapping, which causes the generated data to logically not meet the requirements of the actual scene. Due to the randomness of the mapping scheme, logical problems such as target object overlap and position errors may occur. The training data may contain bias, errors or false information, resulting in the generated content may be inaccurate or misleading to users.
[0005] Therefore, there is a need for a data generation method that can ensure the authenticity of the generated data, enhance the rationality and logic of the data, and obtain high-quality data. Summary of the invention
[0006] The embodiments of the present invention provide a data generation method to solve the problems of high technical threshold, insufficient authenticity and diversity of generated data, target overlap, position error and logical disorder in the prior art. The technical solution is as follows:
[0007] According to one aspect of the present invention, a data generation method comprises: training a large image segmentation model through an image data set to obtain a trained segmentation model; acquiring an image to be processed, performing semantic segmentation on the image to be processed through the segmentation model to obtain a category prediction map of each pixel in the image to be processed; performing semantic recognition based on the category prediction map of each pixel to obtain multiple semantic regions of the image to be processed; adding a corresponding enhancement model MOD to each of the semantic regions and performing overlapping, position and logical judgments, and adjusting the enhancement model MOD to generate image data based on the judgment results.
[0008] In one of the embodiments, the image segmentation large model is trained by an image dataset to obtain a trained segmentation model by the following steps: obtaining an image segmentation dataset and a test set, using a visual transformer VIT as a feature extractor, and combining a downsampling network to construct a large image segmentation model; optimizing the parameters of the large image segmentation model by iterative training of the image segmentation dataset; evaluating the performance of the large image segmentation model by the test set, and saving the trained model weights and configuration files to obtain a trained segmentation model.
[0009] In one of the embodiments, a corresponding enhancement model MOD is added to each of the semantic regions, and overlap, position and logic judgment are performed to obtain a judgment result, and the enhancement model MOD is adjusted to generate image data according to the judgment result through the following steps: the size, proportion, position, orientation and angle of the corresponding enhancement model MOD are set according to the type, size and position of each of the semantic regions; overlap judgment is performed for each of the semantic regions and its corresponding enhancement model MOD to obtain an overlap degree, and if the overlap degree exceeds a set threshold, the position of the enhancement model MOD is reset; position judgment is performed for each of the semantic regions and its corresponding enhancement model MOD to obtain a coordination degree, and if the coordination degree does not meet the set conditions, the orientation and angle of the enhancement model MOD are reset; logic judgment is performed for each of the semantic regions and its corresponding enhancement model MOD to obtain authenticity, and if the authenticity does not meet the set conditions, the enhancement model MOD is relocated.
[0010] In one of the embodiments, adding a corresponding enhanced model MOD to each of the semantic regions for overlap judgment to obtain an overlap degree is achieved by the following steps: the overlap degree is obtained by calculating the ratio of the intersection area and the union area of the bounding box of the enhanced model MOD and the bounding boxes of each of the semantic regions.
[0011] In one of the embodiments, adding a corresponding enhanced model MOD to each of the semantic areas for position judgment to obtain a coordination degree is achieved by the following steps: obtaining a coordination degree based on the matching degree between the size, shape, direction, orientation, angle, size and proportion of the enhanced model MOD and each of the semantic areas.
[0012] In one of the embodiments, adding a corresponding enhancement model MOD to each of the semantic regions for logical judgment to obtain the authenticity is achieved through the following steps: judging whether the enhancement model MOD matches the overall theme and background of the image to be processed; judging whether the enhancement model MOD conforms to common sense and physical laws; judging whether the enhancement model MOD can enhance the amount of information or expression effect of the image to be processed.
[0013] In one of the embodiments, the method further comprises the following steps: if the image to be processed is a continuous frame image, then judging the dynamic effect of the enhanced model MOD and the degree of synchronization with the image to be processed.
[0014] According to one aspect of the present invention, a data generating device comprises: a segmentation model training module, which is used to train a large image segmentation model through an image data set to obtain a trained segmentation model; a semantic segmentation module, which is used to obtain an image to be processed, perform semantic segmentation on the image to be processed through the segmentation model, and obtain a category prediction map of each pixel in the image to be processed; a semantic recognition module, which is used to perform semantic recognition based on the category prediction map of each pixel to obtain multiple semantic regions of the image to be processed; and an image generation module, which is used to add a corresponding enhancement model MOD to each of the semantic regions and perform overlapping, position and logical judgments, and adjust the enhancement model MOD to generate image data according to the judgment results.
[0015] According to one aspect of the present invention, an electronic device includes at least one processor and at least one memory, wherein the memory stores computer-readable instructions; the computer-readable instructions are executed by one or more of the processors, so that the electronic device implements the data generation method as described above.
[0016] According to one aspect of the present invention, a storage medium stores computer-readable instructions thereon, wherein the computer-readable instructions are executed by one or more processors to implement the data generating method as described above.
[0017] The beneficial effects brought by the technical solution provided by the present invention are:
[0018] In the above technical scheme, the present invention first trains the image segmentation model through the image data set to obtain the trained segmentation model, obtains the image to be processed, performs semantic segmentation on the image to be processed through the segmentation model, obtains the category prediction map of each pixel in the image to be processed, performs semantic recognition according to the category prediction map of each pixel, obtains multiple semantic regions of the image to be processed, adds a corresponding enhancement model MOD to each semantic region and performs overlapping, position and logical judgment, adjusts the enhancement model MOD according to the judgment result to generate image data, and realizes intelligent processing and modification of image and video content by integrating image segmentation, region recognition and MOD addition, which not only ensures the authenticity of the generated data, but also enhances the rationality and logic of the data, thereby effectively solving the problems of high technical threshold, insufficient authenticity and diversity of the generated data, target overlap, position errors and logical confusion in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in describing the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative work.
[0020] Figure 1 is a flow chart of a data generation method according to an exemplary embodiment;
[0021] Figure 2 yes Figure 1 Flow chart corresponding to step 110 in the embodiment;
[0022] Figure 3 yes Figure 1 Flow chart corresponding to step 170 in the embodiment;
[0023] Figure 4 It is a flowchart of a data generation method in an application scenario;
[0024] Figure 5 yes Figure 4 Schematic diagram of the process of training the segmentation model in the corresponding application scenario;
[0025] Figure 6 is a block diagram of a data generating device according to an exemplary embodiment;
[0026] Figure 7 is a hardware structure diagram of an electronic device according to an exemplary embodiment;
[0027] Figure 8 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION
[0028] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.
[0029] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present disclosure refers to the presence of the features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.
[0030] Data generation: refers to the process of creating or preparing data for training or testing models in fields such as image processing, machine learning, or deep learning. In the context of image segmentation, data generation involves the following aspects: dataset collection, data augmentation, data preprocessing, and simulated data generation.
[0031] MOD (Modification): refers to the modification or enhancement of the original image to add new elements, effects or information. It includes the following types: virtual object addition, visual effects, dynamic elements, etc. It can also be dynamic elements that change over time, such as moving vehicles, walking people or changing backgrounds, etc., to create richer visual effects.
[0032] The present invention provides a data generation method, which realizes intelligent processing and modification of image and video content by integrating image segmentation, region recognition and MOD addition, not only ensuring the authenticity of generated data, but also enhancing the rationality and logic of data, solving the problems of high technical threshold, insufficient authenticity and diversity of generated data, target overlap, position error and logical disorder in the prior art, and the data generation method is applicable to a data generation device, which can be an electronic device. The data generation method in the embodiment of the present invention can be applied to a variety of scenarios, such as image data generation for engineering construction, etc.
[0033] See also Figure 1 , an embodiment of the present invention provides a data generating method, which is applicable to electronic equipment.
[0034] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.
[0035] like Figure 1As shown, the method may include the following steps:
[0036] Step 110, training the large image segmentation model using the image data set to obtain a trained segmentation model.
[0037] Specifically, it is necessary to collect and organize an image dataset containing multiple semantic categories, such as COCO, ADE20K or PASCAL, and use these datasets to train a large image segmentation model (SegModel) based on Visual Transformer (VIT). During the training process, the model will learn how to extract features from the input image and restore these features to the original image size through the upsample network to generate a category prediction for each pixel. Through multiple iterative training and using loss functions such as binary cross entropy loss (BCELOSS) to optimize model parameters, a trained segmentation model that can accurately segment different semantic areas in the image is finally obtained.
[0038] Step 130, obtaining an image to be processed, performing semantic segmentation on the image to be processed using a segmentation model, and obtaining a category prediction map for each pixel in the image to be processed.
[0039] In one possible implementation, such as Figure 2 As shown, step 110 may include the following steps:
[0040] Step 210, obtain an image segmentation dataset and a test set, use the visual transformer VIT as a feature extractor, and build a large image segmentation model in combination with a downsampling network.
[0041] Step 230, optimizing the image segmentation model parameters through iterative training of the image segmentation dataset.
[0042] Step 250, evaluate the performance of the large image segmentation model through the test set, and save the trained model weights and configuration files to obtain a trained segmentation model.
[0043] Specifically, the image to be processed is input into the trained segmentation model, and the model will classify and predict each pixel in the image, generating a category prediction map of the same size as the original image. Each pixel on this prediction map corresponds to a semantic category label, such as road, vehicle, tree, person, etc., thus achieving semantic segmentation of the image.
[0044] Step 150 , performing semantic recognition according to the category prediction map of each pixel point to obtain multiple semantic regions of the image to be processed.
[0045] Specifically, semantic recognition is performed based on the category prediction map, and adjacent pixels with the same or similar semantic labels can be identified and divided into different semantic regions. These regions represent different objects or scenes in the image, such as streets, buildings, green belts, vehicles, pedestrians, etc. Through this process, multiple semantic regions of the image to be processed can be obtained.
[0046] Step 170, adding a corresponding enhancement model MOD to each semantic region and performing overlapping, position and logic judgments, and adjusting the enhancement model MOD to generate image data according to the judgment results.
[0047] In one possible implementation, such as Figure 3 As shown, step 170 may include the following steps:
[0048] Step 310: setting the size, proportion, position, orientation and angle of the corresponding enhanced model MOD according to the type, size and position of each semantic region.
[0049] Step 330 , performing overlap judgment for each semantic region and its corresponding enhanced model MOD to obtain an overlap degree. If the overlap degree exceeds a set threshold, the position of the enhanced model MOD is reset.
[0050] In a possible implementation, the overlap degree is obtained by calculating the ratio of the intersection area to the union area of the bounding box of the enhanced model MOD and the bounding boxes of each semantic region.
[0051] Step 350 , performing position determination for each semantic region and its corresponding enhanced model MOD to obtain a coordination degree. If the coordination degree does not meet a set condition, the direction and angle of the enhanced model MOD are reset.
[0052] In a possible implementation, the coordination degree is obtained according to the matching degree between the size, shape, direction, orientation, angle, size and ratio of the enhanced model MOD and each semantic area.
[0053] Step 370 , performing logical judgment for each semantic region and its corresponding enhanced model MOD to obtain the authenticity. If the authenticity does not meet the set conditions, the enhanced model MOD is relocated.
[0054] Specifically, further judgments are made on each semantic area. These judgments include overlap judgment (ensuring that the MOD does not overlap too much with existing objects), position judgment (determining the reasonable placement of the MOD) and logic judgment (ensuring that the addition of the MOD complies with logic and scene requirements). Through these judgments, a decision can be made about where and how to add the MOD. Based on this decision, the MOD is added to the corresponding position in the image to be processed to generate new image data. The MOD can be any form of graphics, animation or video clips used to enhance or modify the content of the original image, which is not limited here.
[0055] In one possible implementation, it is determined whether the enhancement model MOD matches the overall theme and background of the image to be processed; whether the enhancement model MOD conforms to common sense and physical laws; and whether the enhancement model MOD can enhance the amount of information or the expression effect of the image to be processed.
[0056] In a possible implementation, if the image to be processed is a continuous frame image, the dynamic effect of the enhanced model MOD and the degree of synchronization with the image to be processed are determined.
[0057] Through the above process, the present invention first trains the image segmentation model through the image data set to obtain the trained segmentation model, obtains the image to be processed, performs semantic segmentation on the image to be processed through the segmentation model, obtains the category prediction map of each pixel in the image to be processed, performs semantic recognition according to the category prediction map of each pixel, obtains multiple semantic regions of the image to be processed, adds a corresponding enhancement model MOD to each semantic region and performs overlapping, position and logical judgment, adjusts the enhancement model MOD according to the judgment result to generate image data, and realizes intelligent processing and modification of image and video content by integrating image segmentation, region recognition and MOD addition, which not only ensures the authenticity of the generated data, but also enhances the rationality and logic of the data, thereby effectively solving the problems of high technical threshold, insufficient authenticity and diversity of the generated data, target overlap, position error and logical disorder in the prior art.
[0058] In one application scenario, Figure 4 A flowchart showing the data generation method.
[0059] like Figure 4As shown in the figure, the user prepares the event-related Mod library (such as vehicles, flames, smoke, car accident scenes, etc.) in advance, inputs a picture to be generated or a video into the segmentation model SegMode l, obtains all the segmentation results of the original picture and obtains the generated area: people, cars, buildings, trees, streets, etc. and records them, and then adds the Mod to the specified location after area judgment, overlap judgment, position judgment, and logical judgment, and finally generates pictures and videos, in which the dynamic MOD is added according to the frame rate to obtain a dynamic effect.
[0060] Among them, area judgment is for the generated street area, building area, green area, vehicles, pedestrians, non-motor vehicles, etc.; overlap judgment is based on area and MOD judgment, and the overlap between MOD and vehicles, pedestrians, and non-motor vehicles is not allowed to be greater than 0.3; position judgment refers to the position and size of data generation referring to existing targets (try to conform to the actual situation; logical judgment refers to the rationality of data generation, for example, traffic incidents should occur next to roads or green belts, but not on buildings or other areas.
[0061] Specifically, overlap judgment is to evaluate the degree of spatial overlap between the MOD to be added (such as vehicles, flames, smoke, etc.) and the objects already existing in the image to be processed (such as vehicles, pedestrians, non-motor vehicles, etc.). The purpose of this step is to ensure that the addition of the MOD does not excessively block or affect the visibility and recognition of the original objects in the image. In specific implementation, the degree of overlap can be quantified by calculating the ratio of the intersection area to the union area of the bounding box of the MOD and the bounding boxes of other objects in the image (i.e., the intersection-over-union ratio IoU). If the overlap exceeds a preset threshold (such as 0.3), it is considered that the placement of the MOD is inappropriate and needs to be adjusted or reselected.
[0062] Furthermore, position judgment is to determine the best placement position of the MOD in the image to be processed. This step needs to consider multiple factors, including the size, shape, direction of the MOD and its coordination with the surrounding environment. First, based on the results of image segmentation, semantic areas suitable for placing the MOD, such as streets, vacant lots, green belts, etc., can be identified. Then, based on the size and position of existing objects (such as vehicles and pedestrians), as well as the size and proportion of the MOD itself, the reasonable placement position of the MOD is calculated. In addition, the orientation and angle of the MOD need to be considered to ensure that it maintains a consistent perspective relationship and visual coherence with other elements in the image.
[0063] Furthermore, logical judgment is to evaluate whether the addition of MOD meets the logic and scene requirements. This step is based on the overall understanding and analysis of the image content to judge whether the addition of MOD will make the image content more reasonable, real or interesting. When implementing it specifically, the following aspects need to be considered: First, the addition of MOD should be coordinated with the theme and background of the image, and no abrupt or illogical elements should appear; second, the addition of MOD should be in line with common sense and physical laws, such as vehicles should be driving on the road, not suspended in the air; finally, the addition of MOD should be able to enhance the information content or expression effect of the image, such as highlighting the sense of urgency of the fire scene by adding smoke effects. Through logical judgment, it can be ensured that the addition of MOD is not only beautiful but also meaningful.
[0064] like Figure 5 As shown in the figure, the goal of semantic understanding based on the image segmentation model is to obtain the segmentation result of the original image by training a large model based on Transformer, that is, semantic-level understanding, which can output the semantic information corresponding to each pixel, such as roads, vehicles, trees, people, buildings, sidewalks, zebra crossings, etc.
[0065] Specifically, in the training phase: collect and use open source image segmentation datasets (such as COCO / ADE20K / PASCAL, etc.) to train the segmentation model SegMode l, obtain the features of each image by batch sampling each image in the dataset and inputting it into the VIT network, and then obtain the final output feature map through the upsamp le network, use BCELOSS to calculate the loss for each pixel and update the network, set the training epoch to 200 rounds, and obtain the segmentation model SegModel for understanding the semantics of the image.
[0066] Furthermore, in the usage stage, the user inputs a picture, the network predicts the category corresponding to each pixel, and saves it as a file for subsequent task judgment.
[0067] Among them, VIT (Vision Transformer) refers to an image recognition method. VIT replaces the traditional convolutional neural network (CNN) with the Transformer architecture, achieving significant performance improvements in image recognition tasks. It regards the image as a series of fixed-size patches, which are then expanded and converted into 1D vectors and processed through the Transformer encoder to capture the global context and dependencies of the entire image.
[0068] Through the above process, an embodiment of the present invention provides a data generation method based on a large image segmentation model. After training an image segmentation model to semantically understand the original data, MOD is used to generate data in combination with area, overlap, position, and logical judgment. By integrating image segmentation, area recognition, and MOD addition, intelligent processing and modification of image and video content are achieved, which not only ensures the authenticity of the generated data, but also enhances the rationality and logic of the data, thereby effectively solving the problems of high technical threshold, insufficient authenticity and diversity of generated data, target overlap, position errors, and logical confusion in the prior art.
[0069] The following is an embodiment of the device of the present invention, which can be used to execute the data generation method involved in the present invention. For details not disclosed in the embodiment of the device of the present invention, please refer to the method embodiment of the data generation method involved in the present invention.
[0070] See also Figure 6 , an embodiment of the present invention provides a data generating device 800.
[0071] The device 800 includes but is not limited to: a segmentation model training module 810, a semantic segmentation module 830, a semantic recognition module 850 and an image generation module 870.
[0072] The segmentation model training module 810 is used to train the large image segmentation model using the image data set to obtain a trained segmentation model.
[0073] The semantic segmentation module 830 is used to obtain the image to be processed, perform semantic segmentation on the image to be processed through a segmentation model, and obtain a category prediction map of each pixel in the image to be processed.
[0074] The semantic recognition module 850 is used to perform semantic recognition according to the category prediction map of each pixel point to obtain multiple semantic regions of the image to be processed.
[0075] The image generation module 870 is used to add a corresponding enhancement model MOD to each semantic area and perform overlapping, position and logic judgment, and adjust the enhancement model MOD to generate image data according to the judgment result.
[0076] It should be noted that, when the data is generated in the above-mentioned embodiment, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the data generating device will be divided into different functional modules to complete all or part of the functions described above.
[0077] In addition, the data generation device and data generation method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs the operation has been described in detail in the method embodiment and will not be repeated here.
[0078] Figure 7 The present invention schematically illustrates the structure of an electronic device according to an exemplary embodiment.
[0079] It should be noted that the electronic device is only an example adapted to the present invention and cannot be considered to provide any limitation on the scope of use of the present invention. The electronic device cannot be interpreted as requiring dependence on or having Figure 7 One or more components of exemplary electronic device 2000 are shown.
[0080] The hardware structure of the electronic device 2000 may vary greatly due to differences in configuration or performance, such as Figure 7 As shown, the electronic device 2000 includes: a power supply 210, an interface 230, at least one memory 250, and at least one central processing unit (CPU) 270.
[0081] Specifically, the power supply 210 is used to provide operating voltage for each hardware device on the electronic device 2000 .
[0082] The interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. Of course, in other examples adapted by the present invention, the interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input-output interface 235, and at least one USB interface 237, etc. Figure 7 As shown, this is not intended to be a specific limitation.
[0083] The memory 250 is a carrier for storing resources, which may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 251, an application 253 and data 255, etc. The storage method may be temporary storage or permanent storage.
[0084] Among them, the operating system 251 is used to manage and control various hardware devices and application programs 253 on the electronic device 2000 to enable the central processing unit 270 to calculate and process the massive data 255 in the memory 250. It can be Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
[0085] The application program 253 is a computer-readable instruction that performs at least one specific task based on the operating system 251, and may include at least one module ( Figure 7 (not shown), each module may respectively include computer-readable instructions for the electronic device 2000. For example, the data generating device may be regarded as an application 253 deployed on the electronic device 2000.
[0086] The data 255 may be signal information, etc., stored in the memory 250 .
[0087] The central processor 270 may include one or more processors and is configured to communicate with the memory 250 through at least one communication bus to read the computer-readable instructions stored in the memory 250, thereby realizing the operation and processing of the mass data 255 in the memory 250. For example, the data generation method is completed by the central processor 270 reading a series of computer-readable instructions stored in the memory 250.
[0088] In addition, the present invention can also be implemented by hardware circuits or hardware circuits combined with software. Therefore, the implementation of the present invention is not limited to any specific hardware circuits, software, or a combination of the two.
[0089] See also Figure 8 In an embodiment of the present invention, an electronic device 4000 is provided. The electronic device 4000 may include: a desktop computer, a laptop computer, a server, etc. with sensor recognition capability.
[0090] exist Figure 8 In the embodiment, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.
[0091] The data interaction between the processor 4001 and the memory 4003 can be realized through at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0092] Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception, etc. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.
[0093] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of the present invention. Processor 4001 may also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.
[0094] The memory 4003 can be a ROM (Read On Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrical Erasable Programmable Read On Memory), a CD-ROM (Compact Disk Read On Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program instructions or codes in the form of instructions or data structures and can be accessed by the electronic device 4000, but is not limited to this.
[0095] Computer-readable instructions are stored in the memory 4003 , and the processor 4001 can read the computer-readable instructions stored in the memory 4003 through the communication bus 4002 .
[0096] The computer-readable instructions are executed by one or more processors 4001 to implement the data generation method in the above-mentioned embodiments.
[0097] In addition, an embodiment of the present invention provides a storage medium on which computer-readable instructions are stored. The computer-readable instructions are executed by one or more processors to implement the data generation method as described above.
[0098] A computer program product is provided in an embodiment of the present invention. The computer program product includes computer-readable instructions. The computer-readable instructions are stored in a storage medium. One or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements the data generation method as described above.
[0099] Compared with the related art, the present invention has the following beneficial effects:
[0100] 1. The present invention first trains an image segmentation large model through an image data set to obtain a trained segmentation model, obtains an image to be processed, performs semantic segmentation on the image to be processed through the segmentation model, obtains a category prediction map of each pixel in the image to be processed, performs semantic recognition according to the category prediction map of each pixel, obtains multiple semantic regions of the image to be processed, adds a corresponding enhancement model MOD to each semantic region and performs overlapping, position and logical judgment, adjusts the enhancement model MOD according to the judgment result to generate image data, and realizes intelligent processing and modification of image and video content by integrating image segmentation, region recognition and MOD addition, which not only ensures the authenticity of the generated data, but also enhances the rationality and logic of the data, thereby effectively solving the problems of high technical threshold, insufficient authenticity and diversity of the generated data, target overlap, position error and logical disorder in the prior art.
[0101] 2. The present invention combines the semantic understanding ability of the image segmentation model to perform detailed regional division and semantic annotation on the original image. Then, based on this semantic information, specific rules (such as area, overlap, position, logical judgment) are designed to guide MOD (which may refer to a model-driven generation process or method, and the specific name may vary depending on the context) to generate new data. This method not only ensures the authenticity of the generated data (that is, the generated image is visually close to the real world), but also enhances the rationality and logic of the data (that is, the generated image content is semantically and logically in line with expectations).
[0102] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.
[0103] The above descriptions are only partial implementations of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.
Claims
1. A data generation method, characterized in that: The method comprises: The image segmentation model is trained by using the image data set to obtain a trained segmentation model; Acquire an image to be processed, perform semantic segmentation on the image to be processed using the segmentation model, and obtain a category prediction map for each pixel in the image to be processed; Performing semantic recognition according to the category prediction map of each pixel point to obtain multiple semantic regions of the image to be processed; A corresponding enhancement model MOD is added to each of the semantic regions and overlap, position and logic judgments are performed, and the enhancement model MOD is adjusted according to the judgment results to generate image data.
2. The data generation method according to claim 1, characterized in that: The image segmentation model is trained by using the image data set to obtain a trained segmentation model, including: Obtain image segmentation datasets and test sets, use the visual transformer VIT as a feature extractor, and build a large image segmentation model in combination with a downsampling network; Optimizing the image segmentation model parameters through iterative training of the image segmentation dataset; The performance of the large image segmentation model is evaluated through the test set, and the trained model weights and configuration files are saved to obtain a trained segmentation model.
3. The data generation method according to claim 1, characterized in that: The adding a corresponding enhancement model MOD to each of the semantic regions and performing overlapping, position and logic judgment to obtain a judgment result, and adjusting the enhancement model MOD to generate image data according to the judgment result, includes: According to the type, size and position of each semantic area, the size, proportion, position, orientation and angle of the corresponding enhanced model MOD are set; Performing overlap judgment for each of the semantic regions and its corresponding enhanced model MOD to obtain an overlap degree, and if the overlap degree exceeds a set threshold, resetting the position of the enhanced model MOD; Performing position judgment for each of the semantic regions and its corresponding enhanced model MOD to obtain a coordination degree, and if the coordination degree does not meet a set condition, resetting the orientation and angle of the enhanced model MOD; A logical judgment is performed for each of the semantic regions and its corresponding enhanced model MOD to obtain a degree of authenticity. If the degree of authenticity does not meet a set condition, the enhanced model MOD is relocated.
4. The data generation method according to claim 3, characterized in that: The adding a corresponding enhancement model MOD to each semantic region to perform overlap judgment to obtain an overlap degree includes: The overlap degree is obtained by calculating the ratio of the intersection area to the union area of the bounding box of the enhanced model MOD and the bounding boxes of each semantic region.
5. The data generation method according to claim 3, characterized in that: The adding a corresponding enhancement model MOD to each semantic region to perform position judgment and obtain coordination degree includes: The coordination degree is obtained according to the matching degree between the size, shape, direction, orientation, angle, size and proportion of the enhanced model MOD and each of the semantic areas.
6. The data generation method according to claim 3, characterized in that: The adding of the corresponding enhancement model MOD to each semantic region to perform logical judgment to obtain the authenticity includes: Determine whether the enhanced model MOD matches the overall subject and background of the image to be processed; Determine whether the enhanced model MOD conforms to common sense and physical laws; It is determined whether the enhancement model MOD can enhance the information content or expression effect of the image to be processed.
7. The data generation method according to claim 1, characterized in that: The method further comprises: If the image to be processed is a continuous frame image, the dynamic effect of the enhanced model MOD and the degree of synchronization with the image to be processed are determined.
8. A data generating device, characterized in that: The device comprises: The segmentation model training module is used to train the image segmentation model through the image data set to obtain a trained segmentation model; A semantic segmentation module is used to obtain an image to be processed, perform semantic segmentation on the image to be processed using the segmentation model, and obtain a category prediction map of each pixel in the image to be processed; A semantic recognition module, used for performing semantic recognition according to the category prediction map of each pixel point to obtain multiple semantic regions of the image to be processed; The image generation module is used to add a corresponding enhancement model MOD to each of the semantic areas and perform overlapping, position and logic judgments, and adjust the enhancement model MOD to generate image data according to the judgment results.
9. An electronic device, characterized in that: include: at least one processor and at least one memory, wherein: The memory has computer-readable instructions stored thereon; The computer-readable instructions are executed by one or more of the processors so that the electronic device implements the data generating method according to any one of claims 1 to 7.
10. A storage medium having computer-readable instructions stored thereon, characterized in that: The computer-readable instructions are executed by one or more processors to implement the data generating method according to any one of claims 1 to 7.
