Virtual material number synthesis method and device, equipment and storage medium

Abstract

The invention discloses a virtual material number synthesis method and device, equipment and a storage medium, and relates to the field of image processing. Selecting a contour image of the material plate and a semantic image corresponding to the contour image from the design database; labeling at least one target contour object in the contour image, and obtaining use case data of the target contour object; importing the marked contour image, semantic image and use case data into a material number synthesis model; and the material number synthesis model updates a target contour object in the semantic image and / or the contour image based on processing information in the use case data, and generates a virtual material number graph in combination with the contour position features and semantic features in the semantic image. According to the scheme, the color virtual material number graph meeting the design requirement is generated by designing the use case data and combining the contour image and the semantic image, so that the purpose of fully expanding the material plate image is achieved, massive defect use case images are provided for later defect detection, and the defect detection precision and the detection efficiency can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a virtual material number synthesis method, device, equipment and storage medium. Background Art

[0002] In the field of integrated circuits, defect detection of material boards is an essential inspection item, focusing on testing the pass rate of produced materials and screening, removing, or repairing defective materials. Most defect detection methods use CAM image matching and recognition methods. Some advanced technologies use AI model training to identify defects in material board images, but AI models are highly dependent on the recognition capabilities of the model structure and the training data set. Although this can be compensated by augmenting the training data set, the data set only serves as a data set by screening images with defects based on CAM images. This type of trained model is insensitive to slight changes in the same type of material board, resulting in poor recognition accuracy. For example, slight offsets of the gold surface, gold wire, or the material board as a whole, slight rotations, and slight changes in size cannot be identified, and the final accuracy of the material board production does not meet the standards. Summary of the Invention

[0003] The present application provides a virtual material number synthesis method, device, equipment and storage medium, which constructs a variety of virtual material number plates with possible slight changes through the material master map, and uses this as a basis to generate defect images, thereby expanding the richness of the training set and solving the problem of substandard model detection accuracy.

[0004] In one aspect, the present application provides a method for synthesizing a virtual material number, the method comprising: Selecting an outline image of a material board and a semantic image corresponding to the outline image from a design database; marking at least one target contour object in the contour image and obtaining use case data for the target contour object; the use case data includes processing information for the selected target contour object; The annotated contour image, semantic image, and use case data are imported into a material number synthesis model; the material number synthesis model updates the target contour objects in the semantic image and / or contour image based on the processing information in the use case data, and generates a virtual material number map by combining the contour position features and the semantic features in the semantic image.

[0005] Specifically, the material number synthesis model is generated based on image training in a material graph training set, a semantic graph training set, and a contour graph training set; A mapping matching relationship is established among the images in the material map training set, the semantic map training set, and the contour map training set; and the material map training set contains actual material maps of various material material information, the semantic map training set contains semantic maps corresponding to all actual material maps, and the contour map training set contains contour maps corresponding to all actual material maps.

[0006] Specifically, the process of training and generating the material number synthesis model includes: Marking the target area in the actual material image with a corresponding texture material label; Aligning the labeled actual material image with the corresponding semantic image and contour image, and mapping the labeled target area to the semantic image and contour image; Extracting the target contour area and target semantic area corresponding to the target area from the semantic image and the contour image respectively; The pixel content of the target area in the actual material image is used as supervision, and the target contour area and the labeled target semantic area are used as sample inputs to extract the contour position features in the contour image and the semantic features in the semantic image to train the material number synthesis model; the target contour area is used to locate the semantic image and define the output size during the supervision process.

[0007] Specifically, the loss function of training the material number synthesis model is It is expressed as follows:

[0008] Among them and represents the loss weight, represents pixel-level loss, represents the label classification loss; Indicates the image size, represents the real pixel value, Represents the model's predicted pixel value; represents the number of samples, Indicates the The true labels of samples, Represents the model prediction label.

[0009] Specifically, the material number synthesis model determines a target operation image according to the type of processing information in the use case data; the target operation image is a contour image or a semantic image; The processing information includes contour data and texture material data, wherein the contour data includes size and posture information of at least one target contour object, and the texture material data includes texture information and material information of at least one target contour object.

[0010] Specifically, when the target operation image is a contour image, updating the target contour object in the contour image based on the processing information in the use case data includes: Mapping the target contour object to a semantic image to determine a target adjustment area and a corresponding texture material label; Adjusting the target contour object according to the size and position information of the target contour object to generate an adjusted contour map; the texture material label of the target contour object remains unchanged before and after the adjustment; wherein the size and position information includes at least one of size, offset, angle, shift, increase or decrease, and scaling ratio, and performing size adjustment, coordinate offset adjustment, shift adjustment, angle adjustment, increase or decrease of the target contour object, or scaling adjustment on the target contour object; The adjustment contour image is remapped into the semantic image, and the semantic pixel values ​​of the target adjustment area in the semantic image are filled and updated according to the contour position and texture material label in the adjustment contour image.

[0011] Specifically, when the target operation image is a semantic image, updating the semantic image based on the processing information in the use case data includes: Mapping the target contour object to a semantic image to determine a target adjustment area and a corresponding texture material label; According to the size and position information of the target contour object, the texture information and / or material information of the target adjustment area is modified, a new texture material label is regenerated, and position matching is performed with the target contour object in the contour image.

[0012] On the other hand, the present application provides a virtual material number synthesis device, the device comprising: An acquisition module, configured to select a contour image of a material board and a semantic image corresponding to the contour image from a design database; a marking module, configured to mark at least one target contour object in the contour image and obtain use case data for the target contour object; the use case data includes processing information for the selected target contour object; A synthesis module is used to import the annotated contour image, semantic image, and use case data into a material number synthesis model; the material number synthesis model updates the target contour objects in the semantic image and / or contour image based on the processing information in the use case data, and generates a virtual material number map by combining the contour position features and the semantic features in the semantic image.

[0013] On the other hand, the present application provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the virtual material number synthesis method described in the above aspect.

[0014] On the other hand, the present application provides a computer-readable storage medium, which stores at least one instruction, at least one program, code set or instruction set. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the virtual material number synthesis method described in the above aspect.

[0015] The beneficial effects brought about by the technical solution provided by the embodiment of the present application include at least the following: the present application establishes a design database of material contour maps and semantic maps, and determines the targets that need to be designed and modified by marking the target contour objects in the contour image, and designs the corresponding use case data. By feeding them into the imported material number synthesis model, the contour position features in the contour map and the semantic features in the semantic image are fully identified and extracted, and the semantic image and contour image are processed in combination with the processing information in the use case data to generate a color virtual material number map that meets the design requirements, so as to achieve the purpose of fully expanding the material board image. Providing a large number of defect use case images for later defect detection can effectively improve the defect detection accuracy and detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of a virtual material number synthesis method provided in an embodiment of the present application; Figure 2 A schematic diagram showing a possible form of a selected contour image and a corresponding semantic image is shown; Figure 3 A color virtual material number diagram of a possible model output is shown; Figure 4 A flow chart of a method for training a material number synthesis model is shown; Figure 5 A schematic diagram showing the offset processing, angle rotation processing, and size enlargement or reduction processing of the target metal surface in the actual material image; Figure 6 Schematic diagram for generating adjustment contour map and semantic map for enlarging the circular gold surface; Figure 7 This example shows how to perform compound modification on a material diagram to generate a virtual material number diagram. Figure 8 The following is a structural block diagram of a virtual material number synthesis device provided in an embodiment of the present application; Figure 9A structural block diagram of a computer device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0017] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0018] In this document, "plurality" refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0019] Figure 1 Flowchart of the virtual material number synthesis method provided in an embodiment of the present application, comprising the following steps: S1. Selecting a contour image of a material board and a semantic image corresponding to the contour image from a design database; The design database mentioned in this embodiment is a database specifically used to store Gerber files and semantic images for global or local designs of integrated circuit boards. Gerber files specifically contain board configuration parameters, drill data, physical properties, and layer layout information for printed circuit boards / boards. In actual production, Gerber files play a crucial role, including but not limited to providing accurate manufacturing information, guiding production and manufacturing, ensuring design confidentiality, and facilitating supply chain collaboration.

[0020] In this embodiment, numerous Geber file data constitute a design database. Since it is for defect detection of material boards, this embodiment mainly uses various components, texts, chips and wiring outline information as the utilization objects, which are expressed as line outlines in the CAD template drawing, so it is called a contour image.

[0021] Semantic images mainly serve AI models. The (semantic) pixel values ​​in the semantic images are obtained based on the panoramic image or local high-definition image of the material board taken by a color camera, and are displayed as grayscale images after grayscale processing. The semantic pixels in them can be used for AI model analysis, such as pixel-level image recognition technology, which is particularly used to identify components, chips and wiring conditions in this embodiment.

[0022] In this embodiment, the design database should contain contour images and semantic images of a large number of material palettes. To facilitate network model processing and virtual image synthesis, the database can be designed as two image sets, and a matching relationship should be established between the contour images and the semantic images. When a contour image is selected, the matching semantic image is directly selected, and vice versa.

[0023] S2. Annotate at least one target contour object in the contour image and obtain use case data for the target contour object; Figure 2 A schematic diagram of a selected contour image and a corresponding semantic image in one possible form is shown, where their sizes match each other, and the line contour objects in the contour image correspond one-to-one to the pixel areas in the semantic image. Annotating the target contour objects in the contour image and designing use case data are essentially for inputting design requirements. The contour image is used as the annotation object here, mainly to clearly distinguish between components and contours. In the case of a large material board with complex multi-layer wiring, the semantic image cannot be accurately positioned by relying solely on the distinction between different grayscales, especially complex wiring diagrams. After annotating specific target contour objects in the contour image, the area of ​​the semantic image can actually be determined based on the mapping relationship.

[0024] The use case data is the design requirement for (virtual) generating a material number image, which can be a specific programmatic file that records the description of the target contour object. Note that at least one target contour object can refer to a component, text, circuit, or the entire carrier board in the image. For example Figure 2 The circular gold surface outline indicated by the arrow on the right side of the outline diagram (which can also be bolded or framed) is the marked target outline object. The description in the use case data then represents the operation performed on this circular gold surface. For example, if you create multiple copies of the circular gold surface and place them in designated locations, the generated virtual part number diagram will display a color image containing multiple circular gold surfaces.

[0025] Another important use of generating virtual part number diagrams using use case data is to synthesize a large number of use case images to represent different types of material boards. For example, a version upgrade or iterative update of a material board may show changes in the number, position, shape, and material of some metal surfaces, components, and circuits on the carrier board. Using use case data and a design database, a material template diagram can be quickly synthesized. For a specific material board, this quickly synthesized material template diagram can serve as a massive set of defect images for defect detection. Crucially, use case data is used to construct a variety of virtual part number boards with potentially minor variations, which serve as the basis for generating defect images and defect detection.

[0026] S3. Import the annotated contour image, semantic image, and use case data into the material number synthesis model; the material number synthesis model updates the target contour objects in the semantic image and / or contour image based on the processing information in the use case data, and generates a virtual material number map by combining the contour position features and the semantic features in the semantic image.

[0027] The material number synthesis model is an AI model designed by the embodiment specifically for generating color virtual material number images. The contour image and semantic image are both non-color images. This is why the AI ​​model is used. It can arbitrarily design use case data based on a small amount of images to synthesize a large number of homologous but different material board images, providing massive data for subsequent material board defect detection.

[0028] Note that the "color virtual material number map" here does not render it into a specific color, but generates the same effect as in the actual defect detection scene, as captured by a panoramic camera or high-definition camera, such as Figure 3 This image shows a possible output of a color virtual part number for replicating a circular gold surface. Multiple replicas of the circular gold surface are distributed near the right side of the image. Because actual defect detection involves both dirt and color detection, high-precision color cameras are typically used. Different colors represent different textures and materials, which are key areas of focus during defect detection.

[0029] During the initial training phase, the material number synthesis model uses actual color material plate images as supervision. Based on the position of the input contour image and the semantic pixel content of the semantic image, the model is trained and regressed by extracting contour position features and semantic features. Therefore, upon inputting use case data requirements, the trained model parameters accurately extract and analyze contour position features and semantic features, outputting a virtual material number map that meets the requirements. Figure 3 In the color virtual part number image shown, multiple replicated circular gold surfaces are distributed near the right side of the image. Of course, in some embodiments, all wafers, components, and circuits in the entire image can be rotated or offset before output, simulating the deviation caused by component placement or etching during the material board production process, and using this as a defect image to assist in later high-precision defect detection.

[0030] In summary, the embodiment of the present application establishes a design database of material contour images and semantic images, and determines the targets that need to be designed and modified by marking the target contour objects in the contour image, and designs the corresponding use case data. By feeding them into the imported material number synthesis model, the contour position features in the contour image and the semantic features in the semantic image are fully identified and extracted, and the semantic image and contour image are processed in combination with the processing information in the use case data to generate a color virtual material number map that meets the design requirements, so as to achieve the purpose of fully expanding the material board image. Providing a large number of defect use case images for later defect detection can effectively improve the defect detection accuracy and detection efficiency.

[0031] Because a virtual material number map that conforms to the use case data is generated by importing the material number synthesis model, the function and accuracy of the material number synthesis model are crucial to the output result. For this reason, the present application embodiment provides a training method for generating a material number synthesis model. Figure 4 As shown, the specific steps include: Step 401: label the target area in the actual material image with the corresponding texture material label; The material number synthesis model is generated based on image training from a material image training set, a semantic image training set, and a contour image training set. The material image training set contains actual material images with various material information; the semantic image training set contains semantic images corresponding to all actual material images; and the contour image training set contains contour images corresponding to all actual material images. Correspondingly, all images in the material image training set, the semantic image training set, and the contour image training set are mapped and matched. Selecting an image from one set also selects images from the other two sets.

[0032] The training phase relies on labeling images, so manual annotation is required early on. Texture material labels are used to record the texture and material information of various component circuits in color actual material images, such as textured gold-plated, tin-plated, and chrome-plated surfaces, as well as gold or copper wires adapted to different functions. Texture information can include the color, texture, and shading characteristics of a specific material.

[0033] In this step, the manually annotated texture material labels are mainly determined based on the contour map. The actual material map is determined by contour map mapping, and then the texture material labels are annotated based on the mapped target area and used as a model verification set.

[0034] Step 402: align the labeled actual material graph with the corresponding semantic graph and contour graph, and map the labeled target area to the semantic graph and contour graph; This step mainly establishes a mapping relationship between the actual material map, semantic map and contour map, and determines the correspondence between the actually selected target object, target contour and target area in the entire image size to facilitate quick matching.

[0035] Step 403, extracting a target contour area and a target semantic area corresponding to the target area from the semantic image and the contour image respectively; In step 404, the pixel content of the target area in the actual material image is used as supervision, the target contour area and the labeled target semantic area are used as sample input, the contour position features in the contour image and the semantic features in the semantic image are extracted, and the material number synthesis model is trained.

[0036] When building an AI model, the loss structure type of the model should be given priority. Because the training is based on the contour map and semantic map, and the verification is based on the actual material map, it should be considered from two dimensions: the target contour position and the semantic pixel value. The label loss based on position recognition and the label loss based on pixel-level recognition should be constructed. The loss function It is expressed as follows:

[0037] Among them and represents the loss weight, represents pixel-level loss, represents the label classification loss; Indicates the image size, represents the real pixel value, Represents the model's predicted pixel value; represents the number of samples, Indicates the The true labels of samples, Represents the model prediction label.

[0038] By comparing the predicted label type with the actual label type, the material number synthesis model is obtained after cross-validation.

[0039] In some embodiments, after acquiring the use case data, the material number synthesis model further analyzes and determines the use case data. Specifically, the material number synthesis model determines the target operation image based on the type of information processed in the use case data. As previously mentioned, because the input image is divided into contour images and semantic images, and although the use case data is set based on the contour annotation of at least one target contour object, its primary target can be either contour images or semantic images.

[0040] In this embodiment, the processed information may include contour data and texture material data. The contour data includes size and position information of at least one target contour object, and primarily acts on the contour image. The texture material data includes texture information and material information of at least one target contour object, and primarily acts on the semantic image, specifically, the pixel region that matches the target contour object.

[0041] The following discusses contour images and semantic images separately.

[0042] 1. When the target operation image is a contour image, the process of updating the target contour object in the contour image based on the processing information in the use case data is summarized as follows: 1) Map the target contour object to the semantic image and determine the target adjustment area and the corresponding texture material label; 2) Adjust the target contour object according to its size and posture information to generate an adjusted contour map; The specific process needs to be determined according to the adjustment requirements, and the texture material label of the target contour object remains unchanged before and after the adjustment. This is the key to generating a qualified color virtual image.

[0043] In this embodiment, the size and posture information includes but is not limited to size, offset, angle, shift, increase or decrease, and scaling ratio. The corresponding operations are to adjust the size, coordinate offset, shift, angle, increase or decrease the target contour object, or scale the target contour object.

[0044] Figure 5 The diagram shows how to perform offset processing, angle rotation processing, and size enlargement or reduction processing on the target metal surface in the actual material image. In actual operation, it is also possible to shift, rotate, and scale up or down the content of multiple or the entire image.

[0045] Of course, in some embodiments, there are situations where text, components, etc. are added or deleted in a specific area (such as a carrier board). In this case, the target outline object can represent the carrier board, and the corresponding operation can be performed in combination with the added or deleted position coordinate information.

[0046] The contour adjustment is based on the original contour image, which is an operation to adjust the target contour line according to the needs. Figure 2 The outline in the figure is enlarged to illustrate the scale. Figure 6 This is a schematic diagram of how to magnify a circular gold face to generate an adjusted contour map and semantic map. The arrows in the left and right figures indicate the display effect after the magnification operation.

[0047] 3) Remap the adjusted contour image to the semantic image, and fill and update the semantic pixel values ​​of the target adjustment area in the semantic image according to the contour position and texture material label in the adjusted contour image.

[0048] Because the newly generated target contour changes, the coordinate data is also updated. This is why it is necessary to synchronize the changes to the semantic information. The inspiration of this synchronization process is pixel recognition and contour filling. The changes in the corresponding pixel area of ​​the original target contour are redefined according to the new target contour area, and then the semantic pixel value is filled in according to the original label information. Figure 6 The middle right picture shows the effect.

[0049] 2. When the target operation image is a semantic image, the semantic image is updated based on the processing information in the use case data, including: 1) Map the target contour object to the semantic image and determine the target adjustment area and the corresponding texture material label; 2) According to the size and posture information of the target contour object, the texture information and / or material information of the target adjustment area is modified, a new texture material label is regenerated, and the position is matched with the target contour object in the contour image.

[0050] This step mainly targets changes in materials and textures, such as changing a gold-plated surface to a chrome-plated surface, or changing a copper wire to a gold wire. Such changes are not visible on the contour map, so they can only be located through contours and then modified in the semantic map, that is, updating the semantic pixel values ​​of the target adjustment area.

[0051] Of course, in actual use cases, the size and posture information of multiple targets are often obtained, and multiple modifications to the texture and material are also included to achieve the purpose of generating massive virtual images.

[0052] Figure 7 The schematic diagram of generating a virtual material number image by compositely modifying a material image is shown as an example. The leftmost column shows the original material image, the second column shows the semantic image after grayscale processing, and the third column shows the modified semantic image generated based on the use case data. It can be clearly seen in the re-synthesized virtual material number image on the far right that the texture of the rectangular gold surface in the first row has been changed to a solid color, and the material connecting the bent gold wire to the substrate in the second row has been modified. Similarly, the texture material of the inner ring of the circular crystal surface in the fourth row has been replaced. By comparing the slight modification differences before and after the first and fourth columns, it is possible to simulate local etching defects or errors in the material board. It can also represent the material and size differences of material boards with different material numbers. The defect image training set created by these material images can provide rich detection experience for subsequent defect detection.

[0053] Figure 8 The following is a structural block diagram of a virtual material number synthesis device provided in an embodiment of the present application, wherein the device includes: An acquisition module, configured to select a contour image of a material board and a semantic image corresponding to the contour image from a design database; a marking module, configured to mark at least one target contour object in the contour image and obtain use case data for the target contour object; the use case data includes processing information for the selected target contour object; A synthesis module is used to import the annotated contour image, semantic image, and use case data into a material number synthesis model; the material number synthesis model updates the target contour objects in the semantic image and / or contour image based on the processing information in the use case data, and generates a virtual material number map by combining the contour position features and the semantic features in the semantic image.

[0054] In some embodiments, the material number synthesis model is generated based on image training in a material graph training set, a semantic graph training set, and a contour graph training set; A mapping matching relationship is established among the images in the material map training set, the semantic map training set, and the contour map training set; and the material map training set contains actual material maps of various material material information, the semantic map training set contains semantic maps corresponding to all actual material maps, and the contour map training set contains contour maps corresponding to all actual material maps.

[0055] In some embodiments, the process of training and generating the material number synthesis model includes: Marking the target area in the actual material image with a corresponding texture material label; Aligning the labeled actual material image with the corresponding semantic image and contour image, and mapping the labeled target area to the semantic image and contour image; Extracting the target contour area and target semantic area corresponding to the target area from the semantic image and the contour image respectively; The pixel content of the target area in the actual material image is used as supervision, and the target contour area and the labeled target semantic area are used as sample inputs to extract the contour position features in the contour image and the semantic features in the semantic image to train the material number synthesis model; the target contour area is used to locate the semantic image and define the output size during the supervision process.

[0056] In some embodiments, the loss function for training the material number synthesis model is It is expressed as follows:

[0057] Among them and represents the loss weight, represents pixel-level loss, represents the label classification loss; Indicates the image size, represents the real pixel value, Represents the model's predicted pixel value; represents the number of samples, Indicates the The true labels of samples, Represents the model prediction label.

[0058] In some embodiments, the material number synthesis model determines a target operation image according to the type of processing information in the use case data; the target operation image is a contour image or a semantic image; The processing information includes contour data and texture material data, wherein the contour data includes size and posture information of at least one target contour object, and the texture material data includes texture information and material information of at least one target contour object.

[0059] In some embodiments, when the target operation image is a contour image, the synthesis module is further configured to: Mapping the target contour object to a semantic image to determine a target adjustment area and a corresponding texture material label; Adjusting the target contour object according to the size and position information of the target contour object to generate an adjusted contour map; the texture material label of the target contour object remains unchanged before and after the adjustment; wherein the size and position information includes at least one of size, offset, angle, shift, increase or decrease, and scaling ratio, and performing size adjustment, coordinate offset adjustment, shift adjustment, angle adjustment, increase or decrease of the target contour object, or scaling adjustment on the target contour object; The adjustment contour image is remapped into the semantic image, and the semantic pixel values ​​of the target adjustment area in the semantic image are filled and updated according to the contour position and texture material label in the adjustment contour image.

[0060] In some embodiments, when the target operation image is a semantic image, the synthesis module is further configured to: Mapping the target contour object to a semantic image to determine a target adjustment area and a corresponding texture material label; According to the size and position information of the target contour object, the texture information and / or material information of the target adjustment area is modified, a new texture material label is regenerated, and position matching is performed with the target contour object in the contour image.

[0061] In summary, the implementation of this plan can bring the following beneficial effects: Quality: Improve detection accuracy: By generating defect images that are close to reality, the model can learn more defect features, reduce missed detections and false detections, and improve product quality. Improve quality stability: Prepare materials and adjust parameters in advance to reduce quality fluctuations in the production process, making the production process more controllable. Cost: Reduce sample collection costs: Reduce reliance on actual defect samples, saving manpower, material resources, and time costs for sample collection; Reduce production preparation costs: Prepare in advance to reduce additional costs caused by delays in physical material supply and improper parameter adjustments.

[0062] Delivery: Shorten delivery cycle: quickly launch material numbers and improve defect detection efficiency, reduce production downtime, and ensure on-time product delivery; Improve delivery flexibility: Quickly adjust production plans and switch material numbers in a timely manner to meet customer order changes.

[0063] Service: Improve customer satisfaction: High-quality products reduce customer complaints, fast delivery and flexible response to customer needs enhance customer experience; Enhance market competitiveness: With advantages in quality, cost and delivery, establish a good brand image, attract more customers and enhance market competitiveness.

[0064] The virtual material number synthesis device provided in the embodiment of the present application can be applied to the virtual material number synthesis method provided in the above embodiment. For relevant details, please refer to the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0065] It should be noted that the virtual material number synthesis device provided in the embodiment of the present application is only illustrated by the division of the above-mentioned functional modules / functional units. In actual applications, the above-mentioned functions can be assigned to different functional modules / functional units as needed, that is, the internal structure of the virtual material number synthesis device is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation method of the virtual material number synthesis method provided in the above method embodiment and the implementation method of the virtual material number synthesis device provided in this embodiment belong to the same concept. The specific implementation process of the virtual material number synthesis device provided in this embodiment is detailed in the above method embodiment and will not be repeated here.

[0066] Figure 9 The following is a block diagram of the structure of a computer device provided by an exemplary embodiment of the present application. The computer device is a desktop computer, a laptop computer, a PDA, a cloud server, and the like. The computer device may include, but is not limited to, a processor and a memory. The processor and the memory may be connected via a bus or other means. The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, graphics processing units (GPU), embedded neural network processors (NPU) or other dedicated deep learning coprocessors, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0067] The processor may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor may be implemented in at least one of the following hardware forms: a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), or a PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is used to process data while awake, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data while in standby mode. In some embodiments, the processor may integrate a graphics processing unit (GPU), which is responsible for rendering and drawing content displayed on the display. In some embodiments, the processor may also include an artificial intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0068] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above-mentioned embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the method in the above-mentioned method embodiment is implemented. The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0069] In some embodiments, the computer device may optionally include a peripheral device interface and at least one peripheral device. The processor, memory, and peripheral device interface may be connected via a bus or signal lines. Each peripheral device may be connected to the peripheral device interface via a bus, signal lines, or circuit boards. Specifically, the peripheral device includes at least one of a radio frequency circuit, a display screen, and a keyboard.

[0070] The peripheral device interface can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor and memory. In some embodiments, the processor, memory, and peripheral device interface are integrated on the same chip or circuit board. In other embodiments, any one or two of the processor, memory, and peripheral device interface can be implemented on separate chips or circuit boards, although this embodiment is not limited to this.

[0071] The display screen is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, or any combination thereof. When the display screen is a touch screen, it also has the ability to capture touch signals on or above the surface of the display screen. The touch signals can be input as control signals to a processor for processing. In this case, the display screen can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, there can be one display screen, disposed on the front panel of the computer device; in other embodiments, there can be at least two display screens, disposed on different surfaces of the computer device or in a foldable design; in still other embodiments, the display screen can be a flexible display screen, disposed on a curved or foldable surface of the computer device. Furthermore, the display screen can be configured as a non-rectangular, irregular shape, also known as a special-shaped screen. The display screen can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0072] A power supply is used to power various components in computer equipment. The power supply can be AC, DC, disposable batteries, or rechargeable batteries. When the power supply includes a rechargeable battery, it can be wired or wirelessly rechargeable. A wired rechargeable battery is charged via a wired line, while a wireless rechargeable battery is charged via a wireless coil. The rechargeable battery can also support fast charging technology.

[0073] Those skilled in the art will understand that the structure shown in this embodiment does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0074] The embodiment of the present application also discloses a computer-readable storage medium. Specifically, the computer-readable storage medium is used to store a computer program, and when the computer program is executed by the processor, the method in the above-mentioned method implementation is implemented. Those skilled in the art will understand that all or part of the processes in the above-mentioned method implementation of the present application can be completed by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the implementation of the above-mentioned methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0075] This specific embodiment is merely an explanation of the present invention and is not intended to limit the present invention. After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed. However, as long as such modifications are within the scope of the claims of the present invention, they are protected by patent law.

Claims

1. A virtual material number synthesis method, characterized in that: The method comprises: Selecting an outline image of a material board and a semantic image corresponding to the outline image from a design database; marking at least one target contour object in the contour image and obtaining use case data for the target contour object; the use case data includes processing information for the selected target contour object; The annotated contour image, semantic image, and use case data are imported into a material number synthesis model; the material number synthesis model updates the target contour objects in the semantic image and / or contour image based on the processing information in the use case data, and generates a virtual material number map by combining the contour position features and the semantic features in the semantic image.

2. The method according to claim 1, characterized in that The material number synthesis model is generated based on image training in the material graph training set, the semantic graph training set, and the contour graph training set; A mapping matching relationship is established among the images in the material map training set, the semantic map training set, and the contour map training set; and the material map training set contains actual material maps of various material material information, the semantic map training set contains semantic maps corresponding to all actual material maps, and the contour map training set contains contour maps corresponding to all actual material maps.

3. The method according to claim 2, characterized in that The process of training and generating the material number synthesis model includes: Marking the target area in the actual material image with a corresponding texture material label; Aligning the labeled actual material image with the corresponding semantic image and contour image, and mapping the labeled target area to the semantic image and contour image; Extracting the target contour area and target semantic area corresponding to the target area from the semantic image and the contour image respectively; The pixel content of the target area in the actual material image is used as supervision, and the target contour area and the labeled target semantic area are used as sample inputs to extract the contour position features in the contour image and the semantic features in the semantic image to train the material number synthesis model; the target contour area is used to locate the semantic image and define the output size during the supervision process.

4. The method according to claim 3, characterized in that The loss function for training the material number synthesis model It is expressed as follows: Among them and represents the loss weight, represents pixel-level loss, represents the label classification loss; Indicates the image size, represents the real pixel value, Represents the model's predicted pixel value; represents the number of samples, Indicates the The true labels of samples, Represents the model prediction label.

5. The method according to any one of claims 1 to 4, characterized in that: The material number synthesis model determines a target operation image according to the type of processing information in the use case data; the target operation image is a contour image or a semantic image; The processing information includes contour data and texture material data, wherein the contour data includes size and posture information of at least one target contour object, and the texture material data includes texture information and material information of at least one target contour object.

6. The method according to claim 5, characterized in that When the target operation image is a contour image, updating the target contour object in the contour image based on the processing information in the use case data includes: Mapping the target contour object to a semantic image to determine a target adjustment area and a corresponding texture material label; Adjusting the target contour object according to the size and position information of the target contour object to generate an adjusted contour map; the texture material label of the target contour object remains unchanged before and after the adjustment; wherein the size and position information includes at least one of size, offset, angle, shift, increase or decrease, and scaling ratio, and performing size adjustment, coordinate offset adjustment, shift adjustment, angle adjustment, increase or decrease of the target contour object, or scaling adjustment on the target contour object; The adjustment contour image is remapped into the semantic image, and the semantic pixel values ​​of the target adjustment area in the semantic image are filled and updated according to the contour position and texture material label in the adjustment contour image.

7. The method according to claim 6, characterized in that When the target operation image is a semantic image, updating the semantic image based on the processing information in the use case data includes: Mapping the target contour object to a semantic image to determine a target adjustment area and a corresponding texture material label; According to the size and position information of the target contour object, the texture information and / or material information of the target adjustment area is modified, a new texture material label is regenerated, and position matching is performed with the target contour object in the contour image.

8. A virtual material number synthesis device, characterized in that: The device comprises: An acquisition module, configured to select a contour image of a material board and a semantic image corresponding to the contour image from a design database; a marking module, configured to mark at least one target contour object in the contour image and obtain use case data for the target contour object; the use case data includes processing information for the selected target contour object; A synthesis module is used to import the annotated contour image, semantic image, and use case data into a material number synthesis model; the material number synthesis model updates the target contour objects in the semantic image and / or contour image based on the processing information in the use case data, and generates a virtual material number map by combining the contour position features and the semantic features in the semantic image.

9. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the virtual material number synthesis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the virtual material number synthesis method as described in any one of claims 1 to 7.

Images