Method and device for generating multi-view industrial design images based on engineering constraints

Abstract

This invention provides a method and apparatus for generating multi-view industrial design images based on engineering constraints, comprising: acquiring a mixing instruction, which includes an image text description of the industrial design image to be generated and engineering parameter constraints of the industrial design image to be generated; inputting the mixing instruction into a pre-trained image generation model to obtain a base reference image containing multiple views output by the image generation model, wherein the image generation model is used to generate a base reference image matching the mixing instruction; performing format standardization processing on the base reference image to obtain a standard reference image matching the base reference image; and readjusting the standard reference image based on the image text description to obtain the industrial design image to be generated, wherein the view images of the industrial design image to be generated have consistency across multiple views. This achieves consistency between multiple views while ensuring that the generated industrial design image meets the engineering parameter constraints.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for generating multi-view industrial design images based on engineering constraints. Background Technology

[0002] The industrial design of rail transit equipment is a crucial step in determining the product's aesthetic features, aerodynamic performance, and brand recognition. The design outcomes are typically presented in the form of technical drawings, including front, side, and top views. Historically, this process has heavily relied on the professional experience and manual labor of designers, requiring the sequential completion of multiple stages such as conceptual sketching, 2D rendering, and 3D CAD modeling, resulting in high labor costs.

[0003] In recent years, with the rapid development of generative artificial intelligence technology, AI-based image generation methods have shown initial application potential in the field of industrial design assistance. However, these technologies often suffer from a lack of consistency across multiple perspectives when facing scenarios with strong engineering constraints, such as rail transit equipment.

[0004] Therefore, finding an efficient and low-cost method for generating industrial design images that ensures a high degree of consistency of design elements across different perspectives has become a current research hotspot. Summary of the Invention

[0005] This invention provides a method and apparatus for generating multi-view industrial design images based on engineering constraints, which enables the generated industrial design images to maintain consistency among multiple perspectives while meeting quantitative engineering parameter constraints.

[0006] This invention provides a method for generating multi-view industrial design images based on engineering constraints. The method includes: acquiring a mixing instruction, wherein the mixing instruction includes an image text description of the industrial design image to be generated and engineering parameter constraints of the industrial design image to be generated, the industrial design image to be generated including view images from multiple perspectives; inputting the mixing instruction into a pre-trained image generation model to obtain a base reference image containing multiple perspectives output by the image generation model, wherein the image generation model is used to generate a base reference image matching the mixing instruction based on the mixing instruction; performing format standardization processing on the base reference image to obtain a standard reference image matching the base reference image; and readjusting the standard reference image based on the image text description to obtain the industrial design image to be generated, wherein the view images from multiple perspectives of the industrial design image to be generated have consistency.

[0007] According to the present invention, a method for generating multi-view industrial design images based on engineering constraints, before performing format standardization processing on the base reference image to obtain a standard reference image matching the base reference image, the method further includes: obtaining a pre-configured target image bounding box, wherein the target image bounding box defines standard position information and standard size information of the view images from each viewpoint in the standard industrial design image; the step of performing format standardization processing on the base reference image to obtain a standard reference image matching the base reference image includes: performing bounding box localization detection on the base reference image to obtain source bounding boxes of view images from each viewpoint in the base reference image; extracting the image content of view images from each viewpoint within the source bounding boxes based on the source bounding boxes; and performing an affine transformation on the image content to the target image bounding box to obtain a standard reference image matching the base reference image.

[0008] According to the present invention, a method for generating multi-view industrial design images based on engineering constraints includes the following steps: performing bounding box localization and detection on the base reference image to obtain the source bounding boxes of the bottom view images in each view of the base reference image; calling a pre-trained target detection model, wherein the target detection model is used to perform bounding box localization and detection on the image; inputting the base reference image into the target detection model to obtain the source bounding boxes of the bottom view images in each view of the base reference image output by the target detection model.

[0009] According to the present invention, a method for generating multi-view industrial design images based on engineering constraints, before the step of readjusting the standard reference image based on the image text description to obtain the industrial design image to be generated, the method further includes: calling a pre-trained ControlNet model, wherein the ControlNet model is used to obtain the industrial design image to be generated based on the image text description; the step of readjusting the standard reference image based on the image text description to obtain the industrial design image to be generated includes: inputting the image text description and the standard reference image into the ControlNet model to obtain the industrial design image to be generated output by the ControlNet model.

[0010] According to the present invention, a method for generating multi-view industrial design images based on engineering constraints includes the following steps: inputting the image text description and the standard reference image into the ControlNet model to obtain the industrial design image to be generated output by the ControlNet model; inputting the image text description and the standard reference image into the ControlNet model; extracting structural information from the standard reference image through the ControlNet model to obtain structural constraints for generating the industrial design image to be generated; and readjusting the standard reference image through the ControlNet model based on the structural constraints and the image text description to obtain the industrial design image to be generated output by the ControlNet model.

[0011] According to the present invention, a multi-view industrial design image generation method based on engineering constraints is provided. The image generation model is trained in the following manner: a training dataset is obtained, wherein the training dataset includes multiple training data, the training data includes industrial design image samples and mixed instruction samples corresponding to the industrial design image samples; the image generation model is trained based on the training dataset to obtain a trained image generation model.

[0012] This invention also provides a multi-view industrial design image generation device based on engineering constraints. The device includes: an acquisition module for acquiring a mixing instruction, wherein the mixing instruction includes an image text description of the industrial design image to be generated and engineering parameter constraints of the industrial design image to be generated, the industrial design image to be generated including view images from multiple perspectives; an acquisition module for inputting the mixing instruction into a pre-trained image generation model to obtain a base reference image containing multiple perspectives output by the image generation model, wherein the image generation model is used to generate a base reference image matching the mixing instruction based on the mixing instruction; a standardization module for performing format standardization processing on the base reference image to obtain a standard reference image matching the base reference image; and a generation module for readjusting the standard reference image based on the image text description to obtain the industrial design image to be generated, wherein the view images from multiple perspectives of the industrial design image to be generated have consistency.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-view industrial design image generation method based on engineering constraints as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-view industrial design image generation method based on engineering constraints as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the multi-view industrial design image generation method based on engineering constraints as described above.

[0016] This invention provides a method and apparatus for generating multi-view industrial design images based on engineering constraints. The method includes: acquiring a mixing instruction, wherein the mixing instruction includes an image text description of the industrial design image to be generated and engineering parameter constraints of the industrial design image to be generated, and the industrial design image to be generated includes view images from multiple perspectives; inputting the mixing instruction into a pre-trained image generation model to obtain a base reference image containing multiple perspectives output by the image generation model, wherein the image generation model is used to generate a base reference image matching the mixing instruction; performing format standardization processing on the base reference image to obtain a standard reference image matching the base reference image; and readjusting the standard reference image based on the image text description to obtain the industrial design image to be generated, wherein the view images from multiple perspectives of the industrial design image to be generated have consistency. This achieves the goal of maintaining consistency among multiple perspectives while ensuring that the generated industrial design image meets the engineering parameter constraints. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the multi-view industrial design image generation method based on engineering constraints provided by the present invention.

[0019] Figure 2 This is a schematic diagram of the process provided by the present invention for performing format standardization processing on the base reference image to obtain a standard reference image that matches the base reference image.

[0020] Figure 3 This is a schematic diagram of the structure of the multi-view industrial design image generation device based on engineering constraints provided by the present invention.

[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] This invention provides a multi-view industrial design image generation method based on engineering constraints, aiming to solve the problems of general AI image generation models struggling to integrate qualitative descriptions and quantitative engineering parameters, as well as inconsistencies across multiple perspectives, in industrial design applications. This method can be widely applied to the conceptual design, engineering iteration, and manufacturing adaptation of rail transit vehicles such as high-speed trains, bullet trains, and subways.

[0024] Figure 1 This is a flowchart illustrating the multi-view industrial design image generation method based on engineering constraints provided by the present invention.

[0025] The following will combine Figure 1 The process of the multi-view industrial design image generation method based on engineering constraints provided by the present invention is described.

[0026] In an exemplary embodiment of the present invention, combined with Figure 1 As can be seen, the multi-view industrial design image generation method based on engineering constraints can include steps 110 to 140, and each step will be described below.

[0027] In step 110, a blending instruction is obtained, wherein the blending instruction includes an image text description of the industrial design image to be generated and engineering parameter constraints of the industrial design image to be generated, and the industrial design image to be generated includes multiple perspective view images.

[0028] In one embodiment, the image text description can be a qualitative description in natural language, such as "a streamlined high-speed train with a bullet-shaped front, windows arranged in a continuous strip, and a white body with a blue waistline." Engineering parameter constraints can be structured or semi-structured quantitative constraints, such as "the front view shows a car body height of 400 pixels and a width of 600 pixels; the side view shows a car body length of 1200 pixels and a height of 350 pixels; the distance from the center of the window to the rail surface is 220 pixels in the front view and 200 pixels in the side view." In one example, engineering parameter constraints can also be input via preset templates, parameter tables, or key-value pair text.

[0029] In step 120, the mixing instructions are input into a pre-trained image generation model to obtain a base reference image containing multiple viewpoints output by the image generation model. The image generation model is used to generate a base reference image that matches the mixing instructions based on the mixing instructions.

[0030] In one embodiment, the acquired complete hybrid instructions can be encapsulated as a model input tensor and input into a pre-trained image generation model. The image generation model can be a general, large-scale image generation model, such as a deep neural network based on a diffusion architecture. In application, end-to-end training can be performed on a large-scale dataset containing multi-view industrial design images and corresponding hybrid instruction annotations, enabling it to simultaneously parse textual semantics and numerical layout constraints, and generate multi-view composite images conforming to the hybrid instructions.

[0031] In step 130, the base reference image is format-normalized to obtain a standard reference image that matches the base reference image.

[0032] In one embodiment, a basic reference image with layout deviations can be converted into a standard reference image whose layout strictly conforms to engineering parameter constraints.

[0033] In one example, a lightweight object detector, such as YOLO or an open-vocabulary detection model, can be used to process the base reference image. The detector uses broad category words such as "train," "front," and "side" as prompts to automatically identify and regress the source bounding box coordinates of the regions in the front and side views of the image. For example, the detector outputs the source bounding box for the front view as [50, 200, 750, 600] and the source bounding box for the side view as [1000, 180, 2100, 530]. Further, the target bounding boxes are read from an engineering parameter library (front view: [0, 312, 800, 712]; side view: [900, 337, 2100, 687]). For the front view, the image content within the source bounding box [50,200,750,600] can be precisely mapped to the target bounding box [0,312,800,712] region through affine transformations, such as scaling and translation; for the side view, the content within the source bounding box [1000,180,2100,530] can be mapped to the target bounding box [900,337,2100,687].

[0034] Based on the aforementioned mapping process, a new image, namely the standard reference image, can be generated. The size, position, and scale of the front view and side view in this image are completely consistent with the engineering parameter constraints, but there may be slight edge burrs or detail blurring due to stretching transformation inside the image.

[0035] In step 140, the standard reference image is readjusted based on the image text description to obtain the industrial design image to be generated, wherein the view images of the industrial design image to be generated from multiple perspectives have consistency.

[0036] In another embodiment, the standard reference image can be readjusted based on the image text description to obtain the industrial design image to be generated. In one example, the standard reference image and the image text description can be input together into a conditional image generation pipeline, such as a ControlNet model. This conditional generation pipeline uses a pre-trained diffusion model and loads low-rank adapter weights fine-tuned for the rail transit domain. During the generation process, the model uses the standard reference image as a structural control condition, for example, extracting its edge contours through the ControlNet model as constraints, and uses the original text description as semantic guidance. While strictly maintaining the view boundaries, component positions, and size ratios defined by the standard reference image, it performs fine-tuned rendering of the image's internal details, textures, lighting transitions, color saturation, etc. After this step, the model outputs the final industrial design image to be generated. This image perfectly matches the image text description and engineering parameter constraints.

[0037] In this embodiment, the diverse and arbitrarily laid-out content from various perspectives in the base reference image is uniformly mapped to target bounding boxes defined by the same set of engineering parameters, thereby obtaining a standard reference image. This operation allows all perspectives to share the same spatial structural skeleton. In the subsequent readjustment and generation stage, the model is rendered using this shared skeleton as a rigid template under the guidance of the same text description. Therefore, the proportions, color attributes, and texture details in any perspective will naturally align with other perspectives.

[0038] This invention provides a method for generating multi-view industrial design images based on engineering constraints, comprising: acquiring a mixing instruction, wherein the mixing instruction includes an image text description of the industrial design image to be generated and engineering parameter constraints of the industrial design image to be generated, and the industrial design image to be generated includes view images from multiple perspectives; inputting the mixing instruction into a pre-trained image generation model to obtain a base reference image containing multiple perspectives output by the image generation model, wherein the image generation model is used to generate a base reference image matching the mixing instruction; performing format standardization processing on the base reference image to obtain a standard reference image matching the base reference image; and readjusting the standard reference image based on the image text description to obtain the industrial design image to be generated, wherein the view images from multiple perspectives of the industrial design image to be generated have consistency. This method achieves consistency between multiple perspectives while ensuring that the generated industrial design image meets the engineering parameter constraints.

[0039] Figure 2This is a schematic diagram of the process provided by the present invention for performing format standardization processing on the base reference image to obtain a standard reference image that matches the base reference image.

[0040] The following will combine Figure 2 The process of performing format standardization processing on the basic reference image provided by the present invention to obtain a standard reference image that matches the basic reference image is described.

[0041] In an exemplary embodiment of the present invention, combined with Figure 2 As can be seen, the process of standardizing the format of the base reference image to obtain a standard reference image that matches the base reference image may include steps 210 to 240, and each step will be described below.

[0042] In step 210, a pre-configured target image bounding box is obtained, wherein the target image bounding box defines the standard position information and standard size information of the view images from various perspectives in the standard industrial design image.

[0043] In one embodiment, a pre-configured target image bounding box can be obtained. This bounding box can be a set of standardized target bounding box coordinates read from a preset engineering parameter library. This set of coordinates constitutes a rigid template for the multi-view layout. The target image bounding box fully defines the standard position and size information of each viewpoint in a standard industrial design image. Once configured, the set of target image bounding boxes is persistently stored in the engineering parameter library and can be repeatedly called in the current design task and subsequent similar design tasks, ensuring that drawings output by multiple projects and multiple designers maintain consistency in layout specifications.

[0044] In step 220, bounding box localization detection is performed on the base reference image to obtain the source bounding boxes of the view images from each perspective in the base reference image.

[0045] In one embodiment, an open-vocabulary object detection model can be used, such as a pre-trained detector based on the DETR or GroundingDINO architecture, with broad category words such as "train," "rail transit vehicle," "front of the vehicle," "side of the vehicle," and "roof of the vehicle" for the detection prompts. During application, the detection model can perform forward inference on the base reference image, outputting all instance regions in the image that match the prompts, and returning the detection results as bounding boxes, thereby obtaining the source bounding boxes of the view images from various perspectives in the base reference image.

[0046] In step 230, based on the source bounding box, the image content of the view images of each perspective within the source bounding box is extracted.

[0047] In one embodiment, an image extraction operation can be performed on the base reference image based on the source bounding box coordinates, thereby extracting the image content of the view images from various perspectives within the source bounding box.

[0048] In step 240, the image content is affinely transformed into the bounding box of the target image to obtain a standard reference image that matches the base reference image.

[0049] In one embodiment, the target image bounding box can be used as a rigid target container, and the image content of the view images from various perspectives within the source bounding box can be used as flexible content material. Affine geometric transformation is used to achieve precise matching between the content and the container, thereby obtaining a standard reference image that matches the base reference image.

[0050] In another embodiment, a "remapping" operation can be performed to precisely fill the detected image content into the canvas area defined by the target image bounding box through geometric transformation. This process generates a standardized layout diagram with a rigorous layout where the size and position of each component perfectly conform to preset engineering parameters.

[0051] In yet another exemplary embodiment of the present invention, continuing with the previously described embodiments, the bounding box localization and detection of the base reference image to obtain the source bounding boxes of the view images at each perspective in the base reference image can be achieved in the following manner: A pre-trained object detection model is invoked, wherein the object detection model is used to perform bounding box localization and detection on the image; The base reference image is input into the target detection model to obtain the source bounding boxes of the bottom view images of each perspective in the base reference image output by the target detection model.

[0052] In one embodiment, a pre-trained object detection model can be invoked. This model can be an open-vocabulary object detection model, such as one based on the Transformer architecture. It should be noted that the object detection model has been lightweight and fine-tuned on rail transit professional domain data. Using multiple labeled datasets containing multi-view design drawings of various rail transit vehicles, and with professional category terms such as "vehicle front," "vehicle side," "vehicle roof," "driver's cab window," and "passenger compartment door" as supervision signals, the model is trained iteratively a few times to improve its recall and localization accuracy in rail transit equipment component recognition tasks, such as recognizing bounding boxes.

[0053] In another embodiment, a base reference image can be input into an object detection model to obtain the source bounding boxes of the bottom view images from various perspectives in the base reference image output by the object detection model. In application, an open-vocabulary object detection model can be used, taking a broad category word (such as "train") as input, to analyze the base image, automatically identify and output the coordinates of the source bounding boxes of two or more independent view regions present in the image, thus obtaining the source bounding boxes of the bottom view images from various perspectives in the base reference image.

[0054] In yet another exemplary embodiment of the present invention, continuing with the previously described embodiments as examples, before readjusting the standard reference image based on image text description to obtain the industrial design image to be generated, the multi-view industrial design image generation method based on engineering constraints may further include: The pre-trained ControlNet model is invoked, whereby the ControlNet model is used to obtain the industrial design image to be generated based on the image text description; The process of readjusting a standard reference image based on image-text description to obtain the industrial design image to be generated can be achieved in the following ways: Input the image text description and standard reference image into the ControlNet model to obtain the industrial design image to be generated from the ControlNet model output.

[0055] In one embodiment, the ControlNet model can be a ControlNet model specifically trained for edge / contour conditions. The ControlNet model can include a base diffusion model, a ControlNet copy network, and zero-convolutional connection layers. The base diffusion model can be a pre-trained text-to-image backbone network, which has been pre-trained on large-scale image-text pairing data and possesses strong text semantic understanding and realistic image generation capabilities. The ControlNet copy network can be a copy of the encoder part of the denoising U-Net in the base diffusion model, forming a trainable parallel network copy. The structure of this copy network is completely identical to the encoder of the main network, but its weights are independently initialized and updated in subsequent training. The zero-convolutional connection layers can be zero-initialized convolutional layers that inject multi-scale features extracted by the copy network into the corresponding intermediate layers of the main network. The initial weights of the zero convolutions are zero, ensuring that the output of the copy network is zero at the start of training, without interfering with the original generation capability of the base model. It should be noted that the ControlNet model has been fine-tuned on a dedicated dataset for rail transit industrial design. With this fine-tuning, the ControlNet model has acquired the domain-specific ability to render design details in a refined manner based on text descriptions, while strictly adhering to the geometric structure of standardized layout diagrams.

[0056] In one embodiment, the ControlNet model can receive an image text description and a standard reference image. Further, the image text description can be input into the ControlNet model to generate a text feature vector, which is then injected into the denoising U-Net of the diffusion model via a cross-attention mechanism. The standard reference image is then input into the conditional encoder of the ControlNet replica network to generate a multi-scale conditional feature map. A Gaussian noise latent variable matching the target image size is randomly sampled as the initial state of the generation process. Within a set number of steps, contour, edge, and structural information is extracted from the standardized layout map and used to constrain the generation process, ensuring that the overall composition of the final image, the shape and position of each viewpoint strictly follow the engineering parameters defined in the layout map. Simultaneously, the model parses the text prompts, filling the described colors, textures, lighting, and decorative elements into the structural framework locked by the layout graph. After completing all iterations of denoising, the final latent variables are reconstructed into the final image in RGB pixel space through the decoder of the variational autoencoder, which is the industrial design image to be generated output by the ControlNet model. In each iteration, the ControlNet replica network receives the current noisy latent variables and the generated conditional feature map, and outputs multi-scale control features. The control features are injected into the corresponding encoder layer of the main network U-Net through zero convolutional layers. The main network U-Net also receives text feature vectors and calculates text-image similarity in the cross-attention layer. The main network predicts the noise residual of the current step and updates the noise latent variables until all iterations of denoising are completed. Finally, the final latent variables are reconstructed into the final image in RGB pixel space through the decoder of the variational autoencoder.

[0057] In this embodiment, since all viewpoints exist within the same strictly constrained ControlNet conditional graph and are guided by the same text prompts for detail generation, the model naturally applies the same visual attributes to the corresponding positions in another viewpoint when rendering the features of one viewpoint. This fundamentally ensures seamless correspondence and high consistency of all design elements across different viewpoints. Furthermore, because the layout itself is defined by precise bounding box engineering parameters, the final product also guarantees the engineering accuracy of the layout.

[0058] In yet another exemplary embodiment of the present invention, continuing with the previously described embodiments as an example, the image text description and standard reference image are input into the ControlNet model to obtain the industrial design image to be generated output by the ControlNet model. This can be achieved in the following way: The image text description and the standard reference image are input into the ControlNet model, and the structural information of the standard reference image is extracted by the ControlNet model to obtain the structural constraints for generating the industrial design image to be generated. Based on the structural constraints and the image text description, the standard reference image is readjusted by the ControlNet model to obtain the industrial design image to be generated output by the ControlNet model.

[0059] In one embodiment, the ControlNet model can perform structural information extraction on the input standard reference image to obtain the structural constraints for generating the industrial design image to be generated, namely the multi-scale conditional feature map mentioned above. Further, the ControlNet model then applies the generated structural constraints and image text description together to the iterative denoising process of the diffusion model, achieving a readjustment and generation of the standard reference image, resulting in the industrial design image to be generated output by the ControlNet model. It should be noted that the boundary position of each edge and each region in the industrial design image to be generated maintains pixel-level consistency with the structural information defined by engineering parameters in the standard reference image; the colors, materials, lighting, and decorative details in the image highly match the input text description; and since the generation process of all viewpoints is controlled by the same set of structural constraints (extracted from the same standard reference image), the images from different viewpoints are naturally aligned.

[0060] In yet another exemplary embodiment of the present invention, the image generation model is trained in the following manner, continuing with the previously described embodiments: Obtain a training dataset, wherein the training dataset includes multiple training data, the training data including industrial design image samples and mixed instruction samples corresponding to the industrial design image samples; Based on the training dataset, the image generation model is trained to obtain a trained image generation model.

[0061] In another embodiment, a large-scale training dataset specifically for generating multi-view images for rail transit industrial design can be constructed. The training data in this dataset includes industrial design image samples and corresponding mixed instruction samples. Furthermore, based on this training dataset, the image generation model can be trained to obtain a well-trained image generation model.

[0062] In one embodiment, the image generation model transforms a user's qualitative text description and quantitative canvas size parameters into a preliminary visual design. At its core, it employs a general-purpose, large-scale image generation model, fine-tuned using a domain-specific model weighting algorithm with low-rank adaptation (LoRA). This LoRA weighting is trained on a large dataset of rail transit industrial design images and corresponding text descriptions, injecting expertise on aerodynamic shape, window arrangement, and paint specifications into the general model. After the user inputs text prompts containing engineering parameters and a specified image aspect ratio, the module generates a base image with multiple perspectives on a canvas of the specified size, serving as the starting point and creative blueprint for subsequent processes.

[0063] As described above, this invention provides a method for generating multi-view industrial design images based on engineering constraints, comprising: acquiring a mixing instruction, wherein the mixing instruction includes an image text description of the industrial design image to be generated and engineering parameter constraints of the industrial design image to be generated, and the industrial design image to be generated includes view images from multiple perspectives; inputting the mixing instruction into a pre-trained image generation model to obtain a base reference image containing multiple perspectives output by the image generation model, wherein the image generation model is used to generate a base reference image matching the mixing instruction based on the mixing instruction; performing format standardization processing on the base reference image to obtain a standard reference image matching the base reference image; and readjusting the standard reference image based on the image text description to obtain the industrial design image to be generated, wherein the view images from multiple perspectives of the industrial design image to be generated have consistency. This achieves consistency between multiple perspectives while ensuring that the generated industrial design image meets the engineering parameter constraints.

[0064] The following describes the multi-view industrial design image generation device based on engineering constraints provided by the present invention. The multi-view industrial design image generation device based on engineering constraints described below and the multi-view industrial design image generation method based on engineering constraints described above can be referred to in correspondence.

[0065] Figure 3 This is a schematic diagram of the structure of the multi-view industrial design image generation device based on engineering constraints provided by the present invention.

[0066] The following will combine Figure 3 The structure of the multi-view industrial design image generation device based on engineering constraints provided by the present invention will be described.

[0067] In an exemplary embodiment of the present invention, combined with Figure 3 As can be seen, the multi-view industrial design image generation device based on engineering constraints may include an acquisition module 310, a processing module 320, a standardization module 330, and a generation module 340. Each module will be described in detail below.

[0068] The acquisition module 310 can be configured to acquire mixed instructions, wherein the mixed instructions include an image text description of the industrial design image to be generated and engineering parameter constraints of the industrial design image to be generated, and the industrial design image to be generated includes multiple perspective view images. The processing module 320 can be configured to input the mixing instructions into a pre-trained image generation model to obtain a base reference image containing multiple viewpoints output by the image generation model, wherein the image generation model is used to generate a base reference image that matches the mixing instructions based on the mixing instructions; The standardization module 330 can be configured to perform format standardization processing on the base reference image to obtain a standard reference image that matches the base reference image. The generation module 340 can be configured to readjust the standard reference image based on the image text description to obtain the industrial design image to be generated, wherein the view images of the industrial design image to be generated from multiple perspectives have consistency.

[0069] In an exemplary embodiment of the present invention, the standardization module 330 may further be configured to: Obtain a pre-configured target image bounding box, wherein the target image bounding box defines standard position information and standard size information of the view images from various perspectives in a standard industrial design image; The standardization module 330 can perform format standardization processing on the base reference image in the following manner to obtain a standard reference image that matches the base reference image: Boundary box localization and detection are performed on the base reference image to obtain the source bounding boxes of the bottom view images from each perspective in the base reference image; Based on the source bounding box, extract the image content of the view images from each perspective within the source bounding box; The image content is affinely transformed into the bounding box of the target image to obtain a standard reference image that matches the base reference image.

[0070] In an exemplary embodiment of the present invention, the standardization module 330 can perform bounding box localization and detection on the base reference image in the following manner to obtain the source bounding boxes of the view images at each perspective in the base reference image: A pre-trained object detection model is invoked, wherein the object detection model is used to perform bounding box localization and detection on the image; The base reference image is input into the target detection model to obtain the source bounding boxes of the bottom view images of each perspective in the base reference image output by the target detection model.

[0071] In an exemplary embodiment of the present invention, the generation module 340 may further be configured to: The pre-trained ControlNet model is invoked, wherein the ControlNet model is used to obtain the industrial design image to be generated based on the image text description; The generation module 340 can achieve the following: based on the image text description, it readjusts the standard reference image to obtain the industrial design image to be generated: The image text description and the standard reference image are input into the ControlNet model to obtain the industrial design image to be generated, which is output by the ControlNet model.

[0072] In an exemplary embodiment of the present invention, the generation module 340 may input the image text description and the standard reference image into the ControlNet model in the following manner to obtain the industrial design image to be generated output by the ControlNet model: The image text description and the standard reference image are input into the ControlNet model, and the structural information of the standard reference image is extracted by the ControlNet model to obtain the structural constraints for generating the industrial design image to be generated. Based on the structural constraints and the image text description, the standard reference image is readjusted by the ControlNet model to obtain the industrial design image to be generated output by the ControlNet model.

[0073] In an exemplary embodiment of the present invention, the processing module 320 may train the image generation model in the following manner: Obtain a training dataset, wherein the training dataset includes multiple training data, the training data including industrial design image samples and mixed instruction samples corresponding to the industrial design image samples; Based on the training dataset, the image generation model is trained to obtain a trained image generation model.

[0074] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communications bus 440. Processor 410 can call logic instructions in memory 430 to execute a multi-view industrial design image generation method based on engineering constraints. The method includes: acquiring a hybrid instruction, wherein the hybrid instruction includes an image text description of the industrial design image to be generated and engineering parameter constraints of the industrial design image to be generated, the industrial design image to be generated including view images from multiple perspectives; inputting the hybrid instruction into a pre-trained image generation model to obtain a base reference image containing multiple perspectives output by the image generation model, wherein the image generation model is used to generate a base reference image matching the hybrid instruction based on the hybrid instruction; performing format standardization processing on the base reference image to obtain a standard reference image matching the base reference image; and readjusting the standard reference image based on the image text description to obtain the industrial design image to be generated, wherein the view images from multiple perspectives of the industrial design image to be generated have consistency.

[0075] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0076] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multi-view industrial design image generation method based on engineering constraints provided by the above methods. The method includes: obtaining a mixing instruction, wherein the mixing instruction includes an image text description of the industrial design image to be generated and engineering parameter constraints of the industrial design image to be generated, the industrial design image to be generated including view images from multiple perspectives; inputting the mixing instruction into a pre-trained image generation model to obtain a base reference image containing multiple perspectives output by the image generation model, wherein the image generation model is used to generate a base reference image matching the mixing instruction based on the mixing instruction; performing format standardization processing on the base reference image to obtain a standard reference image matching the base reference image; and readjusting the standard reference image based on the image text description to obtain the industrial design image to be generated, wherein the view images from multiple perspectives of the industrial design image to be generated have consistency.

[0077] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for generating multi-view industrial design images based on engineering constraints provided by the methods described above. This method includes: acquiring a mixing instruction, wherein the mixing instruction includes an image text description of the industrial design image to be generated and engineering parameter constraints of the industrial design image to be generated, the industrial design image to be generated including multiple view images from different perspectives; inputting the mixing instruction into a pre-trained image generation model to obtain a base reference image containing multiple perspectives output by the image generation model, wherein the image generation model is used to generate a base reference image matching the mixing instruction based on the mixing instruction; performing format standardization processing on the base reference image to obtain a standard reference image matching the base reference image; and readjusting the standard reference image based on the image text description to obtain the industrial design image to be generated, wherein the view images of the industrial design image to be generated from multiple perspectives have consistency.

[0078] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0079] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating multi-view industrial design images based on engineering constraints, characterized in that, The method includes: Obtain a blending instruction, wherein the blending instruction includes an image text description of the industrial design image to be generated and engineering parameter constraints of the industrial design image to be generated, and the industrial design image to be generated includes multiple perspective view images; The mixing instructions are input into a pre-trained image generation model to obtain a base reference image containing multiple viewpoints output by the image generation model, wherein the image generation model is used to generate a base reference image that matches the mixing instructions based on the mixing instructions; The base reference image is format-normalized to obtain a standard reference image that matches the base reference image. Based on the image text description, the standard reference image is readjusted to obtain the industrial design image to be generated, wherein the view images of the industrial design image to be generated from multiple perspectives have consistency.

2. The method for generating multi-view industrial design images based on engineering constraints according to claim 1, characterized in that, Before performing format normalization processing on the base reference image to obtain a standard reference image that matches the base reference image, the method further includes: Obtain a pre-configured target image bounding box, wherein the target image bounding box defines standard position information and standard size information of the view images from various perspectives in a standard industrial design image; The step of standardizing the base reference image to obtain a standard reference image that matches the base reference image includes: Boundary box localization and detection are performed on the base reference image to obtain the source bounding boxes of the bottom view images from each perspective in the base reference image; Based on the source bounding box, extract the image content of the view images from each perspective within the source bounding box; The image content is affinely transformed into the bounding box of the target image to obtain a standard reference image that matches the base reference image.

3. The method for generating multi-view industrial design images based on engineering constraints according to claim 2, characterized in that, The step of performing bounding box localization and detection on the base reference image to obtain the source bounding boxes of the view images at each viewpoint in the base reference image includes: A pre-trained object detection model is invoked, wherein the object detection model is used to perform bounding box localization and detection on the image; The base reference image is input into the target detection model to obtain the source bounding boxes of the bottom view images of each perspective in the base reference image output by the target detection model.

4. The method for generating multi-view industrial design images based on engineering constraints according to claim 1, characterized in that, Before the step of readjusting the standard reference image based on the image text description to obtain the industrial design image to be generated, the method further includes: The pre-trained ControlNet model is invoked, wherein the ControlNet model is used to obtain the industrial design image to be generated based on the image text description; The process of readjusting the standard reference image based on the image text description to obtain the industrial design image to be generated includes: The image text description and the standard reference image are input into the ControlNet model to obtain the industrial design image to be generated, which is output by the ControlNet model.

5. The method for generating multi-view industrial design images based on engineering constraints according to claim 4, characterized in that, The step of inputting the image text description and the standard reference image into the ControlNet model to obtain the industrial design image to be generated output by the ControlNet model includes: The image text description and the standard reference image are input into the ControlNet model, and the structural information of the standard reference image is extracted by the ControlNet model to obtain the structural constraints for generating the industrial design image to be generated. Based on the structural constraints and the image text description, the standard reference image is readjusted by the ControlNet model to obtain the industrial design image to be generated output by the ControlNet model.

6. The method for generating multi-view industrial design images based on engineering constraints according to claim 1, characterized in that, The image generation model was trained in the following manner: Obtain a training dataset, wherein the training dataset includes multiple training data, the training data including industrial design image samples and mixed instruction samples corresponding to the industrial design image samples; Based on the training dataset, the image generation model is trained to obtain a trained image generation model.

7. A multi-view industrial design image generation device based on engineering constraints, characterized in that, The device includes: The acquisition module is used to acquire the mixing instructions, wherein the mixing instructions include the image text description of the industrial design image to be generated and the engineering parameter constraints of the industrial design image to be generated, and the industrial design image to be generated includes multiple perspective view images; The processing module is used to input the mixing instructions into a pre-trained image generation model to obtain a base reference image containing multiple viewpoints output by the image generation model, wherein the image generation model is used to generate a base reference image that matches the mixing instructions based on the mixing instructions; The standardization module is used to perform format standardization processing on the base reference image to obtain a standard reference image that matches the base reference image. The generation module is used to readjust the standard reference image based on the image text description to obtain the industrial design image to be generated, wherein the view images of the industrial design image to be generated from multiple perspectives have consistency.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the multi-view industrial design image generation method based on engineering constraints as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-view industrial design image generation method based on engineering constraints as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the multi-view industrial design image generation method based on engineering constraints as described in any one of claims 1 to 6.

Images