Multi-scene equipment design method and device based on deep learning

Through a multi-scene device design method based on deep learning, image annotation, preprocessing and model training are used to solve the problems of personalized needs and inefficiency in audio and video equipment design, and the automation design and equipment performance are improved.

CN118940630APending Publication Date: 2024-11-12GUANGZHOU BAOLUN ELECTRONICS CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411020188.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The prior art relies on experience in the installation and placement design of audio and video equipment, and cannot meet personalized needs, resulting in the inability to effectively perform equipment performance, inefficient efficiency and high time cost.

Method used

Using a multi-scene device design method based on deep learning, the label annotation, image preprocessing, data augmentation and model training are generated by receiving the pending block diagram and scene labels, and the image sample data is generated and divided into training sets and verification sets. and test sets, used to train neural network models to enable automated design drawing generation.

Benefits of technology

It realizes personalized customized equipment scene design, improves the level of equipment performance, improves design efficiency and scientificity, enhances the sensitivity and accuracy of the model to the placement of equipment in different scenarios, and supports the gradual iteration of the model to follow up with the market. change.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940630A_ABST
    Figure CN118940630A_ABST
Patent Text Reader

Abstract

The invention discloses a multi-scene equipment design method and device based on deep learning, and the method comprises the steps: carrying out the label labeling of a to-be-processed block image through a received scene label, obtaining and carrying out the image graying, noise filtering, edge detection and contour extraction processing of a scene block image, obtaining a recognition labeled image, and carrying out the recognition labeled image; generating an image variant according to the image and performing normalization processing on the image variant to obtain image sample data; dividing image sample data into a training set, a verification set and a test set to train, evaluate and test the initial neural network model, performing scene equipment design on the block graph through the tested model, and outputting a design result graph. According to the method, through graying, noise filtering, edge identification and contour extraction, the identification sensitivity and accuracy of the model on the application scene and the placement position of the block graph are improved; the image data is enhanced to generate variants, so that the generalization ability of the model is improved; and the image is normalized, so that the model training efficiency and stability are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image data processing and neural network deep learning technology, and in particular to a multi-scenario equipment design method and device based on deep learning. Background Art

[0002] With the vigorous development of AI technology, deep learning technology, as an algorithmic tool that can effectively discover the characteristics of drawings and apply them for learning, can use a large amount of historical design data to enable the model to effectively learn to deal with different design situations in different scenarios, output designs for users in a more efficient way, and at the same time maximize the effect of audio and video equipment, improve the user experience, and thus promote industry progress. Audio and video equipment includes but is not limited to screens, microphones, speakers, speakers, and other equipment used to express sound and light phenomena. For the installation and placement design of scenario-based audio and video equipment in the industry, the current existing technology often relies only on experience and fixed thinking to develop equipment.

[0003] However, if we only rely on workers' experience to install and place equipment, there are the following shortcomings: 1) Different audio and video equipment scenarios often require more personalized designs, and traditional designs cannot effectively meet this demand; 2) Audio and video equipment is limited by the scene and venue, as well as the overall equipment parameters, and often cannot effectively exert the performance of the equipment, resulting in a poor experience; 3) The design and placement of audio and video equipment requires the solution engineer to provide CAD drawings before implementation, which is inefficient and time-consuming. Summary of the invention

[0004] The present invention provides a multi-scenario equipment design method and device based on deep learning, so as to realize the technical effect of automatically generating design drawings, personalizing equipment scenarios, and improving the degree of equipment performance.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a multi-scenario device design method based on deep learning, comprising the following steps:

[0006] Receiving a block diagram to be processed and a scene label, and labeling the block diagram to be processed according to the scene label to obtain a scene block diagram to be processed;

[0007] The image block image to be processed is subjected to image graying, image noise filtering, image edge recognition and image contour extraction processing in sequence to obtain an identification and annotation image, an image variant is generated according to the identification and annotation image, and the image variant is normalized to obtain image sample data;

[0008] The image sample data is divided into an image training set, an image verification set and an image test set according to a preset first ratio, and then the initial neural network model is trained according to the image training set, and the performance of the trained model is evaluated and tested through the image verification set and the image test set. The scene equipment is designed for the scene block diagram to be processed through the tested model, and the corresponding design result diagram is output.

[0009] The multi-scenario device design method provided by the present invention, after receiving the block diagram to be processed and the scene label input by the user, first labels the block diagram to be processed according to the scene label, that is, determines which scenario the device in each block diagram should be applied to. If the device in a block diagram can be applied to multiple scenarios, the block diagram is labeled multiple times by the scene label, and all the scenarios to which the device in the block diagram can be applied are labeled. After the model learns and trains the block diagram, the specific application scenario and position placement of the device in each block diagram can be clarified, thereby improving the sensitivity and accuracy of the model for the design of device position placement in different scenarios.

[0010] After the block diagram is labeled with scene labels, the system will also perform image grayscale, noise filtering, edge recognition and contour extraction on the labeled scene block diagram in turn, so as to achieve secondary recognition and labeling of the scene block diagram, and further improve the subsequent model's recognition sensitivity and accuracy of the application scenario and placement of each block diagram after training and learning based on the images after secondary recognition and labeling. After the above image preprocessing, generating image variants based on the original image can improve the generalization ability of the model, and the generated image variants are normalized so that the image sample data formed by each image variant after normalization meets the input data format requirements of the model, improving the efficiency and stability of model training.

[0011] Dividing the image sample data into image training set, validation set and test set can improve the system's control over the model training effect, so that the system can evaluate and determine the model's training effect and performance through the validation set and test set, and then determine whether the output model used to design the block diagram has met the user's needs, so that the corresponding design results can be output according to the block diagram design. It can not only improve the design efficiency, scientificity and efficiency of homogeneous scenes, but also realize the personalized customization of equipment placement scheme design through subdividing scenes, and improve the generalization of the design. At the same time, because the model is trainable, users can continuously input new training sample data to make the model follow up market changes and realize the gradual iteration and follow-up of model design.

[0012] As a preferred example, labeling the block diagram to be processed according to the scene label to obtain the block diagram of the scene to be processed specifically includes:

[0013] Identify the type of the block diagram to be processed, and classify the block diagram to be processed into a scene overall design diagram and an element identification diagram according to the identification result;

[0014] If it is determined by classification that the block diagram to be processed is the overall design diagram of the scene, labeling the block diagram to be processed according to the scene label to obtain the corresponding block diagram of the scene;

[0015] If the classification determines that the block diagram to be processed is the element identification diagram, the element identification diagram is subjected to element standardized identification and labeling, and the result diagram after standardized identification and labeling is labeled according to the scene label to obtain the scene block diagram.

[0016] Before labeling the block diagram, its type will be identified first, and it will be divided into the overall scene design diagram and the element identification diagram. If the block diagram is determined to be the overall scene design diagram, the image can be directly labeled with the scene label. Since it is already the overall equipment placement design diagram of a certain application scene, the system can also input it as sample data into the model for training, so that the model is familiar with the equipment placement design in the scene. If it is determined that the block diagram is only an element identification diagram, it is also necessary to perform standardized identification and labeling on the element identification diagram, that is, to perform standardized labeling on the equipment in the element identification diagram, so that the standardized labeled image meets the system's requirements for scene labeling.

[0017] As a preferred example, the image grayscale, image noise filtering, image edge recognition and image contour extraction are sequentially performed on the scene block image to be processed to obtain an identification annotation image, specifically:

[0018] Calculating the grayscale value of the scene block image to be processed by a grayscale processing weighted average method so that each pixel of the scene block image to be processed has a corresponding grayscale value, and obtaining a corresponding grayscale image;

[0019] Filtering the grayscale image using a Gaussian filter, so that the Gaussian filter removes noise points from the grayscale image in both horizontal and vertical directions to obtain a filtered image;

[0020] Performing edge detection processing on the filter graph by using an edge detection algorithm so that strong edges in the filter graph are retained and connected, thereby generating an edge graph;

[0021] The edge map is subjected to contour extraction by a contour extraction function, so that the contours in the edge map are identified and returned to the scene block map to be processed, and the identified annotation map is obtained by output.

[0022] In order to further improve the sensitivity of the model to the equipment placement design, the system of the present invention performs secondary annotation on the scene block map, including graying, noise filtering, edge recognition and contour extraction on the scene block map, and then obtains the recognition and annotation map.

[0023] First, the image is grayed, that is, the gray value of each pixel in the image is calculated, and then the grayscale image corresponding to the image, that is, the binary image, is obtained, which improves the efficiency of the subsequent model to process the image. The grayscale image is filtered by a Gaussian filter, and the image is denoised from both the horizontal and vertical directions, that is, the image is denoised from both the X and Y directions, so as to achieve image denoising. Then the edge detection algorithm is used to detect the edge of the image, and the strong edges and weak edges in the image are detected by two preset thresholds, and the strong edges and weak edges connected to the strong edges are retained, and the remaining weak edges are removed, and the retained points are connected to form a clear edge of the image. Finally, contour extraction is used to identify and extract the contours in the image through the preset contour extraction function, and the contours extracted from the binary image are returned to the block map, so as to achieve secondary annotation recognition of the scene block map and obtain the recognition annotation map.

[0024] As a preferred example, the generating of image variants according to the recognition annotation graph and normalizing the image variants to obtain image sample data is specifically as follows:

[0025] Performing data enhancement processing on the identified annotation image, generating corresponding rotation image variants, scaled image variants and color image variants according to rotation, scaling and color changes;

[0026] The rotated image variant, the scaled image variant and the color image variant are integrated into the image variant, and pixel value normalization processing is performed on the image variant to obtain and output the image sample data.

[0027] Before generating image variants based on the recognition annotation map, the image will also be subjected to data augmentation processing to expand the image data set and enhance the generalization ability of the model. Specifically, rotated image variants, scaled image variants and color image variants of the original image will be generated according to rotation, scaling and color changes, so as to achieve data expansion of the recognition annotation map and improve the fitting degree and generalization ability of subsequent models.

[0028] After expanding the image, the system will also normalize the image to ensure overfitting of subsequent model training. At the same time, it can also improve the convergence speed and stability of model training, which helps to avoid errors in model output due to excessive sensitivity of the model to input data.

[0029] As a preferred example, the training of the initial neural network model according to the image training set and the performance evaluation and testing of the trained model through the image verification set and the image test set specifically include:

[0030] Inputting the image training set into the initial neural network model so that the initial neural network model performs deep learning and training on the image training set; wherein the loss function of the initial neural network model is a multi-classification cross entropy loss function;

[0031] The performance of the trained model is evaluated by the image verification set to obtain an evaluation result, and then the parameters of the trained model are adjusted according to the evaluation result, and training is performed according to the adjusted model until the evaluation result meets the preset evaluation requirements;

[0032] The image test set is called to test the model whose evaluation result meets the requirements, until the test result obtained by the test meets the preset test requirements, and the model whose test result meets the requirements is output as the post-test model.

[0033] The initial neural network model is trained by generating an image training set, an image verification set, and an image test set, so that the initial neural network model can perform deep learning on the data in the image training set. The loss function adopted by the model is a multi-classification cross entropy loss function, which is used to perform image classification, and measures the distance between the predicted probability distribution and the true probability distribution.

[0034] After one training session, the system will evaluate the performance of the trained model through a verification data set, and determine whether the trained model meets the requirements for starting the test based on the evaluation results, that is, whether the model can be tested before output through an image test set. If the evaluation results determine that it does not meet the requirements for starting the test, the model will be further adjusted in parameters based on the evaluation results, and the model will be controlled to start secondary training and secondary evaluation until the evaluation results determine that it has met the requirements for starting the test and can be tested.

[0035] After determination, the model is tested through the model test set, and whether the model can be output as a device design model is determined based on the test results. If the test results show that the output conditions are not met, it will return to the training step until the test results meet the output conditions of the model. Then the model will be output as a model for device design of the blank block diagram.

[0036] Correspondingly, an embodiment of the present invention further provides a multi-scenario device design device based on deep learning, wherein the multi-scenario device design device includes an image annotation module, an image processing module, and a model training module;

[0037] The image annotation module is used to receive the block diagram to be processed and the scene label, and to label the block diagram to be processed according to the scene label to obtain the scene block diagram to be processed;

[0038] The image processing module is used to sequentially perform image graying, image noise filtering, image edge recognition and image contour extraction processing on the scene block image to be processed to obtain an identification annotation image, generate an image variant according to the identification annotation image, and perform normalization processing on the image variant to obtain image sample data;

[0039] The model training module is used to divide the image sample data into an image training set, an image verification set and an image test set according to a preset first ratio, and then train the initial neural network model according to the image training set, perform performance evaluation and testing on the trained model through the image verification set and the image test set, design scene equipment for the scene block diagram to be processed through the tested model, and output the corresponding design result diagram.

[0040] As a preferred example, the image labeling module labels the block diagram to be processed according to the scene label to obtain the block diagram of the scene to be processed, specifically including:

[0041] Identify the type of the block diagram to be processed, and classify the block diagram to be processed into a scene overall design diagram and an element identification diagram according to the identification result;

[0042] If it is determined by classification that the block diagram to be processed is the overall design diagram of the scene, labeling the block diagram to be processed according to the scene label to obtain the corresponding block diagram of the scene;

[0043] If the classification determines that the block diagram to be processed is the element identification diagram, the element identification diagram is subjected to element standardized identification and labeling, and the result diagram after standardized identification and labeling is labeled according to the scene label to obtain the scene block diagram.

[0044] As a preferred example, the image processing module sequentially performs image graying, image noise filtering, image edge recognition and image contour extraction on the scene block image to be processed to obtain an identification annotation image, specifically:

[0045] Calculating the grayscale value of the image block map to be processed by a grayscale processing weighted average method, so that each pixel of the scene block map to be processed has a corresponding grayscale value, and obtaining a corresponding grayscale map;

[0046] Filtering the grayscale image using a Gaussian filter, so that the Gaussian filter removes noise points from the grayscale image in both horizontal and vertical directions to obtain a filtered image;

[0047] Performing edge detection processing on the filter graph by using an edge detection algorithm so that strong edges in the filter graph are retained and connected, thereby generating an edge graph;

[0048] The edge map is subjected to contour extraction by a contour extraction function, so that the contours in the edge map are identified and returned to the removed scenery block map, and the identified annotation map is obtained by output.

[0049] As a preferred example, the image processing module generates an image variant according to the recognition annotation image, and performs normalization processing on the image variant to obtain image sample data, specifically:

[0050] Performing data enhancement processing on the identified annotation image, generating corresponding rotation image variants, scaled image variants and color image variants according to rotation, scaling and color changes;

[0051] The rotated image variant, the scaled image variant and the color image variant are integrated into the image variant, and pixel value normalization processing is performed on the image variant to obtain and output the image sample data.

[0052] As a preferred example, the model training module trains the initial neural network model according to the image training set, and performs performance evaluation and testing on the trained model through the image verification set and the image test set, specifically including:

[0053] Inputting the image training set into the initial neural network model so that the initial neural network model performs deep learning and training on the image training set; wherein the loss function of the initial neural network model is a multi-classification cross entropy loss function;

[0054] The performance of the trained model is evaluated by the image verification set to obtain an evaluation result, and then the parameters of the trained model are adjusted according to the evaluation result, and training is performed according to the adjusted model until the evaluation result meets the preset evaluation requirements;

[0055] The image test set is called to test the model whose evaluation result meets the requirements, until the test result obtained by the test meets the preset test requirements, and the model whose test result meets the requirements is output as the post-test model. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 : A flowchart of an embodiment of a multi-scenario device design method based on deep learning provided by the present invention;

[0057] Figure 2 : A structural schematic diagram of an embodiment of a multi-scenario device design apparatus based on deep learning provided by the present invention;

[0058] Figure 3 :A training flow chart of a neural network model based on deep learning provided by the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Embodiment 1

[0061] Please refer to Figure 1 , which is a flow chart of an embodiment of a multi-scenario device design method based on deep learning provided by the present invention, including steps 101 to 103, each of which is specifically as follows:

[0062] Step 101: receiving a block diagram to be processed and a scene label, and labeling the block diagram to be processed according to the scene label to obtain a scene block diagram to be processed.

[0063] The multi-scenario device design method provided by the embodiment of the present invention, after receiving the block diagram to be processed and the scene label input by the user, first labels the block diagram to be processed according to the scene label, that is, determines which scenario the device in each block diagram should be applied to. If the device in a block diagram can be applied to multiple scenarios, the block diagram is labeled multiple times by the scene label, and all the scenarios to which the device in the block diagram can be applied are labeled. After learning and training the block diagram, the model can clarify the specific application scenarios and position placement of the equipment in each block diagram, thereby improving the sensitivity and accuracy of the model for the design of device position placement in different scenarios.

[0064] Specifically, in this embodiment, labeling the block diagram to be processed according to the scene label to obtain the block diagram of the scene to be processed specifically includes:

[0065] Identify the type of the block diagram to be processed, and classify the block diagram to be processed into a scene overall design diagram and an element identification diagram according to the identification result;

[0066] If it is determined by classification that the block diagram to be processed is the overall design diagram of the scene, labeling the block diagram to be processed according to the scene label to obtain the corresponding block diagram of the scene;

[0067] If the classification determines that the block diagram to be processed is the element identification diagram, the element identification diagram is subjected to element standardized identification and labeling, and the result diagram after standardized identification and labeling is labeled according to the scene label to obtain the scene block diagram.

[0068] Before labeling the block diagram, its type will be identified first, and it will be divided into the overall scene design diagram and the element identification diagram. If the block diagram is determined to be the overall scene design diagram, the image can be directly labeled with the scene label. Since it is already the overall equipment placement design diagram of a certain application scene, the system can also input it as sample data into the model for training, so that the model is familiar with the equipment placement design in the scene. If it is determined that the block diagram is only an element identification diagram, it is also necessary to perform standardized identification and labeling on the element identification diagram, that is, to perform standardized labeling on the equipment in the element identification diagram, so that the standardized labeled image meets the system's requirements for scene labeling.

[0069] In this embodiment, the overall scene design drawing is a labeled CAD design drawing, i.e., the overall scene CAD design drawing, and the element identification drawing is a labeled CAD element drawing. The labeled CAD element drawing, i.e., the CAD drawing, will use some special graphics to represent different entities during the design and drawing process. For example, in the audio and video wiring industry, a rectangle with an X is generally used to represent a speaker. The model needs to learn the reference of such element drawings in order to analyze which elements some labeled CAD drawings (i.e., the overall scene CAD design drawing) are specifically composed of, and then the design can be further implemented.

[0070] Therefore, after receiving the above two types of CAD drawings, the system can further label the element identification diagram, or directly store it in the system database. If it needs to be further labeled, the element standardization identification and scene labeling are performed on it by the method provided in this embodiment, and it is converted into a scene block diagram; if it is not further labeled, it can be distinguished from the overall scene CAD design drawing according to the label and stored in different sample databases of the system, waiting for model call training.

[0071] In addition, see Figure 3 , Figure 3 The training flow chart of the neural network model based on deep learning provided by the present invention. Figure 3 As shown, when the system receives the existing CAD design drawings and the overall design drawings of the scene described in this embodiment, it will also classify them according to the scene labels, and different scenes will be divided into different categories, namely Figure 3Scene A_CAD, Scene B_CAD and Scene C_CAD are shown, waiting to be called by the model. By classifying the existing CAD drawings by label according to the scene, such as meeting, school, hospital, bedroom, living room, etc., the CAD drawings are stored as the data of the label, which makes it easier for the model to classify and learn according to the label, and improves the design sensitivity of the model for the device application scenario.

[0072] Step 102: sequentially perform image grayscale conversion, image noise filtering, image edge recognition and image contour extraction on the scene block image to be processed to obtain an identification and annotation image, generate an image variant based on the identification and annotation image, and normalize the image variant to obtain image sample data.

[0073] After the block diagram is labeled with scene labels, the system will also perform image grayscale, noise filtering, edge recognition and contour extraction on the labeled scene block diagram in turn, so as to achieve secondary recognition and labeling of the scene block diagram, and further improve the subsequent model's recognition sensitivity and accuracy of the application scenario and placement of each block diagram after training and learning based on the images after secondary recognition and labeling. After the above image preprocessing, generating image variants based on the original image can improve the generalization ability of the model, and the generated image variants are normalized so that the image sample data formed by each image variant after normalization meets the input data format requirements of the model, improving the efficiency and stability of model training.

[0074] Specifically, in this embodiment, the image grayscale, image noise filtering, image edge recognition and image contour extraction are sequentially performed on the scene block image to be processed to obtain an identification annotation image, specifically:

[0075] Calculating the grayscale value of the image block map to be processed by a grayscale processing weighted average method, so that each pixel of the scene block map to be processed has a corresponding grayscale value, and obtaining a corresponding grayscale map;

[0076] Filtering the grayscale image using a Gaussian filter, so that the Gaussian filter removes noise points from the grayscale image in both horizontal and vertical directions to obtain a filtered image;

[0077] Performing edge detection processing on the filter graph by using an edge detection algorithm so that strong edges in the filter graph are retained and connected, thereby generating an edge graph;

[0078] The edge map is subjected to contour extraction by a contour extraction function, so that the contours in the edge map are identified and returned to the removed scenery block map, and the identified annotation map is obtained by output.

[0079] In order to further improve the sensitivity of the model to the equipment placement design, the system of the present invention performs secondary annotation on the scene block map, including graying, noise filtering, edge recognition and contour extraction on the scene block map, and then obtains the recognition and annotation map.

[0080] First, grayscale the image, that is, calculate the grayscale value of each pixel in the image, and then obtain the grayscale image corresponding to the image, that is, the binary image, which improves the efficiency of subsequent model processing of the image. The system's grayscale processing of the image does not affect the texture feature information of the image, and because each pixel in the grayscale image only needs one grayscale value to be represented, the grayscale image improves the image processing efficiency. The grayscale value of each pixel in the image is calculated by the grayscale processing weighted average method, and its value range is 0-255, and the grayscale image corresponding to the image can be obtained.

[0081] The grayscale image is filtered by a Gaussian filter to remove noise from the image in both horizontal and vertical directions, that is, to remove noise from the image in both X and Y directions, thereby achieving image denoising. The preferred Gaussian filter in this embodiment is 3 (5 / 7), and the standard deviation of the Gaussian kernel is set, which is expressed as follows:

[0082]

[0083] Among them, G(x,y) is the Gaussian kernel function and σ is the standard deviation, which is used for image denoising.

[0084] Then the edge detection algorithm performs edge detection on the image. It detects strong edges and weak edges in the image through two preset thresholds, retains strong edges and weak edges connected to strong edges, removes the remaining weak edges, and connects the retained points to form a clear edge of the image. Figure 3 As shown, this embodiment preferably uses the Canny algorithm to detect the edge of the image, and the specific detection steps are as follows:

[0085] 1. Calculate the gradient magnitude and direction of each pixel in the image. The gradient magnitude represents the strength of the edge. The specific gradient magnitude expression is as follows:

[0086]

[0087] Where Gx and Gy are the gradients in the x and y directions respectively.

[0088] 2. Perform non-maximum suppression to retain only the points with the largest gradient, just like flattening a hill and leaving only the peak.

[0089] 3. Use two preset thresholds, namely high threshold and low threshold, to find strong edges and weak edges.

[0090] 4. Strong edges are retained. Weak edges are also retained if they are connected to strong edges, otherwise they are discarded.

[0091] 5. Connect all the retained points to form a clear edge.

[0092] Finally, contour extraction is used to identify and extract the contours in the image through a preset contour extraction function, and the contours extracted from the binary image are returned to the block image, thereby realizing secondary annotation and recognition of the scene block image and obtaining the recognition annotation image. Figure 3 As shown, this embodiment preferably uses the cv2.findContours() function in the OpenCV library to find image contours, retrieve and return all contours in the binary image, and return the contours to the scene block map, thereby realizing secondary annotation of the block map.

[0093] Furthermore, in this embodiment, the image variant is generated according to the recognition annotation image, and the image variant is normalized to obtain the image sample data, specifically:

[0094] Performing data enhancement processing on the identified annotation image, generating corresponding rotation image variants, scaled image variants and color image variants according to rotation, scaling and color changes;

[0095] The rotated image variant, the scaled image variant and the color image variant are integrated into the image variant, and pixel value normalization processing is performed on the image variant to obtain and output the image sample data.

[0096] Before generating image variants based on the recognition annotation map, the image will also be subjected to data augmentation processing to expand the image data set and enhance the generalization ability of the model. Specifically, rotated image variants, scaled image variants and color image variants of the original image will be generated according to rotation, scaling and color changes, so as to achieve data expansion of the recognition annotation map and improve the fitting degree and generalization ability of subsequent models.

[0097] In this embodiment, if Figure 3 As shown, the image is augmented with data to construct three variants of rotation, scaling, and color change, thereby generating Figure 3 The data augmentation variants A1, A2 and A3 are shown, and image normalization is performed on all image variants obtained after data augmentation.

[0098] After expanding the image, the system will also normalize the image to ensure overfitting of subsequent model training. At the same time, it can also improve the convergence speed and stability of model training, which helps to avoid errors in model output due to excessive sensitivity of the model to input data.

[0099] Step 103: Divide the image sample data into an image training set, an image verification set and an image test set according to a preset first ratio, then train the initial neural network model according to the image training set, perform performance evaluation and testing on the trained model through the image verification set and the image test set, perform scene equipment design on the scene block diagram to be processed through the tested model, and output the corresponding design result diagram.

[0100] Dividing the image sample data into image training set, validation set and test set can improve the system's control over the model training effect, so that the system can evaluate and determine the model's training effect and performance through the validation set and test set, and then determine whether the output model used to design the block diagram has met the user's needs, so that the corresponding design results can be output according to the block diagram design. It can not only improve the design efficiency, scientificity and efficiency of homogeneous scenes, but also realize the personalized customization of equipment placement scheme design through subdividing scenes, and improve the generalization of the design. At the same time, because the model is trainable, users can continuously input new training sample data to make the model follow up market changes and realize the gradual iteration and follow-up of model design.

[0101] In this embodiment, the system preferably divides the data into an image training set, an image test set, and an image verification set in a ratio of 7:2:1, that is, the first ratio is preferably 7:2:1.

[0102] Specifically, the present embodiment trains the initial neural network model according to the image training set, and performs performance evaluation and testing on the trained model through the image verification set and the image test set, specifically including:

[0103] Inputting the image training set into the initial neural network model so that the initial neural network model performs deep learning and training on the image training set; wherein the loss function of the initial neural network model is a multi-classification cross entropy loss function;

[0104] The performance of the trained model is evaluated by the image verification set to obtain an evaluation result, and then the parameters of the trained model are adjusted according to the evaluation result, and training is performed according to the adjusted model until the evaluation result meets the preset evaluation requirements;

[0105] The image test set is called to test the model whose evaluation result meets the requirements, until the test result obtained by the test meets the preset test requirements, and the model whose test result meets the requirements is output as the post-test model.

[0106] The initial neural network model is trained by the generated image training set, image verification set and image test set, so that the initial neural network model performs deep learning on the data in the image training set, and the loss function adopted by the model is a multi-classification cross entropy loss function, which is used to classify images, and measures the distance between the predicted probability distribution and the true probability distribution. At the same time, this embodiment uses the Keras framework to define the CNN model.

[0107] After one training session, the system will evaluate the performance of the trained model through a verification data set, and determine whether the trained model meets the requirements for starting the test based on the evaluation results, that is, whether the model can be tested before output through an image test set. If the evaluation results determine that it does not meet the requirements for starting the test, the model will be further adjusted in parameters based on the evaluation results, and the model will be controlled to start secondary training and secondary evaluation until the evaluation results determine that it has met the requirements for starting the test and can be tested.

[0108] After determination, the model is tested through the model test set, and whether the model can be output as a device design model is determined based on the test results. If the test results show that the output conditions are not met, it will return to the training step until the test results meet the output conditions of the model. Then the model will be output as a model for device design of the blank block diagram.

[0109] When the model training is completed and passed the test, the user can directly input a blank CAD block diagram into the model, label the blocks planned for design, and enter the corresponding scene labels such as: meeting, classroom, etc. When the model receives the corresponding input data, it will call the pre-trained model that has passed the test to cut the scene module, and generate corresponding different design results in different modules and output them to the user. In addition, the user can select the CAD design solution that best meets the actual needs in the output design results through multiple inputs.

[0110] In summary, this embodiment uses the CNN model to learn the design rules of audio and video equipment in different scenarios, and precipitates the design rules that cannot be expressed in words by solution personnel and designers. This not only improves the design efficiency of homogeneous scenarios, but also enables personalized customization of audio and video equipment solution design through segmented scenarios, thereby improving the generalization of the design. In addition, since the model supports a persistent training method, the model design method provided in this embodiment can effectively follow up market changes and realize the gradual iterative follow-up of the design.

[0111] In order to better illustrate the working principle and step flow of the multi-scenario equipment design method and device based on deep learning of the present invention, you can but are not limited to referring to the relevant records above.

[0112] Accordingly, see Figure 2 , Figure 2 This is a schematic diagram of the structure of an embodiment of the multi-scenario device design apparatus based on deep learning provided by the present invention. Figure 2 As shown, the multi-scenario equipment design device includes an image annotation module 201, an image processing module 202 and a model training module 203.

[0113] The image labeling module 201 is used to receive the block image to be processed and the scene label, and label the block image to be processed according to the scene label to obtain the scene block image to be processed.

[0114] Furthermore, the image labeling module 201 labels the block diagram to be processed according to the scene label to obtain the block diagram of the scene to be processed, which specifically includes:

[0115] Identify the type of the block diagram to be processed, and classify the block diagram to be processed into a scene overall design diagram and an element identification diagram according to the identification result;

[0116] If the classification determines that the block diagram to be processed is the overall design diagram of the scene, the block diagram to be processed is labeled according to the scene label to obtain the corresponding scene block diagram; if the classification determines that the block diagram to be processed is the element identification diagram, the element identification diagram is subjected to element standardized identification and labeling, and the result diagram after standardized identification and labeling is labeled according to the scene label to obtain the scene block diagram.

[0117] The image processing module 202 is used to sequentially perform image grayscale, image noise filtering, image edge recognition and image contour extraction processing on the scene block image to be processed to obtain an identification annotation image, generate an image variant according to the identification annotation image, and normalize the image variant to obtain image sample data.

[0118] Furthermore, the image processing module 202 sequentially performs image graying, image noise filtering, image edge recognition and image contour extraction processing on the scene block image to be processed to obtain an identification annotation image, specifically:

[0119] The grayscale value of the image block image to be processed is calculated by a grayscale processing weighted average method, so that each pixel of the scene block image to be processed has a corresponding grayscale value, and a corresponding grayscale image is obtained; the grayscale image is filtered by a Gaussian filter, so that the Gaussian filter removes noise points from the grayscale image in both horizontal and vertical directions, and a filtered image is obtained;

[0120] The filter image is subjected to edge detection processing by an edge detection algorithm so that strong edges in the filter image are retained and connected, thereby generating an edge image; the edge image is subjected to contour extraction by a contour extraction function so that contours in the edge image are identified and returned to the removed scenic area block image, and the identified annotation image is obtained as output.

[0121] Furthermore, the image processing module 202 generates an image variant according to the recognition annotation image, and performs normalization processing on the image variant to obtain image sample data, specifically:

[0122] Performing data enhancement processing on the identification annotation image, generating corresponding rotated image variants, scaled image variants and color image variants according to rotation, scaling and color changes; integrating the rotated image variants, the scaled image variants and the color image variants into the image variant, performing pixel value normalization processing on the image variant, and obtaining and outputting the image sample data.

[0123] The model training module 203 is used to divide the image sample data into an image training set, an image verification set and an image test set according to a preset first ratio, and then train the initial neural network model according to the image training set, perform performance evaluation and testing on the trained model through the image verification set and the image test set, design the scene equipment for the scene block diagram to be processed through the tested model, and output the corresponding design result diagram.

[0124] Furthermore, the model training module 203 trains the initial neural network model according to the image training set, and performs performance evaluation and testing on the trained model through the image verification set and the image test set, specifically including:

[0125] Inputting the image training set into the initial neural network model so that the initial neural network model performs deep learning and training on the image training set; wherein the loss function of the initial neural network model is a multi-classification cross entropy loss function;

[0126] The performance of the trained model is evaluated through the image verification set to obtain an evaluation result, and then the parameters of the trained model are adjusted according to the evaluation result, and training is performed according to the adjusted model until the evaluation result meets the preset evaluation requirements; the image test set is called to test the model whose evaluation result meets the requirements, until the test result obtained by the test meets the preset test requirements, and the model whose test result meets the requirements is output as the tested model.

[0127] In summary, the embodiments of the present invention provide a multi-scenario equipment design method and device based on deep learning, which labels the block diagram to be processed through the received scene labels, obtains and sequentially performs image graying, noise filtering, edge detection and contour extraction on the scene block diagram, obtains the recognition annotation map, generates image variants according to the image and normalizes them to obtain image sample data; then divides the image sample data into training set, verification set and test set to train, evaluate and test the initial neural network model, and designs the scene equipment for the block diagram through the post-test model, and outputs the design result map. The present invention improves the model's recognition sensitivity and accuracy for the application scenario and placement position of the block diagram through graying, noise filtering, edge recognition and contour extraction; enhances the generation of variants for image data to improve the generalization ability of the model; and normalizes the image to improve the model training efficiency and stability.

[0128] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-scenario device design method based on deep learning, characterized in that: The specific steps include: Receiving a block diagram to be processed and a scene label, and labeling the block diagram to be processed according to the scene label to obtain a scene block diagram to be processed; The image block image to be processed is subjected to image graying, image noise filtering, image edge recognition and image contour extraction processing in sequence to obtain an identification and annotation image, an image variant is generated according to the identification and annotation image, and the image variant is normalized to obtain image sample data; The image sample data is divided into an image training set, an image verification set and an image test set according to a preset first ratio, and then the initial neural network model is trained according to the image training set, and the performance of the trained model is evaluated and tested through the image verification set and the image test set. The scene equipment is designed for the scene block diagram to be processed through the tested model, and the corresponding design result diagram is output.

2. A multi-scenario device design method based on deep learning as claimed in claim 1, characterized in that: The step of labeling the block diagram to be processed according to the scene label to obtain the block diagram of the scene to be processed specifically includes: Identify the type of the block diagram to be processed, and classify the block diagram to be processed into a scene overall design diagram and an element identification diagram according to the identification result; If it is determined by classification that the block diagram to be processed is the overall design diagram of the scene, labeling the block diagram to be processed according to the scene label to obtain the corresponding block diagram of the scene; If the classification determines that the block diagram to be processed is the element identification diagram, the element identification diagram is subjected to element standardized identification and labeling, and the result diagram after standardized identification and labeling is labeled according to the scene label to obtain the scene block diagram.

3. A multi-scenario device design method based on deep learning as claimed in claim 1, characterized in that: The image graying, image noise filtering, image edge recognition and image contour extraction are sequentially performed on the scene block image to be processed to obtain an identification annotation image, specifically: Calculating the grayscale value of the scene block image to be processed by a grayscale processing weighted average method so that each pixel of the scene block image to be processed has a corresponding grayscale value, and obtaining a corresponding grayscale image; Filtering the grayscale image using a Gaussian filter, so that the Gaussian filter removes noise points from the grayscale image in both horizontal and vertical directions to obtain a filtered image; Performing edge detection processing on the filter graph by using an edge detection algorithm so that strong edges in the filter graph are retained and connected, thereby generating an edge graph; The edge map is subjected to contour extraction by a contour extraction function, so that the contours in the edge map are identified and returned to the scene block map to be processed, and the identified annotation map is obtained by output.

4. The multi-scenario device design method based on deep learning as claimed in claim 1, characterized in that: The step of generating an image variant according to the recognition annotation graph and performing normalization processing on the image variant to obtain image sample data is specifically as follows: Performing data enhancement processing on the identified annotation image, generating corresponding rotation image variants, scaled image variants and color image variants according to rotation, scaling and color changes; The rotated image variant, the scaled image variant and the color image variant are integrated into the image variant, and pixel value normalization processing is performed on the image variant to obtain and output the image sample data.

5. The multi-scenario device design method based on deep learning as claimed in claim 1, characterized in that: The training of the initial neural network model according to the image training set and the performance evaluation and testing of the trained model through the image verification set and the image test set specifically include: Inputting the image training set into the initial neural network model so that the initial neural network model performs deep learning and training on the image training set; wherein the loss function of the initial neural network model is a multi-classification cross entropy loss function; The performance of the trained model is evaluated by the image verification set to obtain an evaluation result, and then the parameters of the trained model are adjusted according to the evaluation result, and training is performed according to the adjusted model until the evaluation result meets the preset evaluation requirements; The image test set is called to test the model whose evaluation result meets the requirements, until the test result obtained by the test meets the preset test requirements, and the model whose test result meets the requirements is output as the post-test model.

6. A multi-scenario equipment design device based on deep learning, characterized in that: The multi-scenario equipment design device includes an image annotation module, an image processing module and a model training module; The image annotation module is used to receive the block diagram to be processed and the scene label, and to label the block diagram to be processed according to the scene label to obtain the scene block diagram to be processed; The image processing module is used to sequentially perform image graying, image noise filtering, image edge recognition and image contour extraction processing on the scene block image to be processed to obtain an identification annotation image, generate an image variant according to the identification annotation image, and perform normalization processing on the image variant to obtain image sample data; The model training module is used to divide the image sample data into an image training set, an image verification set and an image test set according to a preset first ratio, and then train the initial neural network model according to the image training set, perform performance evaluation and testing on the trained model through the image verification set and the image test set, design scene equipment for the scene block diagram to be processed through the tested model, and output the corresponding design result diagram.

7. The multi-scenario equipment design device based on deep learning as claimed in claim 6, characterized in that: The image labeling module labels the block image to be processed according to the scene label to obtain the block image to be processed, specifically including: Identify the type of the block diagram to be processed, and classify the block diagram to be processed into a scene overall design diagram and an element identification diagram according to the identification result; If it is determined by classification that the block diagram to be processed is the overall design diagram of the scene, labeling the block diagram to be processed according to the scene label to obtain the corresponding block diagram of the scene; If the classification determines that the block diagram to be processed is the element identification diagram, the element identification diagram is subjected to element standardized identification and labeling, and the result diagram after standardized identification and labeling is labeled according to the scene label to obtain the scene block diagram.

8. The multi-scenario equipment design device based on deep learning as claimed in claim 6, characterized in that: The image processing module sequentially performs image graying, image noise filtering, image edge recognition and image contour extraction on the scene block image to be processed to obtain an identification annotation image, specifically: Calculating the grayscale value of the scene block image to be processed by a grayscale processing weighted average method so that each pixel of the scene block image to be processed has a corresponding grayscale value, and obtaining a corresponding grayscale image; Filtering the grayscale image using a Gaussian filter, so that the Gaussian filter removes noise points from the grayscale image in both horizontal and vertical directions to obtain a filtered image; Performing edge detection processing on the filter graph by using an edge detection algorithm so that strong edges in the filter graph are retained and connected, thereby generating an edge graph; The edge map is subjected to contour extraction by a contour extraction function, so that the contours in the edge map are identified and returned to the scene block map to be processed, and the identified annotation map is obtained by output.

9. The multi-scenario equipment design device based on deep learning as claimed in claim 6, characterized in that: The image processing module generates an image variant according to the recognition annotation image, and performs normalization processing on the image variant to obtain image sample data, specifically: Performing data enhancement processing on the identified annotation image, generating corresponding rotation image variants, scaled image variants and color image variants according to rotation, scaling and color changes; The rotated image variant, the scaled image variant and the color image variant are integrated into the image variant, and pixel value normalization processing is performed on the image variant to obtain and output the image sample data.

10. The multi-scenario equipment design device based on deep learning according to claim 6, characterized in that: The model training module trains the initial neural network model according to the image training set, and performs performance evaluation and testing on the trained model through the image verification set and the image test set, specifically including: Inputting the image training set into the initial neural network model so that the initial neural network model performs deep learning and training on the image training set; wherein the loss function of the initial neural network model is a multi-classification cross entropy loss function; The performance of the trained model is evaluated by the image verification set to obtain an evaluation result, and then the parameters of the trained model are adjusted according to the evaluation result, and training is performed according to the adjusted model until the evaluation result meets the preset evaluation requirements; The image test set is called to test the model whose evaluation result meets the requirements, until the test result obtained by the test meets the preset test requirements, and the model whose test result meets the requirements is output as the post-test model.

Citation Information

Patent Citations

  • Furniture placement scheme generation method and device, equipment and storage medium

    CN110363853A

  • Furniture layout drawing generation method and device, computer equipment and storage medium

    CN110751721A

  • Home decoration design method and device, electronic equipment and storage medium

    CN111553012A

  • Converter tapping hole steel flow identification method and system, medium and electronic terminal

    CN112613523A

  • Advertisement screenshot classification method and system, electronic equipment and storage medium

    CN116912560A