A project image material management system based on deep learning

Abstract

The present invention discloses a project picture material management system based on deep learning, which relates to the technical field of picture material management. The present invention improves the material reuse rate. When a new picture is put into the database, the system can recommend similar pictures for reuse, reduce the material expansion speed, and improve the material reuse rate; at the same time, it can quickly locate the target picture. When the picture material is upgraded, the system can quickly find the target picture, reduce the manual screening cost, and reduce the risk of missing pictures; in addition, the retrieval accuracy and efficiency are improved. In the picture retrieval and deletion work, the retrieval accuracy and deletion efficiency are greatly improved, similar pictures can be identified more accurately, and more accurate query results can be provided; it solves the problem that when processing a large number of pictures, the query speed is slow, which affects the work efficiency, and at the same time, for similar but not completely the same pictures, the existing method is difficult to accurately identify, resulting in low accuracy of the query results.

Description

Technical Field

[0001] The present invention relates to the technical field of picture material management, and in particular to a project picture material management system based on deep learning. Background Art

[0002] In terms of engineering picture management, the project department will centrally and uniformly manage engineering image data in accordance with the project archive management regulations of the engineering department, take sub-unit projects as units, and sort and archive them according to divisions, sub-projects, special contents, and shooting time. The image data here are arranged in the order of filing of engineering quality acceptance materials of the engineering project and attached thereto. The image data should have a text description, including image number, image title, brief description of shooting content, shooting time, and location. It can be seen that engineering picture management is extremely important for engineering projects.

[0003] For example, the patent with publication number CN117411974A discloses a picture material classification management system based on wireless communication technology, including a box, a belt conveyor is fixedly installed on the top of the box, a collection camera module is movably installed on the top of the belt conveyor, and a circuit board is fixedly installed inside the box; the invention realizes that the belt conveyor and the collection camera module are controlled by a PLC controller to start at the same time, and then the belt conveyor is used to move the picture material, and then the paper picture material is collected by the collection camera module, and then the Zigbee transmission module is used to wirelessly transmit the picture material, and then the Bluetooth transmission module is used to wirelessly transmit the picture material, and then the WiFi transmission module is used to wirelessly transmit the picture material, and a variety of wireless communication technologies are used to transmit the picture material in real time, and a suitable wireless communication scheme is selected to improve the efficiency of picture material transmission.

[0004] At present, the common engineering image management mechanisms on the market mainly include the following two types: name identical detection, managing images by detecting the identical names of image files; content identical detection, managing images by detecting the identical contents of image files; however, these methods are powerless when dealing with complex management scenarios; when processing a large number of images, the query speed is slow, affecting work efficiency. For example, during the image material upgrade process, many business lines have many icons or design drawings of similar scenes, and the cost of manually confirming the replacement list is very high; at the same time, for similar but not identical images, existing methods are difficult to accurately identify, resulting in low accuracy of query results. For example, when designing materials for new requirements, it is difficult to find that there were similar materials in the past. If the materials are directly reused, the overall design consistency will be better, while avoiding the proliferation of similar materials and strengthening material reuse; therefore, there is an urgent need for a project image material management system based on deep learning to solve such problems. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a project image material management system based on deep learning, which solves the problems in the prior art that when processing a large number of images, the query speed is slow, affecting work efficiency, and at the same time, for similar but not exactly the same images, the existing methods are difficult to accurately identify, resulting in low accuracy of query results.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0007] The present invention provides a project picture material management system based on deep learning, comprising:

[0008] Image resource scanning and aggregation module, used to scan local and cloud image resources, preliminarily organize image files and generate image lists;

[0009] Image resource processing module: Image resource processing specifically includes image vectorization, persistence, and query processing. It uses the convolutional neural network (CNN) algorithm to vectorize image resources and store them persistently for query;

[0010] The vector query module provides vector query functions, vectorizes the input image, calculates the vector similarity of the existing images, and returns a result list based on the similarity score;

[0011] The scene image processing module uses similar image query functions in different application scenarios to upgrade logos and add images;

[0012] OCR processing and filtering module, which extracts text information from images through OCR technology and provides text-based filtering functions;

[0013] Image query module, which provides image upload, API call and result display page for encapsulated PHP page;

[0014] The image upload inspection module, including the project code submission link and the cloud image upload link, needs to call the image query module.

[0015] The present invention is further configured as a picture resource scanning and aggregation module, which scans picture files in the local file system and cloud storage by writing scripts or using existing tools, and the supported file formats include jpg, gif, png, and webp;

[0016] Summarize the scanned image file paths to generate a local image list and a cloud image list;

[0017] The present invention is further configured such that the picture resource processing module performs picture processing in the following manner:

[0018] Create a VGGNet model: Import the Keras library, create and load the pre-trained VGGNet model;

[0019] Feature vector extraction: encapsulates the method of extracting the last layer of convolution features of VGG16 and converts the image into a feature vector;

[0020] Persistent storage: store feature vectors persistently in local (native.h5) and cloud (cloud.h5) databases;

[0021] The present invention is further configured such that the vector query module performs vector query in the following manner:

[0022] Input image vectorization: convert the input image into a feature vector;

[0023] Vector similarity calculation: calculate the similarity between the input image feature vector and the feature vector stored in the database;

[0024] Result sorting and return: Sort by similarity score and return a list of images with the highest similarity;

[0025] The present invention is further configured such that the scene image processing module performs processing in the following manner:

[0026] Input different styles of logos and obtain the most similar images for manual replacement;

[0027] Query similar pictures, integrate and merge them;

[0028] The present invention is further configured to use OCR technology to extract text information from the image in the OCR processing and filtering module;

[0029] And perform text filtering, enter text keywords, and narrow the scope of search results;

[0030] The present invention is further configured that the image upload inspection method is:

[0031] If the results contain results with a similarity greater than x, a manual processing process is required;

[0032] If the old image is adopted, the submission will be rejected; if the new image is adopted, the new image will be merged;

[0033] The present invention is further configured that the project picture material management method is:

[0034] Step 1. Download the project code according to the project address and update the dependencies;

[0035] Step 2. Create `native.h5` and `cloud.h5` databases and initialize them to store image vectors;

[0036] Step 3. Introduce Keras, create the VGGNet model, and encapsulate the method of extracting the last layer of convolution features of VGG16;

[0037] Step 4. Retrieve the image files in the project directory and generate a local image list;

[0038] Step 5. Call the feature vector extraction method in step 3 for each image, generate feature vectors, and save them to `native.h5`;

[0039] Step 6. Get the cloud image list, extract feature vectors for each image, and save them to `cloud.h5`;

[0040] Step 7. Move the generated `native.h5` and `cloud.h5` files to the production environment;

[0041] Step 8. Build a Python-based query service through CherryPy, preload the Keras VGGNet model and feature extraction method, and implement the similarity calculation and result sorting of image vectors;

[0042] Step 9. Encapsulate the PHP page, provide the image upload function, and call the API to display the results;

[0043] Step 10. Implement the image upload inspection module, including the project code submission link and the cloud image upload link, call the query module, process the images whose similarity reaches the threshold, and conduct manual review and processing.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention improves the reuse of materials. When new images are stored in the database, the system can recommend similar images for reuse, thereby reducing the material expansion rate and improving the reuse rate of materials.

[0046] The present invention can quickly locate the target image. When the image material is upgraded, the system can quickly find the target image, reduce the cost of manual screening, and reduce the risk of missing images.

[0047] The present invention improves the retrieval accuracy and efficiency. In the work of image retrieval and deletion, the retrieval accuracy and deletion efficiency are greatly improved, and similar images can be more accurately identified, providing more accurate query results.

[0048] The invention solves the problem that when processing a large number of pictures, the query speed is slow, which affects the work efficiency, and the existing method is difficult to accurately identify similar but not completely the same pictures, resulting in low accuracy of the query results. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flowchart for preparing picture vector data for a project picture material management system based on deep learning according to the present invention;

[0050] Figure 2 This is a picture vector database query flow chart of the project picture material management system based on deep learning of the present invention;

[0051] Figure 3 This is a picture upload check flow chart of the project picture material management system based on deep learning of the present invention. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0053] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0054] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0055] Example 1

[0056] See also Figure 1-Figure 3 The present invention provides a project picture material management system based on deep learning, comprising:

[0057] Image resource scanning and aggregation module, used to scan local and cloud image resources, preliminarily organize image files and generate image lists;

[0058] Image resource scanning and aggregation module, which scans image files in the local file system and cloud storage by writing scripts or using existing tools. The supported file formats include jpg, gif, png, and webp;

[0059] Summarize the scanned image file paths to generate a local image list and a cloud image list;

[0060] Specifically, taking Python as an example, the implementation steps of the image resource scanning and aggregation module are as follows:

[0061] import os

[0062] def scan_local_images(directory):

[0063] image_list = []

[0064] for root,dirs,files in os.walk(directory):

[0065] for file in files:

[0066] if file.endswith((".jpg",".gif",".png",".webp")):

[0067] image_list.append(os.path.join(root,file))

[0068] return image_list

[0069] local_images=scan_local_images(" / path / to / local / images")

[0070] cloud_images=scan_local_images(" / path / to / cloud / images")

[0071] Image resource processing module: Image resource processing specifically includes image vectorization, persistence, and query processing. It uses the convolutional neural network (CNN) algorithm to vectorize image resources and store them persistently for query;

[0072] The image resource processing module processes images in the following ways:

[0073] Create a VGGNet model: Import the Keras library, create and load the pre-trained VGGNet model;

[0074] Feature vector extraction: encapsulates the method of extracting the last layer of convolution features of VGG16 and converts the image into a feature vector;

[0075] Persistent storage: store feature vectors persistently in local (native.h5) and cloud (cloud.h5) databases;

[0076] Specifically, taking Python as an example, the implementation steps of the image resource processing module are:

[0077]

[0078]

[0079] The vector query module provides vector query functions, vectorizes the input image, calculates the vector similarity of the existing images, and returns a result list based on the similarity score;

[0080] The vector query module performs vector query in the following ways:

[0081] Input image vectorization: convert the input image into a feature vector;

[0082] Vector similarity calculation: calculate the similarity (such as inner product) between the input image feature vector and the feature vector stored in the database;

[0083] Result sorting and return: Sort by similarity score and return a list of images with the highest similarity;

[0084] Specifically, taking Python as an example, the implementation steps of the vector query module are as follows:

[0085]

[0086] results=query_similar_images(' / path / to / query / image.jpg','native.h5')

[0087] The scene image processing module uses similar image query functions in different application scenarios to upgrade logos and add images;

[0088] The scene image processing module performs the following processing methods:

[0089] Input different styles of logos and obtain the most similar images for manual replacement;

[0090] Query similar pictures, integrate and merge them;

[0091] Specifically, taking Python as an example, the implementation steps of the scene image processing module are:

[0092] #Assume that results have been obtained from the previous step

[0093] def handle_logo_upgrade(results):

[0094] top_images=results[:10]

[0095] #Add manual review and replacement logic here

[0096] return top_images

[0097] top_images=handle_logo_upgrade(results)

[0098] OCR processing and filtering module, which extracts text information from images through OCR technology and provides text-based filtering functions;

[0099] In the OCR processing and filtering module, OCR technology is used to extract text information from images;

[0100] And perform text filtering, enter text keywords, and narrow the scope of search results;

[0101] Image query module, which provides image upload, API call and result display page for encapsulated PHP page;

[0102] The image upload inspection module includes the project code submission link and the cloud image upload link, which requires calling the image query module;

[0103] The image upload check method is:

[0104] If the results contain results with a similarity greater than x (e.g. 0.9), a manual processing process is required;

[0105] If the old image is adopted, the submission will be rejected; if the new image is adopted, the new image will be merged;

[0106] Specifically, taking Python as an example, the implementation steps of the OCR processing and filtering module are:

[0107]

[0108] Project image material management methods are:

[0109] Step 1. Download the project code according to the project address and update the dependencies; Download the project code and update the dependencies (Keras library, etc.);

[0110] Step 2. Create native.h5 and could.h5 to initialize the database for storing image vectors; create `native.h5` and `cloud.h5` files for persistent storage of image feature vectors;

[0111] Step 3. Introduce Keras, create the VGGNet model, and encapsulate the method of extracting the last layer of convolution features of VGG16;

[0112] Step 4. Retrieve jpg, gif, png, webp and other image files in the project directory, generate a local image list, write a script to scan the image files in the local file system, and generate an image list;

[0113] Step 5. For each of the 4 images, call the feature vector method in step 3 to generate the feature vector corresponding to the image. The image and the corresponding vector are saved to native.h5 at the same time.

[0114] Step 6. Get the cloud image list, scan the image files in the cloud storage, and generate the cloud image list;

[0115] Step 7. For each of the 6 images, call the feature vector method in step 3 to generate the feature vector corresponding to the image, save it to could.h5, call the feature vector extraction method for each cloud image, and save the feature vector to `cloud.h5`;

[0116] Step 8. Move the generated native.h5 and could.h5 files to the production environment for use by the query service;

[0117] Step 9. Use CherryPy to build a Python-based query service, preload the Keras VGGNet model and the method for extracting convolutional features. The function is to obtain the path of the input image, input VGGNet, output the target image vector, and perform inner product operations with the vectors in native.h5 and could.h5 to obtain the similarity, and sort them in descending order according to the similarity;

[0118] Step 10. Encapsulate the PHP page to provide image upload, API call and result display page;

[0119] Step 11. The image upload inspection module includes the project code submission link and the cloud image upload link. The query module needs to be called. If the result contains a result with a similarity of x (for example, 0.9), it needs to enter the manual processing flow. If the old image is adopted, the submission is returned. If the new image is adopted, the new image is merged. The image upload inspection module is implemented to monitor the code submission and cloud image upload events, and the query module is called to process the images whose similarity reaches the threshold for manual review and processing.

[0120] The present invention improves the reuse of materials. When new images are stored in the database, the system can recommend similar images for reuse, thereby reducing the material expansion rate and improving the reuse rate of materials.

[0121] The present invention can quickly locate the target image. When the image material is upgraded, the system can quickly find the target image, reduce the cost of manual screening, and reduce the risk of missing images.

[0122] The present invention improves the retrieval accuracy and efficiency, greatly improves the retrieval accuracy and deletion efficiency in image retrieval and deletion work, can more accurately identify similar images, and provide more accurate query results.

[0123] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A project image material management system based on deep learning, characterized in that: include: Image resource scanning and aggregation module, used to scan local and cloud image resources, preliminarily organize image files and generate image lists; In the image resource scanning and summarization module, you can write scripts or use existing tools to scan image files in the local file system and cloud storage. The supported file formats include jpg, gif, png, and webp. The scanned image file paths are summarized to generate a local image list and a cloud image list. Image resource processing module: Image resource processing specifically includes image vectorization, persistence, and query processing. It uses the convolutional neural network (CNN) algorithm to vectorize image resources and store them persistently for query; Image processing is performed in the image resource processing module. The image processing methods include: Create a VGGNet model: Import the Keras library, create and load the pre-trained VGGNet model, Feature vector extraction: encapsulates the method of extracting the last layer of convolution features of VGG16 and converts the image into a feature vector. Persistent storage: feature vectors are persistently stored in the local native.h5 and cloud.h5 databases; The vector query module provides vector query functions, vectorizes the input image, calculates the vector similarity of the existing images, and returns a result list based on the similarity score; Vector query methods include: Input image vectorization: Convert the input image into a feature vector. Vector similarity calculation: Calculate the similarity between the input image feature vector and the feature vector stored in the database. Result sorting and return: Sort by similarity score and return a list of images with the highest similarity; The scene image processing module uses similar image query functions in different application scenarios to upgrade logos and add images; The scene image processing module processes the scene image in the following ways: Input different styles of logos and get the most similar images for manual replacement. Query similar pictures, integrate and merge them; OCR processing and filtering module, which extracts text information from images through OCR technology and provides text-based filtering functions; Image query module, which provides image upload, API call and result display page for encapsulated PHP page; The image upload inspection module includes the project code submission link and the cloud image upload link, which requires calling the image query module; The image upload check method is: If the results contain results with a similarity greater than x, a manual processing process is required; If the old image is adopted, the submission will be rejected. If the new image is adopted, the new image will be merged in.

2. A project image material management system based on deep learning according to claim 1, characterized in that: The project picture material management method of the project picture material management system is: Step 1. Download the project code according to the project address and update the dependencies; Step 2. Create `native.h5` and `cloud.h5` databases and initialize them to store image vectors; Step 3. Introduce Keras, create the VGGNet model, and encapsulate the method of extracting the last layer of convolution features of VGG16; Step 4. Retrieve the image files in the project directory and generate a local image list; Step 5. Call the feature vector extraction method in step 3 for each image, generate feature vectors, and save them to `native.h5`; Step 6. Get the cloud image list, extract feature vectors for each image, and save them to `cloud.h5`; Step 7. Move the generated `native.h5` and `cloud.h5` files to the production environment; Step 8. Build a Python-based query service through CherryPy, preload the Keras VGGNet model and feature extraction method, and implement the similarity calculation and result sorting of image vectors; Step 9. Encapsulate the PHP page, provide the image upload function, and call the API to display the results; Step 10. Implement the image upload inspection module, including the project code submission link and the cloud image upload link, call the query module, process the images whose similarity reaches the threshold, and conduct manual review and processing.

Images