Software system based on image recognition and 3D modeling

Abstract

The invention discloses a software system based on image recognition and 3D modeling, and relates to the technical field of information collection and conversion. The system at least comprises a camera scanning and image clipping module, an image magnification and optimization processing module, an image type identification and modeling module, a shadow measurement and 3D modeling module, an AI algorithm depth information extraction module, a stretching, engraving and refining module, a texture adding module, an illumination rendering module and a client rendering and interaction function module. According to the system, the camera is used for scanning the picture in the book and uploading the picture to the server, the server recognizes the article type through a visual recognition algorithm, then 3D modeling and rendering are conducted through the point cloud technology, the conversion efficiency from 2D to 3D is improved through the system, and by providing the functions of rotation, splitting and detail viewing of the 3D model, the system is more efficient and more convenient to use. And the interactive experience of the user is greatly enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of information collection and conversion, and in particular to a software system based on image recognition and 3D modeling. Background Art

[0002] In the operation of scanning pictures from books, the conventional steps in the prior art are:

[0003] Tools: You will need a scanner or mobile scanning app and good lighting.

[0004] Choose pages: Spread the pages carefully to avoid damaging them. Make sure the pages are flat;

[0005] Scan: Scan the image using a scanner or mobile app. Adjust the settings to get a clear image;

[0006] Process images: Use image editing software to retouch images, such as cropping, adjusting brightness or contrast;

[0007] Save the file: Save in an appropriate format, such as JPEG or PDF, and make sure the file name and storage location are easily identifiable.

[0008] This type of operation can often only focus on 2D images and cannot better achieve confidence collection applications in the 3D dimension;

[0009] Therefore, the prior art often has the following technical problems:

[0010] Traditional paper document digitization mainly focuses on converting text information into electronic format. However, for documents containing complex images or graphics, existing technologies are often unable to effectively convert these images from 2D to 3D models. This limits the further analysis and application of document content, especially in scenarios that require detailed models and high interactivity.

[0011] Therefore, a new solution to the above problems needs to be proposed. Summary of the invention

[0012] The purpose of the present invention is to provide a software system based on image recognition and 3D modeling to solve the technical problems raised in the background technology.

[0013] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a software system based on image recognition and 3D modeling, comprising at least a camera scanning and image cropping module, an image magnification and optimization processing module, an image type recognition and modeling module, a light and shadow measurement and 3D modeling module, an AI algorithm depth information extraction module, a stretching, carving and refinement module, a texture adding module, a lighting rendering module and a client rendering and interactive function module;

[0014] The image magnification and optimization module is on the server, and is used to improve the resolution of the image through an image magnification algorithm, and adjust the brightness, contrast and color saturation of the image through the algorithm;

[0015] The image recognition and modeling module is used to identify the type of image and perform basic modeling in the database;

[0016] The light and shadow measurement and 3D modeling module is used to analyze the light and shadow information in the image, identify common points as reference markers, and calculate the distances and angles between different elements in the image to generate the basis of the 3D model;

[0017] The AI ​​algorithm depth information extraction module is used to extract depth information from 2D images and construct the shape and contour of the 3D model;

[0018] The stretching, carving and thinning module is used to stretch the 2D shape into a three-dimensional shape and perform carving and thinning processing;

[0019] The texture adding module is used to add texture and details to the 3D model;

[0020] The lighting rendering module is used to add light source information and render the 3D model to improve the visual effect;

[0021] The client rendering and interaction function module is used to render the generated 3D model on the client.

[0022] Further, the camera scanning and image cropping module includes a camera scanning module and an image cropping module;

[0023] The camera scanning module is used to scan the image part in the paper document;

[0024] The image cropping module is used to automatically crop the image portion and send it to the server;

[0025] The image cropping module also includes image edge detection and cropping algorithms to accurately extract image parts.

[0026] Furthermore, the image magnification and optimization processing module includes a resolution enhancement algorithm saturation and a contrast enhancement algorithm to better achieve.

[0027] Furthermore, the image recognition and modeling module includes at least a deep learning model for improving the classification accuracy of image recognition.

[0028] Furthermore, the light and shadow measurement and 3D modeling module at least uses point cloud technology to accurately calculate the distance and angle between different elements.

[0029] Furthermore, the AI ​​algorithm depth information extraction module uses at least a convolutional neural network (CNN) to extract depth information.

[0030] Furthermore, the stretching, carving and refinement module includes at least an adaptive mesh refinement algorithm for enhancing the detail expression of the model.

[0031] Furthermore, the texture adding module at least includes a texture mapping algorithm to improve the visual effect of the 3D model.

[0032] Furthermore, the client rendering and interaction function module provides at least basic interaction functions of the types of rotation, enlargement, movement and splitting.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] 1. The present invention uses a camera to scan pictures in books and uploads them to a server. The server uses a visual recognition algorithm to identify the type of items, and then applies point cloud technology for 3D modeling and rendering. The system not only improves the conversion efficiency from 2D to 3D, but also greatly enhances the user's interactive experience by providing the functions of rotating, splitting and viewing details of 3D models.

[0035] 2. The present invention utilizes point cloud, visual recognition, image magnification algorithm, light and shadow measurement algorithm, AI depth information extraction and other technologies to convert 2D images into 3D models, providing users with a richer visual experience. It can be used in application scenarios such as teaching where more detailed 3D models need to be observed from 2D images. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0037] Figure 1 It is a schematic diagram of the overall structure of the present invention. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0039] See also Figure 1, a software system based on image recognition and 3D modeling, including at least a camera scanning and image cropping module, an image magnification and optimization processing module, an image type recognition and modeling module, a light and shadow measurement and 3D modeling module, an AI algorithm depth information extraction module, a stretching, carving and refinement module, a texture adding module, a lighting rendering module and a client rendering and interaction function module;

[0040] The image magnification and optimization module is located on the server. The image magnification and optimization module is used to improve the resolution of the image through an image magnification algorithm, and adjust the brightness, contrast and color saturation of the image through the algorithm;

[0041] The image recognition and modeling module is used to identify the type of image and perform basic modeling in the database;

[0042] The photometry and 3D modeling module is used to analyze the light and shadow information in the image, identify common points as reference markers, and calculate the distances and angles between different elements in the image to generate the basis of the 3D model;

[0043] The AI ​​algorithm depth information extraction module is used to extract depth information from 2D images and construct the shape and outline of the 3D model;

[0044] The Extrusion, Sculpting and Refining module is used to extrude 2D shapes into 3D shapes and perform sculpting and refinement.

[0045] The texture adding module is used to add texture and details to 3D models;

[0046] The lighting rendering module is used to add light source information and render 3D models to improve visual effects;

[0047] The client rendering and interaction function module is used to render the generated 3D model on the client.

[0048] The camera scanning and image cropping module includes a camera scanning module and an image cropping module;

[0049] The camera scanning module is used to scan the image part of the paper document;

[0050] The image cropping module is used to automatically crop the image part and send it to the server;

[0051] The image cropping module also includes image edge detection and cropping algorithms to accurately extract image parts.

[0052] The image magnification and optimization processing module includes resolution enhancement algorithm saturation and contrast enhancement algorithm to better achieve.

[0053] The image recognition and modeling module includes at least a deep learning model, which is used to improve the classification accuracy of image recognition.

[0054] The photometry and 3D modeling modules use at least point cloud technology to accurately calculate the distances and angles between different elements.

[0055] The AI ​​algorithm depth information extraction module uses at least a convolutional neural network (CNN) to extract depth information.

[0056] The stretching, sculpting and refinement module includes at least an adaptive mesh refinement algorithm for enhancing the detail performance of the model.

[0057] The texture adding module at least includes a texture mapping algorithm to improve the visual effect of the 3D model.

[0058] The client rendering and interaction function module provides at least basic interaction functions such as rotation, enlargement, movement and splitting.

[0059] Working principle:

[0060] Log in to the system through the client;

[0061] The camera scanning module uses a camera to shoot paper documents and obtain the image part of the document. The camera converts the image in the paper document into a digital signal and captures the image details through the image sensor to form a preliminary digital image file.

[0062] The image cropping module processes the scanned image, automatically crops out the image portion, and uploads the processed image to the server. The image cropping module uses image edge detection and cropping algorithms to identify the edges and regions of the image, remove irrelevant parts, extract the image region of interest, and then send it to the server for further processing.

[0063] The image enlargement and optimization module increases the resolution of the image and optimizes the image quality to improve the accuracy of subsequent processing. On the server side, image enlargement algorithms (such as bilinear interpolation or super-resolution technology) are used to increase the resolution of the image. Image optimization includes denoising, color correction, and contrast adjustment to enhance image details and recognition capabilities.

[0064] The image recognition and modeling module uses visual recognition algorithms to identify the type of images and perform basic modeling. The server uses deep learning models (such as convolutional neural networks (CNNs)) to classify and extract features from images, and then searches the database for matching models or templates for preliminary modeling.

[0065] The light and shadow measurement and 3D modeling module analyzes the light and shadow information in the image, identifies the common points in the image as reference marks, calculates the distance and angle between elements, and generates the basis for the 3D model. This module uses light and shadow measurement technology to analyze the light and shadow changes in the image and identify the positional relationship of different elements in the image. Point cloud technology is used to calculate the distance and angle between elements and generate the basic data for the 3D model.

[0066] The AI ​​algorithm depth information extraction module extracts depth information from 2D images and constructs the shape and contour of the 3D model. The AI ​​algorithm (such as the convolutional neural network CNN in deep learning) analyzes the features of the image, estimates the depth information, and generates the contour and shape data of the 3D model.

[0067] Function: The stretching, carving and refinement module converts 2D shapes into 3D shapes and performs refinement. This module uses 3D modeling technology to stretch 2D images into 3D shapes and enhances the details of the model, including texture and volume, through carving and refinement algorithms.

[0068] The texture adding module adds texture and details to the 3D model to make it more vivid and realistic. Through texture mapping technology, the texture information of the image is mapped to the surface of the 3D model to increase the level of detail and realism.

[0069] Lighting rendering, the lighting rendering module adds light source information and renders the 3D model to improve the visual effect. This module generates high-quality lighting rendering effects and enhances the visual performance of the model by setting virtual light sources and calculating the impact of lighting on the model.

[0070] Client rendering and interaction function: The client rendering and interaction function module displays the generated 3D model on the client and provides user interaction function. On the client, the 3D model generated by the 3D engine can be rotated, scaled, and translated through the interface to view different angles and details of the model. The interaction function allows users to operate with the model in real time, improving the user experience.

[0071] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A software system based on image recognition and 3D modeling, characterized by: At least including camera scanning and image cropping module, image magnification and optimization processing module, image type recognition and modeling module, light and shadow measurement and 3D modeling module, AI algorithm depth information extraction module, stretching, carving and refinement module, texture adding module, lighting rendering module and client rendering and interactive function module; The image magnification and optimization module is on the server, and is used to improve the resolution of the image through an image magnification algorithm, and adjust the brightness, contrast and color saturation of the image through the algorithm; The image recognition and modeling module is used to identify the type of image and perform basic modeling in the database; The light and shadow measurement and 3D modeling module is used to analyze the light and shadow information in the image, identify common points as reference markers, and calculate the distances and angles between different elements in the image to generate the basis of the 3D model; The AI ​​algorithm depth information extraction module is used to extract depth information from 2D images and construct the shape and contour of the 3D model; The stretching, carving and thinning module is used to stretch the 2D shape into a three-dimensional shape and perform carving and thinning processing; The texture adding module is used to add texture and details to the 3D model; The lighting rendering module is used to add light source information and render the 3D model to improve the visual effect; The client rendering and interaction function module is used to render the generated 3D model on the client.

2. A software system based on image recognition and 3D modeling according to claim 1, characterized in that: The camera scanning and image cropping module includes a camera scanning module and an image cropping module; The camera scanning module is used to scan the image part in the paper document; The image cropping module is used to automatically crop the image portion and send it to the server; The image cropping module also includes image edge detection and cropping algorithms to accurately extract image parts.

3. The software system based on image recognition and 3D modeling according to claim 1, characterized in that: The image magnification and optimization processing module includes a resolution enhancement algorithm, saturation, and contrast enhancement algorithm to better achieve.

4. The software system based on image recognition and 3D modeling according to claim 1, characterized in that: The image recognition and modeling module includes at least a deep learning model, which is used to improve the classification accuracy of image recognition.

5. The software system based on image recognition and 3D modeling according to claim 1, characterized in that: The light and shadow measurement and 3D modeling module at least uses point cloud technology to accurately calculate the distance and angle between different elements.

6. The software system based on image recognition and 3D modeling according to claim 1, characterized in that: The AI ​​algorithm depth information extraction module uses at least a convolutional neural network to extract depth information.

7. The software system based on image recognition and 3D modeling according to claim 1, characterized in that: The stretching, carving and refinement module at least includes an adaptive mesh refinement algorithm for enhancing the detail expression of the model.

8. The software system based on image recognition and 3D modeling according to claim 1, characterized in that: The texture adding module at least includes a texture mapping algorithm to improve the visual effect of the 3D model.

9. The software system based on image recognition and 3D modeling according to claim 1, characterized in that: The client rendering and interaction function module provides at least basic interaction functions of the types of rotation, enlargement, movement and splitting.

Images