Image recognition and modeling method, device, electronic device and storage medium

Abstract

The present invention discloses an image recognition and modeling method, device, electronic device and storage medium, and belongs to the field of image processing technology. The image recognition and modeling method of the present invention comprises acquiring a target two-dimensional image, analyzing a target object in the target two-dimensional image according to a YOLO algorithm, and obtaining characteristic parameters of the target object. According to the characteristic parameters and the YOLO algorithm, a three-dimensional model corresponding to each target object is determined; all three-dimensional models are spliced ​​to obtain a three-dimensional image corresponding to the target two-dimensional image. This image recognition and modeling method can complete modeling according to a target two-dimensional image, greatly shorten the modeling time, and improve image processing efficiency.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image recognition and modeling method, device, electronic device and storage medium. Background Art

[0002] At present, when performing three-dimensional modeling on images, it is often necessary to split the images, model them, and then merge them, which is time-consuming and costly. Therefore, how to provide an image recognition and modeling method that can greatly shorten the modeling time and improve image processing efficiency has become an urgent problem to be solved. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes an image recognition and modeling method, which can greatly shorten the modeling time and improve the image processing efficiency.

[0004] The present invention also provides an image recognition and modeling device having the above-mentioned image recognition and modeling method.

[0005] The present invention also provides an electronic device having the above-mentioned image recognition and modeling method.

[0006] The present invention also provides a computer-readable storage medium.

[0007] The image recognition and modeling method according to the first aspect of the present invention includes:

[0008] Acquire a target two-dimensional image;

[0009] Analyze the target object in the target two-dimensional image according to the YOLO algorithm to obtain feature parameters of the target object;

[0010] Determine a three-dimensional model corresponding to each of the target objects according to the feature parameters and the YOLO algorithm;

[0011] All the three-dimensional models are combined to obtain a three-dimensional image corresponding to the target two-dimensional image.

[0012] The image recognition and modeling method according to the embodiment of the present invention has at least the following beneficial effects: the image recognition and modeling method obtains a target two-dimensional image, analyzes the target object in the target two-dimensional image according to the YOLO algorithm, and obtains the characteristic parameters of the target object. According to the characteristic parameters and the YOLO algorithm, the three-dimensional model corresponding to each target object is determined, which can shorten the time for model construction. Finally, all three-dimensional models are spliced ​​to obtain a three-dimensional image corresponding to the target two-dimensional image. Modeling is completed through a target two-dimensional image, which can greatly shorten the modeling time and improve image processing efficiency.

[0013] According to some embodiments of the present invention, analyzing the target object in the target two-dimensional image according to the YOLO algorithm to obtain the characteristic parameters of the target object includes:

[0014] According to the YOLO algorithm, the target two-dimensional image is divided to obtain a square matrix;

[0015] The target object is analyzed according to the grid matrix to obtain the position parameters and confidence probability of the target object.

[0016] According to some embodiments of the present invention, the feature parameter includes a category parameter, and determining the three-dimensional model corresponding to each of the target objects according to the feature parameter and the YOLO algorithm includes:

[0017] Searching for a reference model according to the category parameter and the YOLO algorithm;

[0018] Screening and classifying the searched reference models to obtain a target reference model set;

[0019] A three-dimensional model corresponding to each of the target objects is determined according to the target reference model set.

[0020] According to some embodiments of the present invention, screening and classifying the searched reference models to obtain a target reference model set includes:

[0021] According to the preset color parameters, the searched reference models are screened and classified to obtain a target reference model set.

[0022] According to some embodiments of the present invention, determining the three-dimensional model corresponding to each of the target objects according to the target reference model set includes:

[0023] Performing similarity measurement retrieval on all three-dimensional models in the target reference model set to determine the similarity between each three-dimensional model and the target object;

[0024] According to the similarity, a three-dimensional model corresponding to each of the target objects is determined.

[0025] According to some embodiments of the present invention, the feature parameters include position parameters and confidence probabilities, and the step of combining all the three-dimensional models to obtain a three-dimensional image corresponding to the target two-dimensional image includes:

[0026] All the three-dimensional models are spliced ​​together according to the position parameters and the confidence probability to obtain the three-dimensional image.

[0027] According to some embodiments of the present invention, the image recognition and modeling method further includes:

[0028] The three-dimensional image is calibrated to obtain a final three-dimensional image.

[0029] According to the second aspect of the present invention, the image recognition and modeling device includes:

[0030] A two-dimensional image acquisition module is used to acquire a target two-dimensional image;

[0031] A feature parameter acquisition module is used to analyze the target object in the target two-dimensional image according to the YOLO algorithm to obtain the feature parameters of the target object;

[0032] A three-dimensional model determination module, used to determine a three-dimensional model corresponding to each of the target objects according to the feature parameters and the YOLO algorithm;

[0033] The three-dimensional image generation module is used to combine all the three-dimensional models to obtain a three-dimensional image corresponding to the target two-dimensional image.

[0034] The image recognition and modeling device according to the embodiment of the present invention has at least the following beneficial effects: the image recognition and modeling device obtains the target two-dimensional image through the two-dimensional image acquisition module, and the feature parameter acquisition module analyzes the target object in the target two-dimensional image according to the YOLO algorithm to obtain the feature parameters of the target object. The three-dimensional model determination module determines the three-dimensional model corresponding to each target object according to the feature parameters and the YOLO algorithm, which can shorten the time for model construction. Finally, the three-dimensional image generation module combines all three-dimensional models to obtain a three-dimensional image corresponding to the target two-dimensional image, and completes the modeling through one target two-dimensional image, which can greatly shorten the modeling time and improve the image processing efficiency.

[0035] An electronic device according to an embodiment of a third aspect of the present invention includes:

[0036] at least one processor, and

[0037] a memory communicatively connected to the at least one processor; wherein,

[0038] The memory stores instructions, and the instructions are executed by the at least one processor so that when the at least one processor executes the instructions, the image recognition and modeling method as described in the first aspect embodiment is implemented.

[0039] The electronic device according to the embodiment of the present invention has at least the following beneficial effects: the electronic device adopts the above-mentioned image recognition and modeling method, obtains the target two-dimensional image, analyzes the target object in the target two-dimensional image according to the YOLO algorithm, and obtains the characteristic parameters of the target object. According to the characteristic parameters and the YOLO algorithm, the three-dimensional model corresponding to each target object is determined, which can shorten the time for model construction. Finally, all three-dimensional models are spliced ​​to obtain a three-dimensional image corresponding to the target two-dimensional image, and the modeling is completed through one target two-dimensional image, which can greatly shorten the modeling time and improve the image processing efficiency.

[0040] According to the computer-readable storage medium of the fourth aspect embodiment of the present invention, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the image recognition and modeling method as described in the first aspect embodiment.

[0041] The computer-readable storage medium according to the embodiment of the present invention has at least the following beneficial effects: the computer-readable storage medium executes the above-mentioned image recognition and modeling method by acquiring a target two-dimensional image, analyzing the target object in the target two-dimensional image according to the YOLO algorithm, and obtaining the characteristic parameters of the target object. According to the characteristic parameters and the YOLO algorithm, the three-dimensional model corresponding to each target object is determined, which can shorten the time for model construction. Finally, all three-dimensional models are spliced ​​to obtain a three-dimensional image corresponding to the target two-dimensional image, and modeling is completed through one target two-dimensional image, which can greatly shorten the modeling time and improve image processing efficiency.

[0042] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present invention will be further described below with reference to the accompanying drawings and embodiments, wherein:

[0044] Figure 1 is a flow chart of an image recognition and modeling method according to an embodiment of the present invention;

[0045] Figure 2 for Figure 1 Flow chart of step S200;

[0046] Figure 3 for Figure 1 Flow chart of step S300;

[0047] Figure 4 for Figure 3 Flow chart of step S330;

[0048] Figure 5 Schematic diagram of the structure of the image recognition and modeling device according to an embodiment of the present invention.

[0049] Reference numerals: 510, two-dimensional image acquisition module; 520, feature parameter acquisition module; 530, three-dimensional model determination module; 540, three-dimensional image generation module. DETAILED DESCRIPTION

[0050] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0051] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0052] In the description of the present invention, "several" means more than one, "many" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0053] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0054] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0055] First, refer to Figure 1 , the image recognition and modeling method of the embodiment of the present invention includes:

[0056] S100, acquiring a target two-dimensional image;

[0057] S200, analyzing the target object in the target two-dimensional image according to the YOLO algorithm to obtain feature parameters of the target object;

[0058] S300, determining a three-dimensional model corresponding to each target object according to the feature parameters and the YOLO algorithm;

[0059] S400: performing mosaic processing on all three-dimensional models to obtain a three-dimensional image corresponding to the target two-dimensional image.

[0060] When recognizing and modeling a two-dimensional image, first obtain a target two-dimensional image. This target two-dimensional image can be a person image, a landscape image, a building image, an animal image, etc., without limitation. According to the YOLO algorithm, the target object in the target two-dimensional image is analyzed, and the category, position, etc. of the target object are analyzed by the YOLO algorithm, so that the characteristic parameters of the target object can be obtained more conveniently, wherein the characteristic parameters include position parameters, confidence probability, category parameters, etc. According to the category parameters and the YOLO algorithm, a reference model is searched in a preset resource library. By screening and classifying the searched reference models, the three-dimensional model corresponding to each target object can be determined. According to the position parameters and confidence probability of each target object, the position of the corresponding three-dimensional model is determined, and all three-dimensional models are spliced ​​to obtain a three-dimensional image corresponding to the target two-dimensional image. Through this image recognition and modeling method, modeling can be completed according to a target two-dimensional image, which greatly shortens the modeling time and improves the image processing efficiency.

[0061] It should be noted that the YOLO algorithm is an object recognition and positioning algorithm based on a deep neural network. The YOLO algorithm solves object detection as a regression problem. After one inference of the input image, the positions of all objects in the image, their categories, and the corresponding confidence probabilities can be obtained. The detection results are divided into two parts for solution through detection algorithms such as rcnn, fast rcnn, or fasterrcnn: object category (i.e., classification problem) and object position (i.e., regression problem).

[0062] Reference Figure 2 In some embodiments, step S200 includes:

[0063] S210, dividing the target two-dimensional image according to the YOLO algorithm to obtain a grid matrix;

[0064] S220, analyzing the target object according to the grid matrix to obtain the location parameters and confidence probability of the target object.

[0065] When recognizing and modeling a two-dimensional image, first obtain the target two-dimensional image. According to the YOLO algorithm, divide the target two-dimensional image into a square matrix. For example, divide the target two-dimensional image into an S×S square matrix. The position and confidence value of the target object are detected through each square. It should be noted that the confidence value represents the confidence that the square contains the target object. The confidence value can be expressed as If there is no target object in the current square, the confidence value is 0. In addition, in order to make the predicted confidence value consistent with the result of the manually labeled evaluation function, each square contains five values: x, y, w, h and confidence. Among them, (x, y) represents the center coordinates of the target object associated with a certain square; (w, h) represents the width and height of the target object related to the full image information; the confidence represents the result of the manually labeled evaluation function for predicting the target object. When predicting the confidence probability of the target object through each square, the predicted conditional probability value of each square is C(Pr(Classi|Object)); among them, the probability value C represents the probability that the square contains a target object, and each square only predicts one category of probability. In this way, each target object can obtain a specific category confidence score by multiplying the category probability and the confidence:

[0066] The confidence score indicates the probability of this category appearing in the target object and the suitability of the category with the target object. The YOLO algorithm can easily obtain the location parameters and confidence probability of the target object with high accuracy.

[0067] Reference Figure 3 In some embodiments, the feature parameter includes a category parameter, and step S300 includes:

[0068] S310, searching for a reference model according to the category parameter and the YOLO algorithm;

[0069] S320, screening and classifying the searched reference models to obtain a target reference model set;

[0070] S330: Determine a three-dimensional model corresponding to each target object according to the target reference model set.

[0071] When recognizing and modeling a two-dimensional image, first obtain a target two-dimensional image. Analyze the target object in the target two-dimensional image according to the YOLO algorithm, and analyze the category, position, etc. of the target object through the YOLO algorithm, so that the characteristic parameters of the target object can be obtained more conveniently, wherein the characteristic parameters include position parameters, confidence probability, category parameters, etc. According to the category parameters and the YOLO algorithm, search for reference models in the preset resource library. By searching in the preset resource library, a large number of reference models matching the target object can be obtained. In order to make the determined three-dimensional model closest to the target object, the searched reference models are screened and classified according to the preset screening and classification basis to obtain a target reference model set. The reference models in the target reference model set are screened twice, so that the closest three-dimensional model can be easily selected, and this three-dimensional model is used as the three-dimensional model corresponding to the target object. The accuracy is high.

[0072] In some embodiments, step S320 includes:

[0073] According to the preset color parameters, the searched reference models are screened and classified to obtain a target reference model set.

[0074] In order to improve the accuracy of modeling, the searched reference models can be screened and classified according to the preset color parameters, and the models with the same color as the target object can be classified into one category to obtain a target reference model set. Then the models in the target reference model set are screened to determine the three-dimensional model corresponding to the target object. It should be noted that the color parameters may include color, resolution, RGB value, etc. In some other embodiments, the model size, etc. can also be used as a basis for screening and classification, but it is not limited thereto.

[0075] Reference Figure 4 In some embodiments, step S330 includes:

[0076] S331, performing similarity measurement retrieval on all three-dimensional models in the target reference model set to determine the similarity between each three-dimensional model and the target object;

[0077] S332: Determine the three-dimensional model corresponding to each target object according to the similarity.

[0078] In order to make the selected 3D model closest to the target object, similarity measurement retrieval is performed on all 3D models in the target reference model set to determine the similarity between each 3D model and the target object. By comparing the similarity between each 3D model and the target object, the 3D model with the highest similarity is selected as the 3D model corresponding to the target object, which can improve the accuracy of modeling, and also improve the accuracy of image recognition and conversion, so that each feature in the final 3D image is closer to the target 2D image.

[0079] In some embodiments, step S400 includes:

[0080] According to the position parameters and confidence probability, all three-dimensional models are spliced ​​to obtain a three-dimensional image.

[0081] When all 3D models are stitched together, the angle and position of the corresponding 3D model in the whole image are determined based on the position of the target object on the X-axis, Y-axis and Z-axis, as well as the scale of the target object, etc., included in the position parameters. Then all 3D models are stitched together so that a complete 3D image is formed by all 3D models. This method can greatly shorten the modeling time and improve the image processing efficiency.

[0082] In some embodiments, the image recognition and modeling method further includes:

[0083] The three-dimensional image is calibrated to obtain a final three-dimensional image.

[0084] In order to improve the accuracy of the three-dimensional image, after all three-dimensional models are combined to obtain the three-dimensional image, the three-dimensional image can be calibrated. For example, the coordinates and colors of each feature of the three-dimensional image can be corrected. By calibrating the three-dimensional image to obtain the final three-dimensional image, the accuracy of image recognition and conversion is improved, so that each feature in the final three-dimensional image is closer to the target two-dimensional image.

[0085] Second, refer to Figure 5 , the image recognition and modeling device of the embodiment of the present invention comprises:

[0086] A two-dimensional image acquisition module 510 is used to acquire a target two-dimensional image;

[0087] A feature parameter acquisition module 520 is used to analyze the target object in the target two-dimensional image according to the YOLO algorithm to obtain feature parameters of the target object;

[0088] A three-dimensional model determination module 530, for determining a three-dimensional model corresponding to each target object according to the feature parameters and the YOLO algorithm;

[0089] The three-dimensional image generation module 540 is used to perform a splicing process on all three-dimensional models to obtain a three-dimensional image corresponding to the target two-dimensional image.

[0090] When recognizing and modeling a two-dimensional image, the two-dimensional image acquisition module 510 first acquires a target two-dimensional image. This target two-dimensional image can be a person image, a landscape image, an animal image, etc., without limitation. The feature parameter acquisition module 520 analyzes the target object in the target two-dimensional image according to the YOLO algorithm, and analyzes the category, position, etc. of the target object through the YOLO algorithm, so that the feature parameters of the target object can be obtained more conveniently, wherein the feature parameters include position parameters, confidence probability, category parameters, etc. The three-dimensional model determination module 530 searches for reference models in a preset resource library according to the category parameters and the YOLO algorithm. By screening and classifying the searched reference models, the three-dimensional model corresponding to each target object can be determined. The three-dimensional image generation module 540 determines the position of the corresponding three-dimensional model according to the position parameters and confidence probability of each target object, and combines all three-dimensional models to obtain a three-dimensional image corresponding to the target two-dimensional image. Through this image recognition and modeling method, modeling can be completed according to a target two-dimensional image, which greatly shortens the modeling time and improves the image processing efficiency.

[0091] In a third aspect, an electronic device according to an embodiment of the present invention comprises at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions, and the instructions are executed by the at least one processor, so that when the at least one processor executes the instructions, an image recognition and modeling method as in the embodiment of the first aspect is implemented.

[0092] The electronic device according to the embodiment of the present invention has at least the following beneficial effects: the electronic device adopts the above-mentioned image recognition and modeling method, obtains the target two-dimensional image, analyzes the target object in the target two-dimensional image according to the YOLO algorithm, and obtains the characteristic parameters of the target object. According to the characteristic parameters and the YOLO algorithm, the three-dimensional model corresponding to each target object is determined, which can shorten the time for model construction. Finally, all three-dimensional models are spliced ​​to obtain a three-dimensional image corresponding to the target two-dimensional image, and the modeling is completed through one target two-dimensional image, which can greatly shorten the modeling time and improve the image processing efficiency.

[0093] In a fourth aspect, the present invention further provides a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the image recognition and modeling method of the first aspect embodiment.

[0094] The computer-readable storage medium according to the embodiment of the present invention has at least the following beneficial effects: the computer-readable storage medium executes the above-mentioned image recognition and modeling method, obtains the target two-dimensional image, analyzes the target object in the target two-dimensional image according to the YOLO algorithm, and obtains the characteristic parameters of the target object. According to the characteristic parameters and the YOLO algorithm, the three-dimensional model corresponding to each target object is determined, which can shorten the time for model construction. Finally, all three-dimensional models are spliced ​​to obtain a three-dimensional image corresponding to the target two-dimensional image, and the modeling is completed through one target two-dimensional image, which can greatly shorten the modeling time and improve the image processing efficiency.

[0095] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in the relevant technical field without departing from the purpose of the present invention. In addition, the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

Claims

1. An image recognition and modeling method, characterized in that: include: Acquire a target two-dimensional image; Analyzing the target object in the target two-dimensional image according to the YOLO algorithm to obtain feature parameters of the target object, including: dividing the target two-dimensional image according to the YOLO algorithm to obtain a grid matrix, and analyzing the target object according to the grid matrix to obtain position parameters and confidence probability of the target object; Determine, according to the feature parameters and the YOLO algorithm, a three-dimensional model corresponding to each of the target objects, wherein the feature parameters include category parameters, and determine, according to the feature parameters and the YOLO algorithm, a three-dimensional model corresponding to each of the target objects, comprising: searching for reference models according to the category parameters and the YOLO algorithm, screening and classifying the searched reference models according to preset color parameters to obtain a target reference model set, performing similarity measurement retrieval on all three-dimensional models in the target reference model set, determining the similarity between each of the three-dimensional models and the target object, and determining, according to the similarity, a three-dimensional model corresponding to each of the target objects; All the three-dimensional models are stitched together to obtain a three-dimensional image corresponding to the target two-dimensional image, including: according to the position parameter and the confidence probability, all the three-dimensional models are stitched together to obtain the three-dimensional image.

2. The image recognition and modeling method according to claim 1, characterized in that: The image recognition and modeling method also includes: The three-dimensional image is calibrated to obtain a final three-dimensional image.

3. An image recognition and modeling device, characterized in that: include: A two-dimensional image acquisition module is used to acquire a target two-dimensional image; A feature parameter acquisition module is used to analyze the target object in the target two-dimensional image according to the YOLO algorithm to obtain the feature parameters of the target object, including: dividing the target two-dimensional image according to the YOLO algorithm to obtain a grid matrix, and analyzing the target object according to the grid matrix to obtain the position parameter and confidence probability of the target object; a three-dimensional model determination module, configured to determine a three-dimensional model corresponding to each of the target objects according to the feature parameters and the YOLO algorithm, wherein the feature parameters include category parameters, and the determining of the three-dimensional model corresponding to each of the target objects according to the feature parameters and the YOLO algorithm comprises: searching for reference models according to the category parameters and the YOLO algorithm, screening and classifying the searched reference models according to preset color parameters to obtain a target reference model set, performing similarity measurement retrieval on all three-dimensional models in the target reference model set, determining the similarity between each of the three-dimensional models and the target object, and determining the three-dimensional model corresponding to each of the target objects according to the similarity; The three-dimensional image generation module is used to combine all the three-dimensional models to obtain a three-dimensional image corresponding to the target two-dimensional image, including: combining all the three-dimensional models according to the position parameters and the confidence probability to obtain the three-dimensional image.

4. An electronic device, characterized in that include: at least one processor, and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions, and the instructions are executed by the at least one processor, so that the at least one processor implements the image recognition and modeling method as described in any one of claims 1 to 2 when executing the instructions.

5. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the image recognition and modeling method according to any one of claims 1 to 2.

Images