Method, device and computer equipment for establishing sparse image template of object

Abstract

The embodiments of the present application provide a method, device and computer equipment for establishing a sparse image template of an object, and relate to the field of computer vision. The present application determines the initial number of virtual cameras and multiple initial points according to the three-dimensional model of the target object and the number of preset points; creates virtual cameras corresponding to the initial points one by one, and the position of the virtual cameras is the same as the position of the corresponding initial points; controls each of the virtual cameras to shoot with the perspective toward the center point of the three-dimensional model as the target perspective, and obtains the template image corresponding to each initial point; performs image similarity matching on the template images corresponding to each of the initial points, and selects multiple target images from the template images corresponding to each of the initial points according to the similarity matching results; obtains the template image of the target object according to the target image. It can reduce the number of template images of the target object in the template library while ensuring the matching accuracy of the template image.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a method, apparatus and computer device for establishing a sparse image template of an object. Background Art

[0002] With the development of modern manufacturing, the requirements for the intelligence of robots are getting higher and higher. In order to achieve accurate grasping and assembly of parts, the posture of the parts can be solved through image matching so that the robot can grasp them. Image matching is the process of matching the template image of the target part object model obtained by the virtual camera in the computer-aided design (CAD) software with two or more images of the part object to be measured obtained during the actual operation. Since two-dimensional images cannot fully describe the appearance of three-dimensional objects, it is necessary to sample the part object to be measured in the CAD software at multiple perspectives through the camera in the CAD software to obtain the template image.

[0003] At present, in order to avoid the error caused by discrete sampling, a large number of samples are generally taken to obtain the template image. For example, for a certain part, tens of thousands of samples can be taken, so that the number of template images reaches tens of thousands. This method can be regarded as using dense sampling to reduce the error caused by discrete sampling.

[0004] However, this dense sampling method requires a high storage space for the templates. In addition, too many templates will also lead to a decrease in matching efficiency. Summary of the invention

[0005] The purpose of the present application includes, for example, providing a method, apparatus and computer device for establishing a sparse image template of an object, which can reduce the number of template images of the target object in the template library while ensuring the accuracy of template image matching, thereby accelerating the image matching speed of the object.

[0006] The embodiments of the present application can be implemented as follows:

[0007] In a first aspect, the present application provides a method for establishing a sparse image template of an object, the method comprising:

[0008] Determine the initial number of virtual cameras and multiple initial points according to the three-dimensional model of the target object and the number of preset points;

[0009] Creating a virtual camera corresponding to the initial point position one by one, wherein the position of the virtual camera is the same as the position of the corresponding initial point position;

[0010] Controlling each of the virtual cameras to shoot with a viewing angle toward the center point of the three-dimensional model as a target viewing angle to obtain a template image corresponding to each initial point position;

[0011] Performing image similarity matching on the template images corresponding to the initial points, and selecting a plurality of target images from the template images corresponding to the initial points according to the similarity matching results;

[0012] A template image of the target object is obtained according to the target image.

[0013] In an optional implementation, performing similarity matching on the template images corresponding to the initial points, and selecting a plurality of target images from the template images corresponding to the initial points according to the similarity matching results, includes:

[0014] Selecting at least one initial point to be measured from each of the initial points, and adding the at least one initial point to be measured to a sequence of points to be measured;

[0015] Acquire adjacent points of each initial point to be measured in the sequence of points to be measured, and add the adjacent points to the sequence of points to be compared;

[0016] Image similarity matching is cyclically performed on the current point in the sequence of points to be tested and the sequence of points to be compared, and the sequence of points to be tested and the sequence of points to be compared are cyclically updated according to the similarity matching results, and the template image corresponding to the point to be tested in the sequence of points to be tested at the end of the cycle is used as the target image.

[0017] In an optional implementation, the cyclically performing image similarity matching on the current point in the sequence of points to be tested and the sequence of points to be compared, and cyclically updating the sequence of points to be tested and the sequence of points to be compared according to the similarity matching result, and taking the template image corresponding to the point to be tested in the sequence of points to be tested at the end of the cycle as the target image, includes:

[0018] A. performing image similarity matching on the current point in the sequence of points to be measured and the sequence of points to be compared, and updating the sequence of points to be measured and the sequence of points to be compared according to the image similarity matching result to obtain a new sequence of points to be measured and a new sequence of points to be compared;

[0019] B. Determine whether the sequence of points to be compared is empty. If so, execute step C and take the next point in the points to be detected as the current point. If not, execute step A;

[0020] C. Determine whether the points to be detected have reached the end of the sequence and the sequence of points to be compared is empty. If so, end all loops and use the template image corresponding to the points to be detected in the current sequence of points to be detected as the target image. If not, execute step A.

[0021] In an optional implementation, performing image similarity matching on the current point in the point sequence to be tested and the point sequence to be compared, updating the point sequence to be tested and the point sequence to be compared according to the image similarity matching result, and obtaining a new point sequence to be tested and a new point sequence to be compared, comprises:

[0022] Performing image similarity matching on the template image corresponding to the first point in the sequence of points to be compared and the template image corresponding to the current point in the sequence of points to be measured, respectively, to obtain a matching result corresponding to the current point in the sequence of points to be measured, wherein the first point is any initial point in the sequence of points to be compared;

[0023] If the matching result between the current point in the sequence of points to be measured and the first point is greater than or equal to a first preset threshold, the first point is deleted from the sequence of points to be compared, and the adjacent points of the first point are added to the sequence of points to be compared to obtain a new sequence of points to be compared;

[0024] If the matching result between the current point in the point sequence to be measured and the first point is less than the first preset threshold, the first point is deleted from the point sequence to be compared, and the first point is added to the point sequence to be measured to obtain a new point sequence to be measured.

[0025] In an optional implementation, obtaining the template image of the target object according to the target image includes:

[0026] If the number of the target images is less than or equal to a second preset threshold, the target images are used as template images of the target objects.

[0027] In an optional implementation, obtaining the template image of the target object according to the target image includes:

[0028] If the number of the target images is greater than a second preset threshold, the number of preset points is increased, and the template image of the target object is re-determined according to the increased number of preset points.

[0029] In an optional implementation, determining the initial number of virtual cameras and the plurality of initial points according to the three-dimensional model of the target object and the number of preset points includes:

[0030] According to the three-dimensional model of the target object and the number of preset points, the Fibonacci spherical points are determined by a Fibonacci grid sampling method;

[0031] The number of the Fibonacci spherical points is used as the initial number, and the Fibonacci spherical points are used as the initial points.

[0032] In a second aspect, the present application provides a device for establishing a sparse image template of an object, the device comprising:

[0033] A determination module, used to determine the initial number of virtual cameras and a plurality of initial points according to the three-dimensional model of the target object and the number of preset points;

[0034] A creation module, used for creating a virtual camera corresponding to the initial point position one by one, wherein the position of the virtual camera is the same as the position of the corresponding initial point position;

[0035] An acquisition module, used for controlling each of the virtual cameras to shoot with a perspective toward the center point of the three-dimensional model as a target perspective, to obtain a template image corresponding to each initial point position;

[0036] A screening module, used for performing image similarity matching between the template images corresponding to the initial points, and screening out a plurality of target images from the template images corresponding to the initial points according to the similarity matching results;

[0037] The acquisition module is also used to obtain a template image of the target object according to the target image.

[0038] The screening module is also specifically used to select at least one initial point to be tested from each of the initial points, and add the at least one initial point to be tested to the sequence of points to be tested. Obtain the adjacent points of each initial point to be tested in the sequence of points to be tested, and add the adjacent points to the sequence of points to be compared. Perform image similarity matching on the current point in the sequence of points to be tested and the sequence of points to be compared in a loop, and cyclically update the sequence of points to be tested and the sequence of points to be compared according to the similarity matching results, and use the template image corresponding to the point to be tested in the sequence of points to be tested at the end of the loop as the target image.

[0039] The screening module is also specifically used for: A. performing image similarity matching on the current point in the sequence of points to be tested and the sequence of points to be compared, and updating the sequence of points to be tested and the sequence of points to be compared according to the image similarity matching result to obtain a new sequence of points to be tested and a new sequence of points to be compared. B. judging whether the sequence of points to be compared is empty, if so, executing step C, and taking the next point in the sequence of points to be tested as the current point, if not, executing step A. C. judging whether the sequence of points to be tested has reached the end of the sequence and the sequence of points to be compared is empty, if so, ending all loops, and taking the template image corresponding to the point to be tested in the current sequence of points to be tested as the target image, if not, executing step A.

[0040] The screening module is also specifically used to perform image similarity matching between the template image corresponding to the first point in the sequence of points to be compared and the template image corresponding to the current point in the sequence of points to be measured, respectively, to obtain a matching result corresponding to the current point in the sequence of points to be measured, wherein the first point is any initial point in the sequence of points to be compared. If the matching result corresponding to the current point in the sequence of points to be measured and the first point is greater than or equal to a first preset threshold, the first point is deleted from the sequence of points to be compared, and the adjacent points of the first point are added to the sequence of points to be compared to obtain a new sequence of points to be compared. If the matching result corresponding to the current point in the sequence of points to be measured and the first point is less than the first preset threshold, the first point is deleted from the sequence of points to be compared, and the first point is added to the sequence of points to be measured to obtain a new sequence of points to be compared.

[0041] The acquisition module is further specifically configured to use the target image as a template image of the target object if the number of the target images is less than or equal to a second preset threshold.

[0042] The acquisition module is further specifically configured to increase the number of preset points if the number of the target images is greater than a second preset threshold, and to re-determine the template image of the target object according to the increased number of preset points.

[0043] The determination module is further specifically used to determine the Fibonacci spherical points according to the three-dimensional model of the target object and the number of preset points by using the Fibonacci grid sampling method, and to use the number of the Fibonacci spherical points as the initial number and the Fibonacci spherical points as the initial points.

[0044] In a third aspect, the present application provides a computer device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the method for establishing a sparse image template of an object as described in any of the aforementioned embodiments.

[0045] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a method for establishing a sparse image template of an object as described in any one of the aforementioned embodiments.

[0046] The beneficial effects of the embodiments of the present application include:

[0047] The method, device and computer equipment for establishing the sparse image template of the object provided by the present application can, firstly, screen out multiple target images as template images of the target object from the template images collected at multiple initial points through the image similarity matching method. In this way, the template images with high similarity collected by virtual cameras of the same or similar viewing angles can be screened out, thereby avoiding a large number of repetitions of similar template images in multiple template images collected for the same object in the template library, and reducing the storage space requirements. In addition, the template images with similar descriptions in multiple template images are eliminated, and the template images with differentiated descriptions are saved as target images, which also maximizes the preservation of the characteristics of the target object and ensures the description accuracy of the target image in the template library for the target object. Secondly, the sparse image template established according to the target image obtained by screening the matching results, compared with the dense sampling method, when the template library is matched with the object to be tested, the number of images to be matched is smaller, so that the computer can obtain the matching results faster, and the matching efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 This is a schematic diagram of the Fibonacci sphere points;

[0050] Figure 2 A schematic diagram of the steps of a method for establishing a sparse image template of an object provided in an embodiment of the present application;

[0051] Figure 3 It is a schematic diagram of the direction of the optical axis of the virtual camera;

[0052] Figure 4 A schematic diagram of another step flow of the method for establishing a sparse image template of an object provided in an embodiment of the present application;

[0053] Figure 5 It is a schematic diagram of adjacent points;

[0054] Figure 6 A schematic diagram of another step flow of the method for establishing a sparse image template of an object provided in an embodiment of the present application;

[0055] Figure 7 A schematic diagram of another step flow of the method for establishing a sparse image template of an object provided in an embodiment of the present application;

[0056] Figure 8 A schematic diagram of another step flow of the method for establishing a sparse image template of an object provided in an embodiment of the present application;

[0057] Fig. 9 A schematic diagram of the structure of a device for establishing a sparse image template of an object provided in an embodiment of the present application;

[0058] Fig.10 A schematic diagram of the structure of a computer device provided in an embodiment of the present application.

[0059] Icons: 10 - device for establishing a sparse image template of an object; 1001 - determination module; 1002 - creation module; 1003 - acquisition module; 1004 - screening module; 2001 - processor; 2002 - memory. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0061] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0062] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0063] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0064] It should be noted that, in the absence of conflict, the features in the embodiments of the present application may be combined with each other.

[0065] Image matching is the process of sampling the target object model with multiple virtual cameras of different perspectives in CAD software to obtain multiple template images, and then matching them with two or more images of the object to be tested obtained during the actual operation. Specifically, the object can be a part, which is used in modern manufacturing to identify, grasp and assemble parts.

[0066] At present, in the process of sampling by the virtual camera of CAD software, in order to minimize the error caused by discrete sampling, a dense template is generally established by dense sampling. The virtual camera is set around the target part in a high-density manner to sample and obtain the template image. Usually, for a target part, the template images obtained by sampling can reach tens of thousands, which not only takes up a lot of storage space, but also leads to a decrease in matching efficiency due to the excessive number of template images.

[0067] Based on this, the applicant has proposed a method, device and computer equipment for establishing a sparse image template of an object. The method can screen out and eliminate template images with high similarity collected by virtual cameras with the same or similar viewing angles from template images collected at multiple initial points, and save template images with differentiated descriptions as target images. While ensuring the matching accuracy of template images, the number of template images of the target object in the template library is reduced, thereby speeding up the image matching speed of the object.

[0068] The following is an explanation of a method, apparatus, and computer device for establishing a sparse image template of an object provided in an embodiment of the present application in combination with a plurality of specific application examples.

[0069] For the packing problems on the sphere, that is, the problem of evenly arranging multiple points on the sphere, it is difficult to obtain a solution because the surface of the sphere has a curvature and the intersections of longitude and latitude obtained at a certain interval of longitude and latitude are not evenly distributed on the sphere. In order to solve this problem, the Fibonacci grid sampling method was proposed. This solution uses a formula with a golden ratio of 0.618 to generate three-dimensional spatial coordinates corresponding to the number of sampling points from the longitude and latitude on the sphere.

[0070] like Figure 1 As shown, the dot matrix formed by the above three-dimensional space coordinates is called a Fibonacci grid (Fibonacci grid or Fibonacci grid) or a Fibonacci spherical point. From a macroscopic point of view, the Fibonacci spherical point conforms to the characteristic that all points are evenly arranged on the spherical surface. It is understandable that after adjusting the number of sampling points, the Fibonacci spherical points with different density and equal spacing can be re-arranged according to the formula.

[0071] In this application, taking the object as a part as an example, the center of the three-dimensional model of the target part in the CAD software can be used as the center of the sphere. According to the complexity of the three-dimensional model of the target part, a Fibonacci spherical point is constructed, and a virtual camera is set at each point to collect the template image of the three-dimensional model of the target part. Then, the template images collected at each point are matched for image similarity, and the template images with greater similarity collected by the virtual cameras of adjacent points are screened and eliminated. In this way, more template images can be collected in the parts of the three-dimensional model of the target part that are more varied and more complex, and fewer template images can be collected in the parts of the three-dimensional model of the target part that are less varied. Thus, while ensuring the accuracy of template image matching, the number of template images of the target part in the template library is reduced, and the storage space occupied by the template library is reduced.

[0072] In the embodiment of the present application, the object may refer to an object that can be described by a three-dimensional model or a drawing, including but not limited to parts, sculpture models, etc.

[0073] Figure 2 The flowchart of the method for establishing a sparse image template of an object provided in an embodiment of the present application is shown. The execution subject of the method may be a computer device with computing processing capabilities. Figure 2 As shown, the method includes:

[0074] S101, determining an initial number of virtual cameras and a plurality of initial points according to a three-dimensional model of a target object and a preset number of points.

[0075] Optionally, before the target object is identified, a template library can be established according to the target object. Specifically, a three-dimensional model can be constructed in proportion in the CAD software according to the size of the target object, and the specific size of the three-dimensional model in the CAD can be constructed as needed.

[0076] According to the complexity of different target objects, a number is selected from a preset range as the number of preset points, and according to the number of preset points, a plurality of initial points can be evenly set around the three-dimensional model of the target object. The preset range can be [100, 1000]. In order to more comprehensively collect the template image of the target object, the spatial distribution of the plurality of initial points can form a virtual sphere surrounding the three-dimensional model of the target object. It can be understood that the higher the complexity of the target object, the greater the number of preset points.

[0077] The number of preset points is used as the initial number of virtual cameras in the CAD software, and the position of each preset point is used as the initial point of each virtual camera, so the initial number and initial point of multiple virtual cameras are determined.

[0078] S102, creating a virtual camera corresponding to the initial point one by one, wherein the position of the virtual camera is the same as the position of the corresponding initial point.

[0079] A virtual camera is created at the above initial point. Optionally, a virtual camera may be created at each initial point to collect a template image of the three-dimensional model of the target object.

[0080] S103, controlling each virtual camera to shoot with a viewing angle toward the center point of the three-dimensional model as a target viewing angle, and obtaining a template image corresponding to each initial point position.

[0081] like Figure 3 As shown, point A is a point on the spherical surface of a virtual sphere surrounding the three-dimensional model of the target object. A virtual camera is set at point A, and the viewing angle of the virtual camera, that is, the optical axis of the virtual camera, is oriented toward the line connecting the point and the virtual sphere center. The virtual sphere center may be the center of the three-dimensional model of the target object.

[0082] In addition, the radius of the virtual sphere, that is, Figure 3 In the figure, the distance from point A to the center point O of the virtual sphere is determined as needed according to the complexity of the object. For example, it can be twice the maximum length of the target object.

[0083] After the position and viewing angle of each virtual camera are determined, the three-dimensional model of the target object at its center can be photographed to acquire multiple template images corresponding to each initial point.

[0084] S104, performing image similarity matching on the template images corresponding to the initial points, and selecting a plurality of target images from the template images corresponding to the initial points according to the similarity matching results.

[0085] The above-mentioned multiple template images capture a comprehensive three-dimensional surround image of the target object. However, since the complexity of different parts of the three-dimensional model may be different, the similarities between the template images captured by the virtual cameras at each initial position and the template images captured by adjacent virtual cameras may also be different.

[0086] In order to describe the parts of the 3D model with more complex changes as much as possible and reduce the description of the parts of the 3D model with slowly changing changes, an image similarity matching method can be used to match the template images captured by each virtual camera with each other. Based on the similarity matching results, template images whose matching degree with other template images exceeds a preset threshold are eliminated, and finally the retained multiple template images are used as target images.

[0087] The image similarity matching method is to extract features from two different template images respectively to obtain feature descriptions of points and lines in the template images. Then, the extracted feature information is used for image similarity matching to determine whether the matching degree between the features corresponding to the two template images exceeds a preset threshold, which can be 95% by way of example.

[0088] S105, obtaining a template image of the target object according to the target image.

[0089] If the target image obtained after image similarity matching does not meet the preset standard, for example, the number of template images is too small to fully describe the three-dimensional model of the target object, the target image is re-determined.

[0090] In this embodiment, through the image similarity matching method, the template images with high similarity collected by the virtual camera with the same or similar viewing angle are eliminated, and only the template images with differentiated descriptions are saved as the target images, which ensures the description accuracy of the target images in the template library for the target object while reducing the storage space requirements. In addition, the number of target images to be matched in the template library is greatly reduced, which also improves the matching speed.

[0091] Alternatively, if Figure 4 As shown, in the above step S104, image similarity matching is performed between the template images corresponding to the initial points, and multiple target images are screened out from the template images corresponding to the initial points according to the similarity matching results, which can be achieved by the following steps S201 to S203.

[0092] S201, selecting at least one initial point to be measured from each initial point, and adding the at least one initial point to be measured to a sequence of points to be measured.

[0093] First, an empty sequence is set as the initial point sequence to be tested. Then, one or more initial points among the multiple initial points determined in the above step S101 are added as initial points to be tested to the initial point sequence to obtain a point sequence to be tested.

[0094] It can be understood that before setting certain initial points as initial points to be tested, the template images captured by the virtual cameras at the above-mentioned initial points can be matched with each other to determine whether the similarity matching results between the template images corresponding to each initial point are less than a preset threshold, or whether the distance between each initial point is greater than a preset threshold, thereby ensuring that there are large differences between the points in the initial point sequence to be tested.

[0095] S202, obtaining adjacent points of each initial point to be measured in the sequence of points to be measured, and adding the adjacent points to the sequence of points to be compared.

[0096] Next, an empty sequence may be set as an initial point to be compared, and then adjacent points of multiple initial points to be tested in the above-mentioned point sequence to be tested are added as points to be compared to the initial points to be compared, so as to obtain a point sequence to be compared.

[0097] The adjacent points of a certain initial point to be measured refer to points whose spatial distance from the initial point to be measured is less than a preset distance threshold, and the preset distance threshold is determined according to the number of initial points. Figure 5 The figure shows the distribution of each initial point when the number of preset points is 100. Among them, if the point represented by the circle is the initial point, its adjacent points are the points represented by the four bold stars around it. Under the condition of the current preset number of points, the spatial distance between the initial point and the adjacent points is evenly maintained between 0.3 and 0.4. Therefore, the preset distance threshold can be set to 0.5 at this time, and the points with a spatial distance less than 0.5 from the initial point to be tested are all regarded as adjacent points of the initial point to be tested and added to the sequence of points to be compared.

[0098] Optionally, the above preset distance threshold can be obtained by formula Get, where n represents the number of initial points.

[0099] It is understandable that if there are multiple initial points to be measured, the adjacent points corresponding to the multiple initial points to be measured are added to the initial sequence of points to be compared to obtain the sequence of points to be compared.

[0100] S203, cyclically performing image similarity matching on the current point in the test point sequence and the point sequence to be compared, and cyclically updating the test point sequence and the point sequence to be compared according to the similarity matching result, and taking the template image corresponding to the test point in the test point sequence at the end of the cycle as the target image.

[0101] The multiple points to be tested in the above-mentioned sequence of points to be tested can be understood as the points whose corresponding template images have been determined to be different and need to be retained. The points to be compared can be understood as the points whose similarity with the template images corresponding to the points to be tested has not yet been determined. Therefore, the multiple points in the points to be compared can be matched with the current point to be tested in the sequence of points to be tested for image similarity, and the sequence of points to be tested and the sequence of points to be compared can be cyclically updated according to each similarity matching result.

[0102] Among them, the above-mentioned current point can be understood as a point in the sequence of points to be compared that has a corresponding relationship with the point to be tested. That is, if the initial sequence of points to be compared contains only one initial point, and the initial sequence of points to be tested contains multiple points adjacent to the initial point, first start the large loop, and according to the order of breadth-first search, start the small loop, match the initial point with a point in the adjacent points, and then match the adjacent points of the adjacent point in turn according to the comparison result, until no unmatched points are retrieved, and the small loop ends. Among them, the adjacent point and the multi-level adjacent points of the adjacent point are all the current point. Then, match the next adjacent point of the initial point with the initial point, start the small loop again, and update the current point until the adjacent points of the initial point are matched and retrieved, and the large loop ends.

[0103] After updating the sequence of points to be tested and the sequence of points to be compared for multiple cycles, until the sequence of points to be compared is empty and the sequence of points to be tested reaches the end, the comparison is terminated, and the template image corresponding to the point to be tested in the sequence of points to be tested is used as the target image and stored in the template library.

[0104] In this embodiment, the adjacent points of the point to be measured are added to the points to be compared and updated cyclically, so that the points to be compared are all points that are close in spatial distance to the point to be measured and may be similar, avoiding too many points in the points to be compared and reducing the calculation speed.

[0105] Alternatively, if Figure 6 As shown, in the above step S203, the image similarity matching is cyclically performed on the sequence of points to be tested and the sequence of points to be compared, and the sequence of points to be tested and the sequence of points to be compared are cyclically updated according to the similarity matching results, and the template image corresponding to the point to be tested in the sequence of points to be tested at the end of the cycle is used as the target image, including the following cyclic steps:

[0106] A. Perform image similarity matching on the current point in the point sequence to be measured and the point sequence to be compared, and update the point sequence to be measured and the point sequence to be compared according to the image similarity matching result to obtain a new point sequence to be measured and a new point sequence to be compared.

[0107] The first point to be compared in the sequence of points to be compared is matched with the current point to be tested in the sequence of points to be tested in sequence for image similarity, and the sequence of points to be tested and the sequence of points to be compared corresponding to this round of comparison are updated according to the similarity matching results. It should be noted that the update part of the above sequence of points to be tested is at the end of the sequence.

[0108] Next, after executing the following step B, the second point to be compared in the updated sequence of points to be compared is matched with the template image corresponding to the current point to be tested in the updated sequence of points to be tested in turn for image similarity, so that the sequence of points to be tested and the sequence of points to be compared are updated again according to the matching results.

[0109] In this way, after each execution of step B, if the loop is not ended, the next point to be compared in the point sequence to be compared updated after the last execution of step A is completed, and the point sequence to be tested updated after the last execution of step A is compared with the point sequence to be compared, and then the point sequence to be tested and the point sequence to be compared are updated to obtain a new point sequence to be tested and a new point sequence to be compared. The updating method of the point sequence to be tested and the point sequence to be compared will be described in detail in the following embodiments.

[0110] B. Determine whether the sequence of points to be compared is empty. If so, execute step C and take the next point in the points to be detected as the current point. If not, execute step A.

[0111] At this time, a small loop is executed. If the sequence of points to be compared is empty, the small loop ends and step C is continued. Otherwise, step A is continued to match the points in the sequence of points to be compared with the current point in the sequence to be tested.

[0112] C. Determine whether the points to be detected have reached the end of the sequence and the sequence of points to be compared is empty. If so, end all loops and use the template image corresponding to the points to be detected in the current sequence of points to be detected as the target image. If not, execute step A.

[0113] Step B is the termination condition of the small loop. After each execution of step A, it is determined whether the sequence to be compared is empty. If so, the termination condition of the small loop is met, the small loop ends, and step C is executed. Step C is the termination condition of the large loop. After executing step B, it is determined whether the sequence to be detected has reached the end. If so, the termination condition of the large loop is met, the large loop ends, and the multiple template images captured by the virtual camera at the test point in the final test point sequence are used as the target image. Otherwise, continue to execute step A.

[0114] In this embodiment, after determining the sequence of points to be tested and the sequence of points to be compared, image similarity matching is performed in multiple cycles, and the template images corresponding to the points with the largest difference between the images, that is, the points to be tested that are finally obtained, are used as target images. Through this cyclic comparison method, all the sequences of points to be tested and the sequences of points to be compared can be compared with each other as much as possible, so that the target images are all template images whose similarity is lower than a preset threshold.

[0115] Alternatively, if Figure 7 As shown, in the above step A, image similarity matching is performed on the current point in the point sequence to be tested and the point sequence to be compared, and the point sequence to be tested and the point sequence to be compared are updated according to the image similarity matching result to obtain a new point sequence to be tested and a new point sequence to be compared, which can be implemented by the following steps S301 to S303.

[0116] S301, performing image similarity matching on the template image corresponding to the first point in the point sequence to be compared and the template image corresponding to the current point in the point sequence to be measured, to obtain matching results corresponding to each point in the point sequence to be measured, wherein the first point is any initial point in the point sequence to be compared.

[0117] After determining the sequence of points to be measured and the sequence of points to be compared, the template image captured by the virtual camera at the first point in the sequence of points to be compared can be matched for image similarity with the template image captured by the virtual camera at the current point to be measured in the sequence of points to be measured.

[0118] S302, if the matching result between the current point in the point sequence to be measured and the first point is greater than or equal to the first preset threshold, the first point is deleted from the point sequence to be compared, and the adjacent points of the first point are added to the point sequence to be compared to obtain a new point sequence to be compared.

[0119] For the above round of image similarity matching, the matching result of the current point and the first point in the point sequence to be measured is compared with the first preset threshold. If the matching result is greater than or equal to the first preset threshold, it is considered that the template image corresponding to the current point and the template image of the first point are too similar, then the first point is deleted from the points to be compared, and the comparison range of the point sequence to be compared is further expanded, the adjacent points of the first point are calculated, added to the point sequence to be compared, and the point sequence to be compared is updated to obtain a new point sequence to be compared.

[0120] The first preset threshold may be set according to actual accuracy requirements, and illustratively, may be 95%.

[0121] S303: If the matching result between the current point in the point sequence to be measured and the first point is less than a first preset threshold, the first point is deleted from the point sequence to be compared, and the first point is added to the point sequence to be measured to obtain a new point sequence to be measured.

[0122] For the above round of image similarity matching, the matching result between the current point and the first point in the sequence of points to be tested is compared with the first preset threshold. If the matching result is less than the first preset threshold, it is considered that the template image corresponding to the first point is not similar to the template image corresponding to any of the points to be tested in the sequence of points to be tested, and it can be added to the sequence of points to be tested.

[0123] In this embodiment, by continuously updating the sequence of points to be tested and the sequence of points to be compared, the length of the sequence of points to be compared is reduced, and the length of the sequence of points to be tested is increased. This method of updating the sequence of points to be tested and the sequence of points to be compared can determine the template image corresponding to the sequence of points to be tested as the target image through the least number of comparisons, thereby improving the calculation efficiency.

[0124] Optionally, in the above step S105, obtaining a template image of the target object according to the target image includes: if the number of target images is less than or equal to a second preset threshold, using the target image as the template image of the target object.

[0125] The second preset threshold may be the minimum number of target images to be retained, which is determined according to the number of preset points, for example, may be two-thirds of the number of preset points.

[0126] It can be understood that if the number of target images is less than or equal to the second preset threshold, it can be considered that according to the current number of preset points, there are a large number of repeated template images in the template image taken by the virtual camera set around the three-dimensional model of the target object. In other words, the currently obtained target image is already able to describe the complete three-dimensional model of the target object, but redundant template images appear. Therefore, the obtained target image can be used as the template image of the target object and stored in the template library.

[0127] Optionally, in the above step S105, obtaining a template image of the target object according to the target image includes: if the number of target images is greater than a second preset threshold, increasing the number of preset points, and re-determining the template image of the target object according to the increased number of preset points.

[0128] Similarly, if the number of target images is greater than the second preset threshold, it is considered that the target images can effectively describe the three-dimensional model of the target object. Therefore, it is necessary to further increase the number of preset points based on the current number of preset points, and re-execute the steps of determining the target image in steps S101 to S104 of the above embodiment according to the increased number of preset points to redefine the target image.

[0129] After the re-determined target image is obtained, the relationship between the number of target images and the second preset threshold is determined again to determine whether to retain the current target image as the template image of the target object. The second preset threshold is re-determined according to the increased number of preset points.

[0130] In this embodiment, by judging the relationship between the number of target images and the second preset threshold, it is determined whether to use the current target image as a template image. This method of determining the template image can avoid the influence of the preset number of points being too large or too small on the accuracy of the template image, and adaptively adjust the number of template images to the best according to the complexity of the target object.

[0131] Alternatively, if Figure 8 As shown, in the above step S101, the initial number of virtual cameras and a plurality of initial points are determined according to the three-dimensional model of the target object and the number of preset points, which can be implemented by the following steps S401 to S402.

[0132] S401, determining Fibonacci spherical points by a Fibonacci grid sampling method according to the three-dimensional model of the target object and the number of preset points.

[0133] The center of the three-dimensional model of the target object is used as the virtual sphere center, and a virtual sphere is constructed at twice the maximum length of the three-dimensional model of the target object. N points are evenly set on the spherical surface of the virtual sphere as the preset number of points. By performing the Fibonacci grid sampling method on the virtual sphere, the calculation method for determining the coordinates of the nth Fibonacci spherical point is shown in the following formula:

[0134] z n =(2n-1) / N-1

[0135]

[0136]

[0137] in, That is the golden ratio.

[0138] S402, taking the number of Fibonacci spherical points as an initial number and taking the Fibonacci spherical points as initial points.

[0139] According to the above formula, the coordinates of N points on the spherical surface of the virtual sphere are solved respectively to obtain the coordinates of N Fibonacci spherical points.

[0140] Next, a virtual camera may be set at each Fibonacci sphere point according to the obtained number and coordinates of the Fibonacci sphere points, and the number of virtual cameras may be recorded as the initial number, and the positions of the virtual cameras may be recorded as the initial points.

[0141] In this embodiment, the Fibonacci sphere points are determined by the Fibonacci grid sampling method, which enables the virtual camera to be evenly and completely distributed around the three-dimensional model of the target object, ensuring that the virtual camera samples the three-dimensional model of the target object completely and evenly.

[0142] The present application also provides a device 10 for establishing a sparse image template of an object. Fig. 9 As shown, the device comprises:

[0143] A determination module 1001 is used to determine the initial number of virtual cameras and a plurality of initial points according to the three-dimensional model of the target object and the number of preset points;

[0144] A creation module 1002 is used to create a virtual camera corresponding to the initial point position one by one, and the position of the virtual camera is the same as the position of the corresponding initial point position;

[0145] An acquisition module 1003 is used to control each virtual camera to shoot with a perspective toward the center point of the three-dimensional model as a target perspective, and obtain a template image corresponding to each initial point position;

[0146] The screening module 1004 is used to perform image similarity matching between the template images corresponding to the initial points, and screen out a plurality of target images from the template images corresponding to the initial points according to the similarity matching results;

[0147] The acquisition module 1003 is further used to obtain a template image of the target object according to the target image.

[0148] The screening module 1004 is also specifically used to select at least one initial point to be tested from each initial point, and add the at least one initial point to be tested to the sequence of points to be tested. Obtain the adjacent points of each initial point to be tested in the sequence of points to be tested, and add the adjacent points to the sequence of points to be compared. Perform image similarity matching for the current point in the sequence of points to be tested and the sequence of points to be compared in a loop, and update the sequence of points to be tested and the sequence of points to be compared in a loop according to the similarity matching results, and use the template image corresponding to the point to be tested in the sequence of points to be tested at the end of the loop as the target image.

[0149] The screening module 1004 is also specifically used for: A. performing image similarity matching on the current point in the sequence of points to be tested and the sequence of points to be compared, and updating the sequence of points to be tested and the sequence of points to be compared according to the image similarity matching result to obtain a new sequence of points to be tested and a new sequence of points to be compared. B. judging whether the sequence of points to be compared is empty, if so, executing step C, and taking the next point in the sequence of points to be tested as the current point, if not, executing step A. C. judging whether the sequence of points to be tested has reached the end of the sequence and the sequence of points to be compared is empty, if so, ending all loops, and taking the template image corresponding to the point to be tested in the current sequence of points to be tested as the target image, if not, executing step A.

[0150] The screening module 1004 is also specifically used to perform image similarity matching between the template image corresponding to the first point in the sequence of points to be compared and the template image corresponding to the current point in the sequence of points to be measured, respectively, to obtain a matching result corresponding to the current point in the sequence of points to be measured, wherein the first point is any initial point in the sequence of points to be compared. If the matching result corresponding to the current point in the sequence of points to be measured and the first point is greater than or equal to a first preset threshold, the first point is deleted from the sequence of points to be compared, and the adjacent points of the first point are added to the sequence of points to be compared to obtain a new sequence of points to be compared. If the matching result corresponding to the current point in the sequence of points to be measured and the first point is less than a first preset threshold, the first point is deleted from the sequence of points to be compared, and the first point is added to the sequence of points to be measured to obtain a new sequence of points to be measured.

[0151] The acquisition module 1003 is further specifically configured to use the target image as a template image of the target object if the number of the target images is less than or equal to a second preset threshold.

[0152] The acquisition module 1003 is further specifically configured to increase the number of preset points if the number of target images is greater than a second preset threshold, and to re-determine the template image of the target object according to the increased number of preset points.

[0153] The determination module 1001 is further specifically used to determine the Fibonacci spherical points according to the three-dimensional model of the target object and the number of preset points by using the Fibonacci grid sampling method, and to use the number of Fibonacci spherical points as the initial number and the Fibonacci spherical points as the initial points.

[0154] See also Fig.10This embodiment also provides a computer device, which includes: a processor 2001, a memory 2002 and a bus, the memory 2002 stores machine-readable instructions executable by the processor 2001, and when the computer device is running, the above machine-readable instructions are executed. The processor 2001 communicates with the memory 2002 through the bus, and the processor 2001 is used to execute the steps of the method for establishing a sparse image template of an object in the above embodiment.

[0155] The memory 2002, the processor 2001 and the bus components are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The data processing device of the system for establishing the sparse image template of the object includes at least one software function module that can be stored in the memory 2002 in the form of software or firmware or solidified in the operating system (OS) of the computer device. The processor 2001 is used to execute the executable modules stored in the memory 2002, such as the software function modules and computer programs included in the data processing device of the system for establishing the sparse image template of the object.

[0156] Among them, the memory 2002 can be, but is not limited to, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0157] Optionally, the present application further provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method embodiment are executed. The specific implementation method and technical effect are similar and will not be repeated here.

[0158] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0159] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or part of the contribution to the prior art or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0160] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for establishing a sparse image template of an object, characterized in that: The method comprises: Determine the initial number of virtual cameras and multiple initial points according to the three-dimensional model of the target object and the number of preset points; Creating a virtual camera corresponding to the initial point position one by one, wherein the position of the virtual camera is the same as the position of the corresponding initial point position; Controlling each of the virtual cameras to shoot with a viewing angle toward the center point of the three-dimensional model as a target viewing angle to obtain a template image corresponding to each initial point position; Performing image similarity matching on the template images corresponding to the initial points, and selecting a plurality of target images from the template images corresponding to the initial points according to the similarity matching results; Obtaining a template image of the target object according to the target image; The similarity matching is performed on the template images corresponding to the initial points, and a plurality of target images are screened out from the template images corresponding to the initial points according to the similarity matching results, including: Selecting at least one initial point to be measured from each of the initial points, and adding the at least one initial point to be measured to a sequence of points to be measured; Acquire adjacent points of each initial point to be measured in the sequence of points to be measured, and add the adjacent points to the sequence of points to be compared; Image similarity matching is cyclically performed on the current point in the sequence of points to be tested and the sequence of points to be compared, and the sequence of points to be tested and the sequence of points to be compared are cyclically updated according to the similarity matching results, and the template image corresponding to the point to be tested in the sequence of points to be tested at the end of the cycle is used as the target image.

2. The method for establishing a sparse image template of an object according to claim 1, characterized in that: The method of cyclically performing image similarity matching on the current point in the sequence of points to be tested and the sequence of points to be compared, and cyclically updating the sequence of points to be tested and the sequence of points to be compared according to the similarity matching result, and taking the template image corresponding to the point to be tested in the sequence of points to be tested at the end of the cycle as the target image, comprises: A. performing image similarity matching on the current point in the sequence of points to be measured and the sequence of points to be compared, and updating the sequence of points to be measured and the sequence of points to be compared according to the image similarity matching result to obtain a new sequence of points to be measured and a new sequence of points to be compared; B. Determine whether the sequence of points to be compared is empty. If so, execute step C and use the next point in the sequence of points to be measured as the current point. If not, execute step A; C. Determine whether the test point has reached the end of the sequence and the sequence of points to be compared is empty. If so, end all loops and use the template image corresponding to the test point in the current sequence of test points as the target image. If not, execute step A.

3. The method for establishing a sparse image template of an object according to claim 2, characterized in that: The method of performing image similarity matching on the current point in the sequence of points to be measured and the sequence of points to be compared, and updating the sequence of points to be measured and the sequence of points to be compared according to the image similarity matching result to obtain a new sequence of points to be measured and a new sequence of points to be compared includes: Performing image similarity matching on the template image corresponding to the first point in the sequence of points to be compared and the template image corresponding to the current point in the sequence of points to be measured, respectively, to obtain a matching result corresponding to the current point in the sequence of points to be measured, wherein the first point is any initial point in the sequence of points to be compared; If the matching result between the current point in the sequence of points to be measured and the first point is greater than or equal to a first preset threshold, the first point is deleted from the sequence of points to be compared, and the adjacent points of the first point are added to the sequence of points to be compared to obtain a new sequence of points to be compared; If the matching result between the current point in the point sequence to be measured and the first point is less than the first preset threshold, the first point is deleted from the point sequence to be compared, and the first point is added to the point sequence to be measured to obtain a new point sequence to be measured.

4. The method for establishing a sparse image template of an object according to any one of claims 1 to 3, characterized in that: The step of obtaining a template image of the target object according to the target image comprises: If the number of the target images is less than or equal to a second preset threshold, the target images are used as template images of the target objects.

5. The method for establishing a sparse image template of an object according to any one of claims 1 to 3, characterized in that: The step of obtaining a template image of the target object according to the target image comprises: If the number of the target images is greater than a second preset threshold, the number of preset points is increased, and the template image of the target object is re-determined according to the increased number of preset points.

6. The method for establishing a sparse image template of an object according to any one of claims 1 to 3, characterized in that: The step of determining the initial number of virtual cameras and a plurality of initial points according to the three-dimensional model of the target object and the number of preset points includes: According to the three-dimensional model of the target object and the number of preset points, the Fibonacci spherical points are determined by a Fibonacci grid sampling method; The number of the Fibonacci spherical points is used as the initial number, and the Fibonacci spherical points are used as the initial points.

7. A device for establishing a sparse image template of an object, characterized in that: The device comprises: A determination module, used to determine the initial number of virtual cameras and a plurality of initial points according to the three-dimensional model of the target object and the number of preset points; A creation module, used for creating a virtual camera corresponding to the initial point position one by one, wherein the position of the virtual camera is the same as the position of the corresponding initial point position; An acquisition module, used for controlling each of the virtual cameras to shoot with a perspective toward the center point of the three-dimensional model as a target perspective, to obtain a template image corresponding to each initial point position; A screening module, used for performing image similarity matching between the template images corresponding to the initial points, and screening out a plurality of target images from the template images corresponding to the initial points according to the similarity matching results; The acquisition module is also used to obtain a template image of the target object according to the target image; The screening module is specifically used to select at least one initial point to be tested from each of the initial points, and add the at least one initial point to be tested to a sequence of points to be tested; obtain adjacent points of each initial point to be tested in the sequence of points to be tested, and add the adjacent points to a sequence of points to be compared; cyclically perform image similarity matching on the current point in the sequence of points to be tested and the sequence of points to be compared, and cyclically update the sequence of points to be tested and the sequence of points to be compared according to the similarity matching result, and use the template image corresponding to the point to be tested in the sequence of points to be tested at the end of the cycle as the target image.

8. A computer device, characterized in that: The computer device includes: a processor, a storage medium and a bus, the storage medium stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the method for establishing a sparse image template of an object as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for establishing a sparse image template of an object as described in any one of claims 1 to 6 are implemented.

Images