An irregular image processing system

Abstract

The present invention provides an irregular image processing system, comprising an acquisition module, a conversion module, a processing module and a result generation module. A filter and a dual channel are arranged in the acquisition module. The filter removes regular images, retains irregular images, and classifies the irregular images by dimension, so that images of two different dimensions enter different channels for image processing respectively. The present invention adds a filter to the image acquisition module, thereby screening out irregular images and classifying the images by dimension. The technology sets up a dual channel, which can process two-dimensional images and three-dimensional images at the same time without interfering with each other, and is convenient for operators to use.

Description

Technical Field

[0001] The invention belongs to the field of industrial visual recognition, and in particular relates to an irregular image processing system. Background Art

[0002] In the industrial sector, industrial vision robots are often needed to test, measure, and detect machines, parts, or images. During the automatic detection of target objects, the equipment obtains the target image, but the two-dimensional image and the three-dimensional image often need to be detected separately, which is inconvenient to use. Summary of the invention

[0003] In view of the above-mentioned defects in the prior art, an object of the present invention is to provide an irregular image processing system.

[0004] The above technical objectives of the present invention are achieved through the following solutions:

[0005] An irregular image processing system, comprising:

[0006] An acquisition module acquires data from different angles of the same target image to form image target data;

[0007] The acquisition module further includes a screener, a first processing channel, and a second processing channel. The screener is used to acquire preset screening data and classification data, and remove target data that meets the screening data. The remaining target data is divided into a first-class target and a second-class target by the screener. The first-class target enters the first processing channel, and the second-class target enters the second processing channel.

[0008] The conversion module includes a first converter and a second converter, which are respectively arranged in the first processing channel and the second processing channel and connected to the output end of the image acquisition module. The first type of target is converted into first initial data through the first converter, and the second type of target is converted into second initial data through the second converter.

[0009] The processing module includes a first processor and a second processor, which are respectively arranged in the first processing channel and the second processing channel, and receive the initial data from the conversion module, the first initial data is processed by the first processor to generate a first processing result, and the second initial data is processed by the second processor to generate a second processing result;

[0010] The result generation module includes a first generator and a second generator, which are respectively arranged in the first processing channel and the second processing channel, and receive the processing data from the processing module to reconstruct and optimize the model.

[0011] Further, the sifter is configured to perform the following screening steps:

[0012] Obtain preset screening data, project the data onto a three-dimensional grid area, and mark preset corner points and reference edges;

[0013] Mapping target corner points and target edges of target image data into a three-dimensional grid area;

[0014] The target image is a regular image if the fit between the target corner point and the marked preset corner point is within a first threshold, and the fit between the target edge and the reference edge is within a second threshold, and the rest are irregular images;

[0015] The first threshold is a preset corner point difference, and the second threshold is a preset edge difference.

[0016] Furthermore, the sifter is further configured to perform the following classification steps:

[0017] Randomly obtain three points in the target image, and connect the three points in sequence to form a closed plane as the reference plane;

[0018] Randomly obtain the fourth point of the target image;

[0019] The edges of the reference plane extend to span the entire three-dimensional grid area, and the reference plane covers the fourth point position, then the target image is the second type of target, and the rest are the first type of targets.

[0020] Further, the conversion module is configured to match the target corner points and target edges in the first-class target and the second-class target with the grid points of the three-dimensional grid area to obtain three-dimensional grid coordinates to form the first initial data and the second initial data respectively.

[0021] Furthermore, the processing module is configured to remove outlier points in the initial data whose fit exceeds a third threshold, calculate the normal vector of each point in the remaining data, compare its consistency with the normal vectors of other points in the neighborhood, mark points inconsistent with the neighborhood as noise points, and filter them out, and the third threshold is the outlier threshold.

[0022] Furthermore, the processing module is also configured to set a reference point, iterate the initial data to match the reference point, and remove points in the initial data whose fit exceeds a fourth threshold; the fourth threshold is an error threshold.

[0023] Further, the result generation module,

[0024] The first generator is configured to map the first processing result into a quadrilateral mesh, generate three-dimensional mesh data, and reconstruct a type of target surface model;

[0025] At the same time, the second generator is configured to project the second processing result to a plane grid at multiple angles, generate plane grid data, and reconstruct the second type of target model.

[0026] Furthermore, the first processing channel and the second processing channel are independent of each other.

[0027] Compared with the prior art, the present invention has the following advantages:

[0028] (1) A filtering function is added to the image acquisition module of this technology, which can filter out irregular images and classify images by dimension.

[0029] (2) This technology has dual channels that can process two-dimensional images and three-dimensional images simultaneously without interfering with each other, making it convenient for operators to use. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the system operation flow of the present invention. DETAILED DESCRIPTION

[0031] The embodiments of the present invention are described in detail below. The following embodiments are implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operation processes are given, but the protection scope of the present invention is not limited to the following embodiments.

[0032] Embodiment 1

[0033] An irregular image processing system, as shown in the figure, includes an acquisition module, a conversion module, a processing module, and a result generation module.

[0034] The acquisition module in this embodiment is connected to a visual camera and an image input client, respectively. An operator can use the visual camera to acquire images, or transmit images to the processing system on the client.

[0035] The acquisition module in this embodiment acquires data from different angles of the same target image to form target data of the image.

[0036] The acquisition module in this embodiment also includes a sifter, a first processing channel and a second processing channel. A screening function is added to the image acquisition module to screen out irregular images and classify images by dimension.

[0037] The first processing channel and the second processing channel in this embodiment are independent of each other.

[0038] The sifter in this embodiment is configured to perform the following screening steps:

[0039] Obtain preset filtered data D, project the data to the three-dimensional grid area G, and use the projection function Proj(D,G) to represent the process of projecting the preset filtered data to the three-dimensional grid area, and mark the preset corner point MarkCorner(D) and the reference edge MarkEdge(D);

[0040] Map the target corners MapCorners(T,G) and the target edges MapEdges(T,G) of the target image data T into the three-dimensional grid area;

[0041] The target image is a regular image if the fit between the target corner point and the marked preset corner point is within a first threshold, and the fit between the target edge and the reference edge is within a second threshold, and the rest are irregular images;

[0042] The first threshold is a preset corner point difference value th1, and the second threshold is a preset edge difference value th2;

[0043] The fit between the target corner point and the marked preset corner point can be expressed using the function FitCorners(Corners1, Corners2, th1), where Corners1 and Corners2 represent two sets of corner point coordinates, and th is the threshold;

[0044] The fit between the target edge and the reference edge can be expressed using the function FitEdges(Edges1, Edges2, th2), where Edges1 and Edges2 represent two sets of edge coordinates, and th is the threshold;

[0045] Regular image: FitCorners(MapCorners(T,G),MarkCorner(D),th1)AND FitEdges(MapEdges(T,G),MarkEdge(D),th2), irregular image: ELSE.

[0046] The sifter of this embodiment is configured to perform the following classification steps:

[0047] Randomly obtain three points P1, P2, and P3 in the target image T. The process of randomly selecting three points can be expressed as: P1, P2, P3 = RandomPoints(T), where RandomPoints(T) is a function of randomly selecting three points from the target image T; and connect the three points in sequence to form a closed plane as a reference plane B = CreatePlane(P1, P2, P3);

[0048] The fourth point P4 of the target image T is randomly obtained. The process of randomly selecting the fourth point can be expressed as: P4 = RandomPoint(T);

[0049] The edges of the reference plane B are extended to span the entire three-dimensional grid area G, and this process can be expressed as ExtendEdges(B, G); if the reference plane B covers the fourth point P4, the target image is the second type of target, and the rest are the first type of targets.

[0050] After the filter of this embodiment screens and classifies the target data, the first type of targets enters the first processing channel, and the second type of targets enters the second processing channel.

[0051] The conversion module in this embodiment includes a first converter and a second converter, which are respectively arranged in the first processing channel and the second processing channel and connected to the output end of the image acquisition module. The first type of target is converted into first initial data through the first converter, and the second type of target is converted into second initial data through the second converter.

[0052] The conversion module in this embodiment is configured to match the target corner point C1 and target edge E1 of the first type of target, the target corner point set C2 and target edge E2 of the second type of target with the grid points of the three-dimensional grid area G;

[0053] The matching process between the target corner points and the grid points can be implemented using algorithms or heuristic methods, such as nearest neighbor matching or feature descriptor-based matching algorithms. Assume that the matching function is MatchPoints(C,G), which accepts the target corner point set C and the grid point set G, and returns the matching result.

[0054] The matching process between target edges and grid points can use similar algorithms, such as distance-based or shape-based matching methods. Assume that the matching function is MatchEdges(E,G), which accepts the target edge set E and the grid point set G and returns the matching result.

[0055] The matching results are M1 (first-class target) and M2 (second-class target).

[0056] The three-dimensional grid coordinates are obtained to form the first initial data and the second initial data respectively.

[0057] The first initial data (a type of target) is expressed as: D1 = TransformPoints(C1, M1), where TransformPoints(C, M) is a function that converts the target corner point set C into three-dimensional grid coordinates according to the matching results.

[0058] The second initial data (second type of target) is expressed as: D2 = TransformPoints(C2, M2), where TransformPoints(C, M) is a function that converts the target corner point set C into three-dimensional grid coordinates according to the matching results.

[0059] The processing module in this embodiment includes a first processor and a second processor, which are respectively arranged in the first processing channel and the second processing channel, and receive initial data from the conversion module. The first initial data generates a first processing result R1 through the first processor, and the second initial data generates a second processing result R2 through the second processor.

[0060] The processing module in this embodiment is configured to remove outliers in the initial data D whose fitting degree exceeds a third threshold, which is represented as RemoveOutliers(D,th3);

[0061] Calculate the normal vector of each point in the remaining data. The process of calculating the normal vector of each point can use geometric calculation methods, such as least squares method or nearest neighbor method, expressed as ComputeNormal(D,k), where D represents the data set and k represents the neighborhood size;

[0062] Compare the consistency of the normal vectors of the points with those of other points in the neighborhood, expressed as CompareNormals(p,N,th3), where p represents the current point and N represents the set of normal vectors of other points in the neighborhood. According to the definition and threshold of consistency, the points inconsistent with the neighborhood are marked as noise points and filtered out; the third threshold is the outlier threshold th3.

[0063] The processing module in this embodiment is further configured to set a reference point A. The selection of the reference point can be determined according to specific requirements and algorithms, for example:

[0064] Manual selection: A point can be manually selected as a benchmark based on experience or prior knowledge. Points with good quality and representativeness are usually selected as benchmarks, such as points with clear structures and high density.

[0065] Fixed point: If there is a fixed point in the registration process, it can be selected as the reference. For example, in multi-view fusion, a point in a reference view can be selected as the reference, and the points in other views can be registered with it.

[0066] Average point: If there are multiple points, their average point can be used as a benchmark. This can be calculated by averaging the point coordinates of all points. The average point can represent the overall characteristics of all point data to a certain extent.

[0067] Global reference point: In some cases, a representative global reference point can be chosen as a benchmark. This can be a pre-determined model, such as a CAD model or map data. Other clouds can be aligned to the global reference point.

[0068] Inertial Coordinate System: You can use an inertial coordinate system as a reference. For example, choose the global earth coordinate system or the local coordinate system as a reference so that the points are aligned to the same coordinate system.

[0069] In this embodiment, the reference point A is set by manual selection, the initial data is iterated to match the reference point, and the points in the initial data whose fitting degree exceeds the fourth threshold are removed. The iterative matching function is IterateAndMatch(D,A,th4); the fourth threshold th4 is the error threshold.

[0070] The result generation module of this embodiment includes a first generator and a second generator, which are respectively arranged in the first processing channel and the second processing channel;

[0071] and is configured such that the first generator maps the first processing result R1 into a quadrilateral mesh M3, generates three-dimensional mesh data, and reconstructs a type of target surface model S1, and the mapping function is MapToQuadMesh(R1,M3);

[0072] At the same time, the second generator is configured to project the second processing result R2 to the plane mesh M4 at multiple angles, generate plane mesh data, and reconstruct the second type of target model S2. The projection function is ProjectToPlanarMesh(R2,M4).

[0073] In this embodiment, the first type of target is a three-dimensional image, and the second type of target is a two-dimensional image.

[0074] Embodiment 2

[0075] The difference between this embodiment and the first embodiment is that the reference point in the processing module adopts a fixed point P S For the point P in the initial data t Perform registration and calculate the error after registration. The formula is as follows:

[0076]

[0077]

[0078] Error function:

[0079]

[0080] In summary, a filter is added to the image acquisition module of this technology, which can filter out irregular images and classify images by dimension; this technology sets up dual channels, which can process two-dimensional images and three-dimensional images at the same time without interfering with each other, making it convenient for operators to use.

[0081] The preferred specific embodiments of the present invention are described in detail above. It should be understood that ordinary technicians in the field can make many modifications and changes based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by technicians in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. An irregular image processing system, characterized in that: include An acquisition module acquires data from different angles of the same target image to form image target data; The acquisition module further includes a screener, a first processing channel, and a second processing channel. The screener is used to acquire preset screening data and classification data, and remove target data that meets the screening data. The remaining target data is divided into a first-class target and a second-class target by the screener. The first-class target enters the first processing channel, and the second-class target enters the second processing channel. The sifter is configured to perform the following screening steps: Obtain preset screening data, project the data onto a three-dimensional grid area, and mark preset corner points and reference edges; Mapping target corner points and target edges of target image data into a three-dimensional grid area; The target image is a regular image if the fit between the target corner point and the marked preset corner point is within a first threshold, and the fit between the target edge and the reference edge is within a second threshold, and the rest are irregular images; The first threshold is a preset corner point difference, and the second threshold is a preset edge difference; The sifter is further configured to perform the following classification steps: Randomly obtain three points in the target image, and connect the three points in sequence to form a closed plane as the reference plane; Randomly obtain the fourth point of the target image; Each side of the reference plane extends to span the entire three-dimensional grid area, and the reference plane covers the fourth point position, then the target image is the second type of target, and the rest are the first type of targets; The conversion module includes a first converter and a second converter, which are respectively arranged in the first processing channel and the second processing channel and connected to the output end of the image acquisition module. The first type of target is converted into first initial data through the first converter, and the second type of target is converted into second initial data through the second converter. The processing module includes a first processor and a second processor, which are respectively arranged in the first processing channel and the second processing channel, and receive the initial data from the conversion module, the first initial data is processed by the first processor to generate a first processing result, and the second initial data is processed by the second processor to generate a second processing result; The result generation module includes a first generator and a second generator, which are respectively arranged in the first processing channel and the second processing channel, and receive the processing data from the processing module to reconstruct and optimize the model.

2. The irregular image processing system according to claim 1, characterized in that: The conversion module is configured to match the target corner points and target edges in the first-class target and the second-class target with the grid points of the three-dimensional grid area to obtain the three-dimensional grid coordinates to form the first initial data and the second initial data respectively.

3. The irregular image processing system according to claim 1, characterized in that: The processing module is configured to remove outlier points in the initial data whose fit exceeds a third threshold, calculate the normal vector of each point in the remaining data, compare its consistency with the normal vectors of other points in the neighborhood, mark points inconsistent with the neighborhood as noise points, and filter them out, and the third threshold is the outlier threshold.

4. The irregular image processing system according to claim 1, characterized in that: The processing module is further configured to set a reference point, iterate the initial data to match the reference point, and remove points in the initial data whose fit exceeds a fourth threshold; the fourth threshold is an error threshold.

5. The irregular image processing system according to claim 1, characterized in that: The result generating module, The first generator is configured to map the first processing result into a quadrilateral mesh, generate three-dimensional mesh data, and reconstruct a type of target surface model; At the same time, the second generator is configured to project the second processing result to a plane grid at multiple angles, generate plane grid data, and reconstruct the second type of target model.

6. The irregular image processing system according to claim 1, characterized in that: The first processing channel and the second processing channel are independent of each other.

Images