Image segmentation method, electronic device, and computer-readable storage medium

Abstract

The embodiments of the present application relate to the field of image processing technology, and disclose an image segmentation method, an electronic device, and a computer-readable storage medium. The method includes: determining a target pattern based on a first image, a preset central diffraction order pattern, and an image matching algorithm; expanding the target pattern according to a preset number of diffraction orders to obtain a second image; determining a second key point in the second image, and determining a first key point corresponding to the second key point in the first image; constructing a distortion model based on the coordinates of the second key point and the coordinates of the first key point; and performing diffraction order segmentation on the first image based on the segmentation result of the second image and the distortion model. The DOE projection image can be accurately and quickly segmented into diffraction orders without the need for precise tooling, thereby providing a scientific data reference for product quality control of projection equipment.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to an image segmentation method, an electronic device, and a computer-readable storage medium. Background Art

[0002] A diffractive optical element (DOE) is an optical element that is based on the principle of optical diffraction and uses integrated circuit manufacturing technology to etch a microstructure on an optical device to split or shape a light beam. DOE is lightweight, compact, inexpensive to manufacture, and easy to integrate, which has led to its widespread application in material processing, medical treatment, projected structured light, and other fields.

[0003] In applications related to structured light projection, the use of DOE is very common. For example, depth cameras based on speckle structured light are integrated with DOE. However, in the specific implementation of structured light projection, the number of projected points or patterns is relatively complex, and more serious distortion usually occurs in the direction of larger angles. Therefore, in order to ensure the quality of the projection equipment, technicians need to test the quality of each diffraction order pattern of the structured light projected by the DOE during the factory inspection or self-inspection of the projection equipment. When performing these tests, the image projected by the DOE needs to be segmented into diffraction orders first.

[0004] However, when projection equipment manufacturers segment the diffraction orders of the image projected by the DOE, they often fix the image area and then analyze the projection pattern of a certain area in the image. This requires precise tooling and strict requirements on the DOE projection imaging angle and installation position. Even slight changes in the internal structure of the camera will affect the segmentation results, and thus affect the detection results. The entire process has poor robustness. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide an image segmentation method, an electronic device, and a computer-readable storage medium, which can accurately and quickly segment the diffraction orders of DOE projection images without the need for precise tooling, thereby providing scientific data reference for product quality control of projection equipment.

[0006] To solve the above technical problems, an embodiment of the present application provides an image segmentation method, comprising the following steps: determining a target pattern based on a first image, a preset pattern of a central diffraction order and a preset image matching algorithm; wherein the first image is a projection image to be segmented, and the target pattern is a pattern of a central diffraction order of the first image; expanding the target pattern according to a preset number of diffraction orders to obtain a second image; wherein the second image is a projection image with no distortion and a segmentation result of a known diffraction order; determining a second key point in the second image, and determining a first key point corresponding to the second key point in the first image; constructing a distortion model based on the coordinates of the second key point and the coordinates of the first key point; and performing diffraction order segmentation on the first image based on the segmentation result of the second image and the distortion model.

[0007] An embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned image segmentation method.

[0008] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned image segmentation method when executed by a processor.

[0009] The image segmentation method, electronic device and computer-readable storage medium provided in the embodiments of the present application determine the pattern of the central diffraction order of the first image as the target pattern based on the projection image to be segmented, i.e., the first image, the preset pattern of the central diffraction order and the preset image matching algorithm, expand the target pattern according to the preset number of diffraction orders, and obtain a projection image without distortion and with known diffraction order segmentation results, i.e., the second image, and then determine the second key point in the second image, and determine the first key point corresponding to the second key point in the first image, and then construct a distortion model based on the coordinates of the second key point and the coordinates of the first key point, and finally perform diffraction order segmentation on the first image based on the segmentation results of the diffraction orders of the second image and the constructed distortion model. Compared with As for the technical solution of fixing the projection image area and then taking the projection pattern of a certain area in the image for analysis, the embodiments of the present application do not require precise tooling, and do not need to strictly stipulate the projection imaging angle and installation position of the DOE of the projection equipment. It is only necessary to find the pattern of the central diffraction order in the projection image to be segmented, and use the pattern of the central diffraction order of the projection image to be segmented to expand the distortion-free projection image. The segmentation of the diffraction order of the expanded image is known. The server determines the distortion relationship based on the projection image to be segmented and the expanded projection image, and distorts the segmentation of the diffraction order of the expanded image. The DOE projection image can be accurately and quickly segmented into diffraction orders, thereby providing scientific data reference for product quality control of the projection equipment.

[0010] In addition, the number of the first key points is N, the number of the second key points is N, and N is an integer greater than 1. The distortion model is constructed according to the coordinates of the second key points and the coordinates of the first key points, including: taking each of the second key points as the target second key point in turn, and constructing an equation according to the preset M unknown parameters, the coordinates of the target second key point, and the coordinates of the first key point corresponding to the target second key point; wherein M is an integer less than N; the N equations constructed jointly form an overdetermined equation group; and using the least squares method to solve the M unknown parameters in the overdetermined equation group. number, and obtain the distortion model. Considering that the projected image to be segmented has certain distortion in all directions, but the distortion law is usually the same, therefore, determining the distortion relationship between the second key point on the second image and the first key point on the first image can represent the distortion relationship of the two images to a certain extent. The embodiment of the present application selects N key points, uses M parameters to fit the distortion relationship, and requires that N is greater than M, that is, constructs an overdetermined set of equations and finds the least squares solution. The distortion model composed of the parameters solved in this way can more accurately, scientifically and reasonably characterize the degree of distortion of the first image compared with the second image.

[0011] In addition, the second image is a rectangle, and the second key points include a second corner point and a second midpoint. The second corner points are the four vertices of the second image, and the second midpoints are the midpoints of the four sides of the second image. The method of determining the first key point corresponding to the second key point in the first image includes: performing edge detection on the first image to determine the edge position of the first image; searching for the upper left point, lower left point, upper right point and lower right point in the edge position to obtain first corner points corresponding to the second corner points; making a horizontal line and a vertical line based on the center of the first image, determining the intersection of the horizontal line and the edge position and the intersection of the vertical line and the edge position, and obtaining first midpoints corresponding to the second midpoints. According to the distortion law, the distortion of the corner points is larger, while the distortion of the midpoints is actually smaller. Therefore, the present application selects eight key points, namely four corner points and four midpoints. The distortion model determined based on four corner points and four midpoints is more scientific and accurate.

[0012] In addition, the edge detection of the first image and determination of the edge position of the first image include: performing continuation processing on the first image to obtain a continuous first image; wherein the continuation processing includes any one or any combination of the following: Gaussian blur, morphological erosion and morphological dilation; according to a preset edge extraction operator, edge detection is performed on the continuous first image to determine the edge position of the continuous first image. The edges of the projected image to be segmented may have burrs and unevenness. Direct edge detection may not accurately determine the edge position. In an embodiment of the present application, the first image is first subjected to continuation processing such as Gaussian blur, morphological erosion, and morphological dilation to make the first image flat and smooth, and then edge detection is performed on the continuous first image to determine a more accurate edge position.

[0013] In addition, the diffraction order segmentation of the first image is performed based on the segmentation result of the second image and the distortion model, including: taking the diffraction order of the second image as the target diffraction order in turn, and determining the coordinates of each edge point of the pattern of the target diffraction order; inputting the coordinates of each edge point of the pattern of the target diffraction order into the distortion model to obtain the coordinates of each segmentation point of the target diffraction order on the first image; wherein, each segmentation point is a point in the first image corresponding to each edge point of the pattern of the target diffraction order; the diffraction order segmentation of the first image is performed based on the coordinates of each segmentation point to obtain the pattern of the target diffraction order of the first image, considering that the points inside the pattern of the diffraction order will not exceed the distorted edge even if distorted, it is only necessary to determine the distorted coordinates of the edge points of the pattern of the diffraction order to determine the area corresponding to the diffraction order in the first image, which greatly improves the accuracy and efficiency of diffraction order segmentation.

[0014] In addition, the preset image matching algorithm is a phase correlation algorithm, and the target pattern is determined based on the first image, the preset central diffraction order pattern and the preset image matching algorithm, including: performing rotation matching in the first image based on the preset central diffraction order pattern to determine the target pattern; determining the rotation angle based on the target pattern and the preset central diffraction order pattern. Taking into account that the projected image to be segmented will not only be distorted, but will also have a certain angle offset, the phase correlation algorithm not only performs image matching in the horizontal and vertical directions, but can also automatically increase the angle for rotation matching, thereby finding the pattern with the highest matching degree in the first image as the target pattern. The target pattern determined in this way is more in line with the actual situation, that is, the pattern of the central diffraction order of the first image determined is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] One or more embodiments are exemplarily described by the figures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments.

[0016] Figure 1 is a flowchart of an image segmentation method according to an embodiment of the present application;

[0017] Figure 2 is a schematic diagram of image matching provided according to an embodiment of the present application;

[0018] Figure 3 is a schematic diagram of another image matching provided according to an embodiment of the present application;

[0019] Figure 4 is a schematic diagram of a second image provided according to an embodiment of the present application;

[0020] Figure 5 is a flowchart of constructing a distortion model based on the coordinates of the second key point and the coordinates of the first key point according to one embodiment of the present application;

[0021] Figure 6 is a flow chart of determining a first key point corresponding to a second key point in a first image according to one embodiment of the present application;

[0022] Figure 7 1 is a schematic diagram of the distribution of a second corner point, a first corner point, a second midpoint, and a first midpoint provided in one embodiment of the present application;

[0023] Figure 8 is a schematic diagram of the distribution of a second corner point, a first corner point, a second midpoint, and a first midpoint according to an embodiment of the present application;

[0024] Figure 9 is a flowchart of performing diffraction order segmentation on a first image based on a segmentation result of a second image and a distortion model according to an embodiment of the present application;

[0025] Figure 10 is a schematic diagram of an image segmentation system according to another embodiment of the present application;

[0026] Figure 11 is a structural diagram of an electronic device according to another embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those skilled in the art that in each embodiment of the present application, many technical details are proposed to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be implemented. The division of the following embodiments is for convenience of description and should not constitute any limitation on the specific implementation of the present application. The various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0028] An embodiment of the present application relates to an image segmentation method, which is applied to an electronic device, wherein the electronic device can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described using a server as an example. The implementation details of the image segmentation method of this embodiment are described in detail below. The following content is only the implementation details provided for easy understanding and is not necessary for implementing this solution.

[0029] The specific process of the image segmentation method of this embodiment can be as follows: Figure 1 Shown, including:

[0030] Step 101 : determining a target pattern according to a first image, a preset central diffraction order pattern, and a preset image matching algorithm.

[0031] Specifically, the first image is the projection image to be segmented, which is an image projected by the DOE of the projection device to be tested. The server uses the image as the first image and performs diffraction order segmentation on the first image.

[0032] In one example, a technician can adjust the projection device to be tested to the working distance of the device (or the imaging distance of a suitable DOE), project the optical pattern diffracted by the DOE onto a preset plane, and capture the projected image with a camera to obtain the projection image to be segmented, wherein the preset plane can be a white plane with good reflectivity and clear imaging.

[0033] In a specific implementation, the server can perform a matching search in the first image based on the preset central diffraction order pattern and the preset image matching algorithm, and determine that the image block with the highest matching value with the preset central diffraction order pattern is the target pattern, and the target pattern is the central diffraction order pattern of the first image, wherein the preset central diffraction order pattern can be set by a technician in this field based on the brand model of the projection device to be tested, and the preset image matching algorithm can be set by a technician in this field according to actual needs.

[0034] In one example, the process of determining the target pattern, that is, performing image matching based on the first image, the pattern of the preset central diffraction order, and the preset image matching algorithm, can be as follows: Figure 2 As shown, the server determines that the image block at the middle position of the first image has the highest pattern matching value with the preset central diffraction order, and the server determines the image block as the target pattern.

[0035] In one example, the preset image matching algorithm may be a scale-invariant feature transform (SIFT) matching algorithm, a function link artificial neural network (FLANN) matching algorithm, a block matching algorithm, or the like.

[0036] In one example, the preset image matching algorithm is a phase-correlation algorithm, which is a rotation matching algorithm. Considering that the projection image to be segmented will not only be distorted but also have a certain offset in angle, the phase correlation algorithm can be used not only to perform image matching in the horizontal and vertical directions, but also to automatically increase the angle for image matching in the rotation direction. That is, the server performs rotation matching in the first image based on the preset central diffraction order pattern to determine the target pattern. After determining the target pattern, the rotation angle can be determined and output based on the determined target pattern and the preset central diffraction order pattern. In this way, the determined target pattern is more in line with the actual situation, that is, the determined central diffraction order pattern of the first image is more accurate.

[0037] In one example, the process of determining the target pattern, that is, performing rotational image matching in the first image based on the pattern of the preset central diffraction order, can be as follows: Figure 3 As shown, after the server rotates the preset central diffraction order pattern at various angles, it searches the first image to find the image block with the highest matching value as the target pattern.

[0038] Step 102: Expand the target pattern according to a preset number of diffraction orders to obtain a second image.

[0039] Specifically, after the server determines the target pattern in the first image, it can expand the target pattern according to the preset number of diffraction orders to obtain a second image. The second image is a projection image of the segmentation result without distortion and with known diffraction orders, and the target pattern is the central diffraction order of the second image. The preset number of diffraction orders can be set by technical personnel in this field based on the requirements of the projection product.

[0040] In one example, the preset number of diffraction orders is 3*3, that is, a total of 9 diffraction orders, the target pattern is the central diffraction order, and the server needs to expand the other 8 diffraction orders to the surrounding areas. The server takes the target pattern as the center and copies a target pattern to the top, bottom, left, right, upper left, lower left, upper right and lower right respectively. These 9 target patterns are spliced ​​to obtain a second image. The second image can be as follows Figure 4 As shown in the figure, there are 9 diffraction orders in total. The central diffraction order is assumed to be the (0, 0) diffraction order. The remaining 8 diffraction orders take the diffraction order to the right of the (0, 0) diffraction order as the starting point, and rotate counterclockwise as follows: (0, 1) diffraction order, (1, 1) diffraction order, (1, 0) diffraction order, (-1, 1) diffraction order, (-1, 0) diffraction order, (-1, -1) diffraction order, (0, -1) diffraction order and (1, -1) diffraction order.

[0041] In another example, the preset number of diffraction orders is 10*7, that is, a total of 70 diffraction orders, and the target pattern is the central diffraction order. The server needs to expand the other 69 diffraction orders in all directions with the target pattern as the center.

[0042] Step 103: determine a second key point in the second image, and determine a first key point corresponding to the second key point in the first image.

[0043] Specifically, after the server generates the second image, it may select several second key points in the second image, and determine first key points corresponding to the second key points in the first image.

[0044] In one example, considering that the second image is a rectangle, the server may select four vertices of the second image as second key points, and find points corresponding to the four vertices in the first image as first key points.

[0045] In another example, in order to make the subsequent diffraction order segmentation extremely accurate, the server may select all edge points of the second image as second key points, and find points corresponding to these edge points in the first image as first key points.

[0046] Step 104: construct a distortion model based on the coordinates of the second key point and the coordinates of the first key point.

[0047] In a specific implementation, after the server determines the second key point and the first key point corresponding to the second key point, it can construct a distortion model based on the coordinates of the second key point and the coordinates of the first key point. For example, the transformation relationship between the second key point and the first key point is constructed by setting unknown parameters, and these unknown parameters are solved based on the coordinates of the second key point and the coordinates of the first key point to obtain the distortion model. For example, using the initial distortion model constructed, the server inputs the coordinates of a sufficient number of second key points and the coordinates of the first key point corresponding to the second key point into the initial distortion model, and iteratively trains the initial distortion model until the training is completed to obtain a mature distortion model.

[0048] Step 105 : Segment the first image into diffraction orders according to the segmentation result of the second image and the distortion model.

[0049] Specifically, after obtaining the distortion model, the server may perform diffraction order segmentation on the first image according to the diffraction order segmentation result of the second image and the constructed distortion model.

[0050] In one example, the server can input the coordinates of all points in the pattern of the diffraction order of a second image into the distortion model to obtain the coordinates of several expected points. The server connects these expected points to obtain the position of the diffraction order in the first image, thereby realizing the diffraction order segmentation of the first image.

[0051] In this embodiment, the server determines the pattern of the central diffraction order of the first image as the target pattern based on the projection image to be segmented, i.e., the first image, the preset pattern of the central diffraction order and the preset image matching algorithm, and expands the target pattern according to the preset number of diffraction orders to obtain a projection image without distortion and with known diffraction order segmentation results, i.e., the second image. Subsequently, the second key point is determined in the second image, and the first key point corresponding to the second key point is determined in the first image. A distortion model is then constructed based on the coordinates of the second key point and the coordinates of the first key point. Finally, the first image is segmented into diffraction orders based on the segmentation results of the diffraction orders of the second image and the constructed distortion model. Compared with the fixed projection image area, the server then segments the diffraction orders into smaller ones. As for the technical solution of analyzing the projection pattern of a certain area in the image, the embodiment of the present application does not require precise tooling, and does not need to strictly stipulate the projection imaging angle and installation position of the DOE of the projection equipment. It only needs to find the pattern of the central diffraction order in the projection image to be segmented, and use the pattern of the central diffraction order of the projection image to be segmented to expand the distortion-free projection image. The segmentation of the diffraction order of the expanded image is known. The server determines the distortion relationship based on the projection image to be segmented and the expanded projection image, and distorts the segmentation of the diffraction order of the expanded image. The DOE projection image can be accurately and quickly segmented into diffraction orders, thereby providing scientific data reference for product quality control of the projection equipment.

[0052] In one embodiment, the server determines that the number of second key points is N, the number of first key points is also N, and N is an integer greater than 1. The server constructs a distortion model based on the coordinates of the second key points and the coordinates of the first key points, which can be achieved by Figure 5 The steps shown in the figure are as follows:

[0053] In step 201, each second key point is sequentially used as a target second key point, and an equation is constructed according to the preset M unknown parameters, the coordinates of the target second key point, and the coordinates of the first key point corresponding to the target second key point.

[0054] Specifically, the embodiment of the present application adopts the method of corresponding parameter fitting to construct a distortion model. After the server determines several second key points and several first key points, it can traverse each second key point, and take each second key point as the target second key point in turn. The equation is constructed based on the preset M unknown parameters, the coordinates of the target second key point, and the coordinates of the first key point corresponding to the target second key point, where M is an integer less than N, which ensures that the number of unknown parameters is less than the number of known point pairs, otherwise the unknown parameters cannot be calculated subsequently.

[0055] In one example, the number of unknown parameters is 5, that is, the first unknown parameter, the second unknown parameter, the third unknown parameter, the fourth unknown parameter, and the fifth unknown parameter are preset. Therefore, the number of second key points determined by the server is at least 6. The server constructs an equation based on the preset M unknown parameters, the coordinates of the target second key point, and the coordinates of the first key point corresponding to the target second key point, which can be expressed by the following formula:

[0056] X d =X u (1+K1*X u 2 +K2*Y u 2 +K3*X u Y u +K4*X u +K5*Y u )

[0057] Y d =Y u (1+K1*X u 2 +K2*Y u 2 +K3*X u Y u +K4*X u +K5*Y u )

[0058] Where, (X u , Y u ) is the coordinate of the second key point of the target, (X d , Y d ) is the coordinate of the first key point corresponding to the second key point of the target, K1 is the first unknown parameter, K2 is the second unknown parameter, K3 is the third unknown parameter, K4 is the fourth unknown parameter, and K5 is the fifth unknown parameter.

[0059] In step 202, the N equations constructed together form an overdetermined system of equations.

[0060] Specifically, N equations can be constructed based on the coordinates of the N second key points and the coordinates of the first key points corresponding to the second key points. The N equations constructed jointly by the server, since the number of known point pairs is greater than the number of unknown parameters, the N equations constructed jointly by the server together constitute an overdetermined system of equations, that is, a system of equations in which the number of equations is greater than the number of unknown quantities.

[0061] Step 203: Use the least square method to solve the M unknown parameters in the overdetermined equations to obtain a distortion model.

[0062] Specifically, the overdetermined system of equations has no exact solution. The server can use the least squares method to find the least squares solution of the overdetermined system of equations, thereby solving the M unknown parameters in the overdetermined system of equations and obtaining the distortion model.

[0063] In this embodiment, the number of the first key points is N, the number of the second key points is N, and N is an integer greater than 1. The distortion model is constructed according to the coordinates of the second key points and the coordinates of the first key points, including: taking each of the second key points as a target second key point in turn, and constructing an equation according to M preset unknown parameters, the coordinates of the target second key point, and the coordinates of the first key point corresponding to the target second key point; wherein M is an integer less than N; the N equations constructed together form an overdetermined equation group; and using the least squares method to solve the M unknown parameters in the overdetermined equation group. Parameters are used to obtain a distortion model. Considering that the projected image to be segmented has certain distortion in all directions, but the distortion law is usually the same, therefore, determining the distortion relationship between the second key point on the second image and the first key point on the first image can represent the distortion relationship between the two images to a certain extent. In the embodiment of the present application, N key points are selected, M parameters are used to fit the distortion relationship, and N is required to be greater than M, that is, an overdetermined set of equations is constructed to find the least squares solution. The distortion model composed of the parameters solved in this way can more accurately, scientifically and reasonably characterize the degree of distortion of the first image compared with the second image.

[0064] In one embodiment, the second image is a rectangle, and the second key points determined by the server include a second corner point and a second midpoint. The second corner points are the four vertices of the second image, and the second midpoints are the midpoints of the four sides of the second image. The server determines the first key point corresponding to the second key point in the first image by, for example, Figure 6 The steps shown are implemented as follows:

[0065] Step 301: perform edge detection on a first image to determine edge positions of the first image.

[0066] Specifically, the server may perform edge detection on the first image according to a preset edge detection algorithm to determine the edge position of the first image.

[0067] In one example, when the server performs edge detection on the first image, it can first perform continuity processing on the first image to obtain a continuity first image. The continuity processing includes but is not limited to Gaussian blur, morphological erosion, and morphological dilation. After obtaining the continuity first image, the server performs edge detection on the continuity first image according to a preset edge extraction operator (such as a canny operator, etc.) to determine the edge position of the continuity first image. Considering that the edges of the projected image to be segmented may have burrs and be rough, directly performing edge detection may not accurately determine the edge position. In an embodiment of the present application, the first image is first subjected to continuity processing such as Gaussian blur, morphological erosion, and morphological dilation to make the first image flat and smooth, and then edge detection is performed on the continuity first image to determine a more accurate edge position.

[0068] Step 302: Search the upper left point, lower left point, upper right point, and lower right point in the edge position to obtain first corner points corresponding to second corner points.

[0069] Specifically, the second corner points are the four vertices of the second image. Even after distortion, these four vertices must be located at the cornermost positions of the edge positions of the first image. Therefore, the server searches for the upper left point, lower left point, upper right point and lower right point in the edge position to obtain the first corner points corresponding to the second corner points.

[0070] In step 303 , a horizontal line and a vertical line are drawn based on the center of the first image, and the intersections of the horizontal line and the edge position and the vertical line and the edge position are determined to obtain first midpoints corresponding to the second midpoints.

[0071] Specifically, the second midpoint is the midpoint of the four sides of the second image. The degree of distortion of the midpoint is relatively small. Taking into account the distortion relationship, the server draws a horizontal line and a vertical line based on the center of the first image, and determines the intersection of the horizontal line and the edge position and the intersection of the vertical line and the edge position, resulting in a total of four intersection points. These four intersection points are the first midpoints corresponding to the second midpoints respectively.

[0072] In one example, the distribution diagram of the second corner point, the first corner point, the second midpoint and the first midpoint can be as follows: Figure 7 As shown, points A, B, C, and D in the second image are the four second corner points, points E, F, G, and H in the second image are the four second midpoints, points A, B, C, and D in the first image are the four first corner points, and points E, F, G, and H in the first image are the four first midpoints.

[0073] In one example, the preset image matching algorithm is a phase correlation algorithm, that is, the server uses rotation matching when performing image matching. The first image has a certain rotation angle. When performing rotation matching, the server can determine the rotation angle based on the determined target pattern and the preset central diffraction order pattern. Therefore, when determining the first midpoint, the server can use the center of the first image as a reference to make a horizontal line and a vertical line respectively, and rotate the horizontal line and the vertical line respectively according to the determined rotation angle to determine the intersection of the rotated horizontal line and the edge position and the intersection of the rotated vertical line and the edge position, and obtain a total of four intersection points. These four intersection points are the first midpoints corresponding to the second midpoints respectively. When the first image has a rotation angle, the distribution of the second corner point, the first corner point, the second midpoint and the first midpoint can be as follows. Figure 8 shown.

[0074] In this embodiment, the second image is a rectangle, and the second key points include second corner points and second midpoints. The second corner points are the four vertices of the second image, and the second midpoints are the midpoints of the four sides of the second image. The determining of the first key points corresponding to the second key points in the first image includes: performing edge detection on the first image to determine the edge position of the first image; searching for the upper left point, lower left point, upper right point and lower right point in the edge position to obtain first corner points corresponding to the second corner points respectively; making a horizontal line and a vertical line with the center of the first image as a reference, determining the intersection of the horizontal line and the edge position and the intersection of the vertical line and the edge position, and obtaining first midpoints corresponding to the second midpoints respectively. According to the distortion law, the distortion of the corner points is larger, while the distortion of the midpoints is actually smaller. Therefore, this application selects eight key points, namely four corner points and four midpoints. The distortion model determined based on the four corner points and the four midpoints is more scientific and accurate.

[0075] In one embodiment, the server performs diffraction order segmentation on the first image based on the segmentation result of the second image and the distortion model, which can be done as follows: Figure 9 The steps shown in the figure are as follows:

[0076] Step 401 : sequentially taking the diffraction orders of the second image as target diffraction orders, and determining the coordinates of each edge point of the pattern of the target diffraction order.

[0077] Step 402 : Input the coordinates of each edge point of the target diffraction order pattern into the distortion model to obtain the coordinates of each segmentation point of the target diffraction order on the first image.

[0078] Specifically, considering that the points inside the pattern of the diffraction order will not exceed the edge after distortion even if they are distorted, it is only necessary to determine the coordinates of the edge points of the pattern of the diffraction order after distortion to determine the area corresponding to the diffraction order in the first image. The server takes each diffraction order of the second image as the target diffraction order in turn, and determines the coordinates of each edge point of the pattern of the target diffraction order. The server inputs the coordinates of each edge point of the pattern of the target diffraction order into the distortion model to obtain the coordinates of each segmentation point of the target diffraction order on the first image. Each segmentation point is the point in the first image corresponding to each edge point of the pattern of the target diffraction order, which is the boundary of the target diffraction order on the first image.

[0079] Step 403 : segment the first image into diffraction orders according to the coordinates of each segmentation point to obtain a pattern of target diffraction orders of the first image.

[0080] Specifically, after the server obtains the coordinates of each segmentation point of the target diffraction order on the first image, it can segment the diffraction order of the first image according to the coordinates of each segmentation point, that is, connect each segmentation point to obtain the pattern of the target diffraction order of the first image, and finally realize the diffraction order segmentation of the first image, which greatly improves the accuracy and efficiency of diffraction order segmentation.

[0081] The steps of the various methods above are divided only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0082] Another embodiment of the present application relates to an image segmentation system. The implementation details of the image segmentation system of this embodiment are described in detail below. The following content is only for the convenience of understanding the implementation details and is not necessary for the implementation of this solution. The schematic diagram of the image segmentation system of this embodiment can be as follows: Figure 10 As shown, it includes: a projection device 51 to be tested, a camera 52 and a segmentation device 53, and the segmentation device 53 includes a matching module 531, an expansion module 532, a distortion model construction module 533 and an execution module 534.

[0083] The projection device 51 to be tested is used to generate a projection image to be segmented.

[0084] The camera 52 is used to capture the projection image to be segmented to obtain a first image, and send the first image to the segmentation device 53 .

[0085] The matching module 531 is used to determine a target pattern according to the first image, a preset pattern of a central diffraction order, and a preset image matching algorithm, wherein the target pattern is a pattern of a central diffraction order of the first image.

[0086] The expansion module 532 is used to expand the target pattern according to a preset number of diffraction orders to obtain a second image, wherein the second image is a projection image of the segmentation result without distortion and with known diffraction orders.

[0087] The distortion model construction module 533 is used to determine a second key point in the second image, determine a first key point corresponding to the second key point in the first image, and construct a distortion model according to the coordinates of the second key point and the coordinates of the first key point.

[0088] The execution module 534 is configured to segment the first image into diffraction orders according to the segmentation result of the diffraction orders of the second image and the distortion model.

[0089] It is worth mentioning that all modules involved in this embodiment are logical modules. In actual applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, to highlight the innovation of this application, this embodiment does not include units that are not closely related to solving the technical problem proposed by this application. However, this does not mean that other units do not exist in this embodiment.

[0090] Another embodiment of the present application relates to an electronic device, such as Figure 11 As shown, it includes: at least one processor 601; and a memory 602 that is communicatively connected to the at least one processor 601; wherein the memory 602 stores instructions that can be executed by the at least one processor 601, and the instructions are executed by the at least one processor 601 to enable the at least one processor 601 to execute the image segmentation method in the above-mentioned embodiments.

[0091] The memory and processor are connected using a bus, which can include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors and memories. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and are therefore not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor.

[0092] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0093] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0094] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program. The program is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, and other media that can store program code.

[0095] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. An image segmentation method, characterized in that: include: Determining a target pattern based on a first image, a preset pattern of a central diffraction order, and a preset image matching algorithm; wherein the first image is a projection image to be segmented, and the target pattern is a pattern of a central diffraction order of the first image; Expanding the target pattern according to a preset number of diffraction orders to obtain a second image; wherein the second image is a projection image of a segmentation result without distortion and with known diffraction orders; determining a second key point in the second image, and determining a first key point corresponding to the second key point in the first image; constructing a distortion model according to the coordinates of the second key point and the coordinates of the first key point; The first image is segmented into diffraction orders according to the segmentation result of the second image and the distortion model.

2. The image segmentation method according to claim 1, wherein: The number of the first key points is N, the number of the second key points is N, where N is an integer greater than 1, and the step of constructing a distortion model based on the coordinates of the second key points and the coordinates of the first key points includes: Sequentially taking each of the second key points as a target second key point, and constructing an equation based on M preset unknown parameters, the coordinates of the target second key point, and the coordinates of the first key point corresponding to the target second key point; wherein M is an integer less than N; The N equations constructed together form an overdetermined system of equations; The least square method is used to solve the M unknown parameters in the overdetermined equations to obtain a distortion model.

3. The image segmentation method according to claim 2, characterized in that The preset M unknown parameters include a first unknown parameter, a second unknown parameter, a third unknown parameter, a fourth unknown parameter, and a fifth unknown parameter. The method further includes: when M is 5, constructing an equation based on the preset M unknown parameters, the coordinates of the second key point of the target, and the coordinates of the first key point corresponding to the second key point of the target, which is expressed by the following formula: X d =X u (1+K1*X u 2 +K2*Y u 2 +K3*X u Y u +K4*X u +K5*Y u ) Y d =Y u (1+K1*X u 2 +K2*Y u 2 +K3*X u Y u +K4*X u +K5*Y u ) Among them, (X u ,Y u ) is the coordinate of the second key point of the target, (X d ,Y d ) is the coordinate of the first key point corresponding to the second key point of the target, K1 is the first unknown parameter, K2 is the second unknown parameter, K3 is the third unknown parameter, K4 is the fourth unknown parameter, and K5 is the fifth unknown parameter.

4. The image segmentation method according to claim 1, wherein: The second image is a rectangle, the second key points include a second corner point and a second midpoint, the second corner points are four vertices of the second image, and the second midpoints are midpoints of four sides of the second image, and determining a first key point corresponding to the second key point in the first image includes: performing edge detection on the first image to determine edge positions of the first image; Searching for an upper left point, a lower left point, an upper right point, and a lower right point in the edge position to obtain first corner points corresponding to the second corner points; A horizontal line and a vertical line are drawn with the center of the first image as a reference, and the intersection of the horizontal line and the edge position and the intersection of the vertical line and the edge position are determined to obtain the first midpoints corresponding to the second midpoints.

5. The image segmentation method according to claim 4, characterized in that: The performing edge detection on the first image to determine the edge position of the first image includes: Performing a continuous process on the first image to obtain a continuous first image; wherein the continuous process includes any one or any combination of the following: Gaussian blur, morphological erosion, and morphological dilation; Edge detection is performed on the continuous first image according to a preset edge extraction operator to determine edge positions of the continuous first image.

6. The image segmentation method according to any one of claims 1 to 4, characterized in that: The step of segmenting the first image into diffraction orders according to the segmentation result of the second image and the distortion model includes: sequentially taking the diffraction orders of the second image as target diffraction orders, and determining the coordinates of each edge point of the pattern of the target diffraction order; Inputting the coordinates of each edge point of the target diffraction order pattern into the distortion model to obtain the coordinates of each segmentation point of the target diffraction order on the first image; wherein each segmentation point is a point in the first image corresponding to each edge point of the target diffraction order pattern; The first image is segmented into diffraction orders according to the coordinates of the segmentation points to obtain a pattern of the target diffraction orders of the first image.

7. The image segmentation method according to any one of claims 1 to 4, characterized in that: The preset image matching algorithm is a phase correlation algorithm, and determining the target pattern according to the first image, the preset central diffraction order pattern, and the preset image matching algorithm includes: Performing rotational matching in the first image based on a pattern of a preset central diffraction order to determine a target pattern; A rotation angle is determined according to the target pattern and the pattern of the preset central diffraction order.

8. An image segmentation system, characterized in that: The system includes: a projection device to be tested, a camera and a segmentation device, wherein the segmentation device includes a matching module, an expansion module, a distortion model building module and an execution module; The projection device to be tested is used to generate a projection image to be segmented; The camera is used to capture the projection image to be segmented to obtain a first image, and send the first image to the segmentation device; The matching module is used to determine a target pattern based on the first image, a preset pattern of a central diffraction order, and a preset image matching algorithm, wherein the target pattern is a pattern of a central diffraction order of the first image; The expansion module is used to expand the target pattern according to a preset number of diffraction orders to obtain a second image, where the second image is a projection image of a segmentation result without distortion and with known diffraction orders; The distortion model construction module is configured to determine a second key point in the second image, determine a first key point corresponding to the second key point in the first image, and construct a distortion model based on coordinates of the second key point and the first key point; The execution module is used to perform diffraction order segmentation on the first image according to the segmentation result of the second image and the distortion model.

9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the image segmentation method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image segmentation method according to any one of claims 1 to 7 is implemented.

Images