Method and device for checking optical waveguide layout, storage medium and computer program

Abstract

The invention discloses an optical waveguide layout inspection method and device, a storage medium and a computer program, and the method comprises the steps: obtaining a to-be-inspected optical waveguide layout, extracting a grating image containing a grating structure from the optical waveguide layout, and enabling the grating image to comprise one grating subarea or at least two grating subareas; inputting the grating image into a trained machine learning model, and outputting grating information in the grating image by the machine learning model, the grating information comprises at least one of the following items: grating parameter information of each grating subarea, grating type information of each grating subarea, information about whether the grating in each grating subarea is abnormal or not, and information about whether the grating between different grating subareas is abnormal or not; wherein the training of the machine learning model comprises the following steps: acquiring a sample optical waveguide layout, and fragmenting the sample optical waveguide layout to obtain a plurality of sample grating images; marking grating information in the sample grating image to obtain a marked image, and constructing a training set based on the marked image; and training a machine learning model based on the constructed training set to obtain a trained machine learning model. According to the method and the device, manual intervention is not needed, and the optical waveguide layout can be efficiently and automatically checked.

Description

Technical Field

[0001] The present application relates to the technical field of optical waveguides, and more particularly to a method, device, storage medium, and computer program for inspecting an optical waveguide layout. Background Art

[0002] An augmented reality (AR) optical waveguide is a key component for AR technology, which can superimpose virtual digital information on the real world and is increasingly applied in fields such as education, medical treatment, entertainment, industrial design, and tourism. A diffractive optical waveguide is a type of AR optical waveguide that uses the diffraction characteristics of a grating to transmit a virtual image to the user's eyes. Diffractive optical waveguides have advantages such as full-color imaging, thin and light, and a large field of view angle, and are one of the mainstream solutions in the current AR glasses market. During the manufacturing process of a diffractive optical waveguide, a graphical design system (GDS) layout is used to describe and layout the structure of the optical waveguide.

[0003] In the prior art, the design of the GDS layout is usually completed manually by engineers, which may lead to design errors and inconsistencies. Especially in a complex optical waveguide system, it becomes increasingly cumbersome to inspect and correct the GDS layout. In addition, due to the microscopic structure of the optical waveguide system, traditional manual inspection methods are difficult to meet the requirements of high efficiency and accuracy. Summary of the Invention

[0004] The present application is proposed to solve the above problems. According to one aspect of the present application, there is provided a method for inspecting an optical waveguide layout, the method including: obtaining an optical waveguide layout to be inspected, extracting a grating image including a grating structure from the optical waveguide layout, where the grating image includes one grating partition or at least two grating partitions; inputting the grating image into a trained machine learning model, and outputting grating information in the grating image by the machine learning model, where the grating information includes at least one of the following: grating parameter information of each grating partition, grating type information of each grating partition, information on whether the grating in each grating partition is abnormal, and information on whether the grating between different grating partitions is abnormal; wherein the training of the machine learning model includes: obtaining a sample optical waveguide layout, performing fragmentation processing on the sample optical waveguide layout to obtain a plurality of sample grating images; annotating the grating information in the sample grating images to obtain an annotated image, and constructing a training set based on the annotated image; and training the machine learning model based on the constructed training set to obtain a trained machine learning model.

[0005] In one embodiment of the present application, the grating parameter information includes at least one of the following: grating period, grating angle, grating line width, grating duty cycle, grating depth, and grating refractive index.

[0006] In one embodiment of the present application, the inspection method further includes: comparing the grating parameter information output by the machine learning model with preset grating parameter information, and when the grating parameter information output by the machine learning model is inconsistent with the preset grating parameter information, outputting a first prompt message.

[0007] In one embodiment of the present application, the inspection method further includes: when the grating parameter information output by the machine learning model is inconsistent with the preset grating parameter information, modifying the optical waveguide layout based on the preset grating parameter information.

[0008] In one embodiment of the present application, the grating type information includes grating dimension and / or grating shape.

[0009] In one embodiment of the present application, the inspection method further includes: comparing the grating type information output by the machine learning model with preset grating type information, and when the grating type information output by the machine learning model is inconsistent with the preset grating type information, outputting a second prompt message.

[0010] In one embodiment of the present application, the information on whether the gratings in each grating partition are abnormal includes the grating normal information and grating abnormal information in each grating partition, where the grating abnormal information includes abnormal grating splicing within the partition and / or missing gratings within the partition.

[0011] In one embodiment of the present application, the information on whether the gratings between different grating partitions are abnormal includes the grating normal information and grating abnormal information between different grating partitions, where the grating abnormal information includes abnormal grating splicing between partitions.

[0012] In one embodiment of the present application, the grating abnormal information further includes information indicating the coordinates of the abnormal location.

[0013] In one embodiment of the present application, the machine learning model includes a first machine learning model, a second machine learning model, and a third machine learning model, where: the first machine learning model includes a regressor for outputting the grating parameter information; the second machine learning model includes a first classifier for outputting the grating type information; the third machine learning model includes a second classifier for outputting the information on whether the gratings are abnormal.

[0014] In one embodiment of the present application, the training of the first machine learning model includes: annotating the grating parameters in the sample grating image to obtain the annotated image, constructing a training set and a validation set based on the annotated image; training the first machine learning model based on the training set to obtain a first machine learning model to be verified; and verifying the first machine learning model to be verified based on the validation set to obtain a trained first machine learning model.

[0015] In one embodiment of the present application, the training of the second machine learning model includes: obtaining sample optical waveguide layouts with different grating dimensions to obtain a plurality of sample grating images; annotating the grating types in the sample grating images to obtain the annotated images, constructing a training set and a validation set based on the annotated images; training the second machine learning model based on the training set to obtain a second machine learning model to be verified; and verifying the second machine learning model to be verified based on the validation set to obtain a trained second machine learning model.

[0016] In one embodiment of the present application, the training of the third machine learning model includes: obtaining a normal sample optical waveguide layout and an abnormal sample optical waveguide layout to obtain a plurality of sample grating images; annotating whether the grating in the sample grating image is abnormal and the abnormal type in the case of abnormality to obtain the annotated image, constructing a training set based on the annotated image; training the third machine learning model based on the training set to obtain a third machine learning model to be verified; and verifying the third machine learning model to be verified based on the validation set to obtain a trained third machine learning model.

[0017] In one embodiment of the present application, the extraction of the grating image including the grating structure from the optical waveguide layout includes: fragmenting the optical waveguide layout to obtain a plurality of grating images including the grating structure; or obtaining user input, and extracting the grating image including the grating structure from the optical waveguide layout based on the user input, where the user input indicates the position coordinate range of the grating image to be extracted in the optical waveguide layout.

[0018] In one embodiment of the present application, the extraction operation of the grating image is performed based on graphic design system software.

[0019] According to another aspect of the present application, there is provided an inspection device for an optical waveguide layout. The device includes a memory and a processor. A computer program is stored on the memory and run by the processor. When the computer program is run by the processor, the processor is caused to perform the following steps: obtaining an optical waveguide layout to be inspected, extracting a grating image including a grating structure from the optical waveguide layout, where the grating image includes one grating partition or at least two grating partitions; inputting the grating image into a trained machine learning model, and outputting grating information in the grating image by the machine learning model, where the grating information includes at least one of the following: grating parameter information of each grating partition, grating type information of each grating partition, information on whether the grating in each grating partition is abnormal, and information on whether the grating between different grating partitions is abnormal; wherein the training of the machine learning model includes: obtaining a sample optical waveguide layout, performing fragmentation processing on the sample optical waveguide layout to obtain a plurality of sample grating images; annotating the grating information in the sample grating images to obtain an annotated image, and constructing a training set based on the annotated image; and training the machine learning model based on the constructed training set to obtain a trained machine learning model.

[0020] In an embodiment of the present application, the grating parameter information includes at least one of the following: grating period, grating angle, grating line width, grating duty cycle, grating depth, and grating refractive index.

[0021] In an embodiment of the present application, when the computer program is run by the processor, the processor is further caused to perform the following steps: comparing the grating parameter information output by the machine learning model with preset grating parameter information, and outputting a first prompt message when the grating parameter information output by the machine learning model is inconsistent with the preset grating parameter information.

[0022] In an embodiment of the present application, when the computer program is run by the processor, the processor is further caused to perform the following steps: when the grating parameter information output by the machine learning model is inconsistent with the preset grating parameter information, modifying the optical waveguide layout based on the preset grating parameter information.

[0023] In an embodiment of the present application, the grating type information includes grating dimension and / or grating shape.

[0024] In an embodiment of the present application, when the computer program is run by the processor, the processor is further caused to perform the following steps: comparing the grating type information output by the machine learning model with preset grating type information, and outputting a second prompt message when the grating type information output by the machine learning model is inconsistent with the preset grating type information.

[0025] In one embodiment of the present application, the information on whether the gratings in each of the grating partitions are abnormal includes the grating normal information and the grating abnormal information in each of the grating partitions, where the grating abnormal information includes abnormal splicing of the gratings within the partition and / or missing of the gratings within the partition.

[0026] In one embodiment of the present application, the information on whether the gratings between different grating partitions are abnormal includes the grating normal information and the grating abnormal information between different grating partitions, where the grating abnormal information includes abnormal splicing of the gratings between partitions.

[0027] In one embodiment of the present application, the grating abnormal information further includes information indicating the coordinates of the abnormal location.

[0028] In one embodiment of the present application, the machine learning model includes a first machine learning model, a second machine learning model, and a third machine learning model, where: the first machine learning model includes a regressor for outputting the grating parameter information; the second machine learning model includes a first classifier for outputting the grating type information; the third machine learning model includes a second classifier for outputting the information on whether the gratings are abnormal.

[0029] In one embodiment of the present application, the training of the first machine learning model includes: annotating the grating parameters in the sample grating image to obtain an annotated image, constructing a training set and a validation set based on the annotated image; training the first machine learning model based on the training set to obtain a first machine learning model to be verified; and verifying the first machine learning model to be verified based on the validation set to obtain a trained first machine learning model.

[0030] In one embodiment of the present application, the training of the second machine learning model includes: obtaining sample optical waveguide layouts of different grating dimensions to obtain a plurality of sample grating images; annotating the grating types in the sample grating images to obtain an annotated image, constructing a training set and a validation set based on the annotated image; training the second machine learning model based on the training set to obtain a second machine learning model to be verified; and verifying the second machine learning model to be verified based on the validation set to obtain a trained second machine learning model.

[0031] In one embodiment of the present application, the training of the third machine learning model includes: obtaining normal sample optical waveguide layouts and abnormal sample optical waveguide layouts to obtain a plurality of sample grating images; annotating whether the gratings in the sample grating images are abnormal and the abnormal types in the case of abnormalities to obtain annotated images, and constructing a training set based on the annotated images; training the third machine learning model based on the training set to obtain a to-be-verified third machine learning model; and verifying the to-be-verified third machine learning model based on the verification set to obtain a trained third machine learning model.

[0032] In one embodiment of the present application, the processor extracts a grating image including a grating structure from the optical waveguide layout, including: fragmenting the optical waveguide layout to obtain a plurality of grating images including a grating structure; or, obtaining user input, and extracting a grating image including a grating structure from the optical waveguide layout based on the user input, where the user input indicates the position coordinate range of the grating image to be extracted in the optical waveguide layout.

[0033] According to another aspect of the present application, there is provided a storage medium on which a computer program run by a processor is stored. When the computer program is run by the processor, the processor is caused to execute the above-mentioned method for inspecting an optical waveguide layout.

[0034] According to yet another aspect of the present application, there is provided a computer program which, when run by a processor, causes the processor to execute the above-mentioned method for inspecting an optical waveguide layout.

[0035] After obtaining the optical waveguide layout to be inspected, the method and device for inspecting an optical waveguide layout according to an embodiment of the present application extract a grating image including a grating structure from the optical waveguide layout, process the grating image through a trained machine learning model, output the grating information therein, and based on the grating information output by the machine learning model, it can be determined whether the grating image is abnormal. The method and device for inspecting an optical waveguide layout do not require manual intervention, can achieve efficient and automatic inspection of the optical waveguide layout; in addition, since the machine learning model is trained with a large amount of data, the inspection accuracy is high; and the machine learning model can process large-scale optical waveguide layout data, improving the comprehensiveness of the inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] By describing the embodiments of the present invention in more detail in conjunction with the drawings, the above and other objects, features and advantages of the present invention will become more obvious. The drawings are used to provide a further understanding of the embodiments of the present invention, and constitute a part of the specification, and are used to explain the present invention together with the embodiments of the present invention, and do not constitute a limitation to the present invention. In the drawings, the same reference numerals generally represent the same components or steps.

[0037] Figure 1 Shows an example layout diagram of a two-dimensional diffractive optical waveguide.

[0038] Figure 2 Shows an example layout diagram of a one-dimensional diffractive optical waveguide.

[0039] Figure 3 Shows an example of an optical waveguide layout.

[0040] Figure 4 Shows a schematic flowchart of a method for inspecting an optical waveguide layout according to an embodiment of the present application.

[0041] Figure 5 Shows an exemplary schematic diagram of fragmenting an optical waveguide layout and extracting a grating image.

[0042] Figure 6 Shows an example diagram of a grating image including one grating partition.

[0043] Figure 7 and Figure 8 Shows an example diagram of a grating image including multiple grating partitions.

[0044] Figure 9 and Figure 10 Shows an example diagram of a one-dimensional grating.

[0045] Figure 11 、 Figure 12 and Figure 13 Shows example diagrams of different grating shapes.

[0046] Figure 14 Shows an example diagram of abnormal grating splicing within a partition.

[0047] Figure 15 Shows an example diagram of missing gratings within a partition.

[0048] Figure 16 Shows an example diagram of abnormal grating splicing between partitions.

[0049] Figure 17 Shows an example diagram of normal grating splicing between partitions.

[0050] Figure 18 Shows an example diagram of abnormal grating splicing within a partition and the corresponding position coordinates.

[0051] Figure 19 Shows an example diagram of missing gratings within a partition and the corresponding position coordinates.

[0052] Figure 20 Shows an example diagram of abnormal grating splicing between partitions and the corresponding position coordinates.

[0053] Figure 21 An example diagram showing normal splicing of inter - zone gratings and corresponding position coordinates.

[0054] Figure 22 A schematic diagram showing training loss and validation loss.

[0055] Figure 23 A schematic diagram showing training accuracy and validation accuracy.

[0056] Figure 24 A schematic diagram showing the prediction result after model application.

[0057] Figure 25 A schematic diagram showing training loss and validation loss.

[0058] Figure 26 A schematic diagram showing training accuracy and validation accuracy.

[0059] Figure 27 A schematic structural block diagram showing an inspection device for an optical waveguide layout according to an embodiment of the present application. Detailed implementation manners

[0060] In order to make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments of the present invention. It should be understood that the present invention is not limited by the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

[0061] An AR optical waveguide is a key component for AR technology, which can superimpose virtual digital information on the real world and is increasingly applied in fields such as education, medical treatment, entertainment, industrial design, and tourism. A diffractive optical waveguide is a type of AR optical waveguide that uses the diffraction characteristics of gratings to transmit virtual images to the user's eyes.

[0062] A diffractive optical waveguide generally includes an input grating region and an output grating region, as Figure 1 shown. For a pure one - dimensional grating, a turning grating region is usually also required, as Figure 2 shown. The light emitted by the optical engine passes through the input grating, enters the planar waveguide, and undergoes total internal reflection propagation therein, and finally the output grating transmits the light to the human eye.

[0063] To provide optical efficiency characteristics such as the efficiency and uniformity of an optical waveguide, multiple different grating partitions are generally set in the grating region (coupling-in, turning, or coupling-out). The shape and size of the partitions can be freely set according to requirements, and the grating parameters of adjacent partitions are generally different (including grating period, duty cycle, tilt angle, depth, refractive index, coating, etc.).

[0064] In the actual fabrication process, the fabrication of diffractive optical waveguides usually adopts micro-nano processing technology. Through processes such as photolithography and etching, the shape and size of the grating structure in the optical waveguide are precisely controlled. Considering that GDS layout files are widely used in mask lithography, laser direct writing, and electron beam lithography (EBL), the grating structure of the optical waveguide also uses the GDS format to describe the grating information to facilitate file reading in subsequent processing stages. Figure 3 An example of an optical waveguide layout is shown. In this example, the coupling-in grating is a single partition, and the coupling-out grating is 996 partitions. Among them, adjacent partitions represent different grating parameters.

[0065] Whether the grating structure of the optical waveguide can be precisely processed will directly affect the optical effect of the optical waveguide product. Therefore, the accuracy of the GDS layout of the optical waveguide has become particularly important.

[0066] Previous optical waveguide layouts Figure 1 Generally, it is checked manually. The checking process is as follows: (1) Open the layout file; (2) Zoom in on the target checking area; (3) Manually judge the correctness of the layout; (4) Zoom out the layout, find the next target area, and repeat (2)(3)(4) for checking.

[0067] Using this checking method, it can achieve good checking speed and accuracy when the number of partitions is relatively small. However, when the number of partitions increases significantly, this manual checking method becomes very inefficient. Moreover, since only some partitions can be sampled due to the large number, there are risks of anomalies in the partitions that are not checked. At the same time, even for some checked partitions, there are risks of misdetection in manual checking.

[0068] Therefore, a fast, global, and automatic inspection method is needed to reasonably judge the correctness of the optical waveguide layout. Based on this, the present application provides an inspection scheme for the optical waveguide layout. The following is described in conjunction with Figures 4 to 27 to describe.

[0069] Figure 4 A schematic flowchart of an inspection method 400 for an optical waveguide layout according to an embodiment of the present application is shown. As Figure 4 shown, the inspection method 400 for the optical waveguide layout may include the following steps:

[0070] In step S410, an optical waveguide layout to be inspected is obtained, and a grating image including a grating structure is extracted from the optical waveguide layout. The grating image includes one grating partition or at least two grating partitions.

[0071] In step S420, the grating image is input into a trained machine learning model, and the grating information in the grating image is output by the machine learning model. The grating information includes at least one of the following: grating parameter information of each grating partition, grating type information of each grating partition, information on whether the grating in each grating partition is abnormal, and information on whether the grating between different grating partitions is abnormal.

[0072] Among them, the training of the machine learning model includes: obtaining a sample optical waveguide layout, performing fragmentation processing on the sample optical waveguide layout to obtain a plurality of sample grating images; annotating the grating information in the sample grating images to obtain an annotated image, and constructing a training set based on the annotated image; training the machine learning model based on the constructed training set to obtain a trained machine learning model.

[0073] In an embodiment of the present application, after obtaining the optical waveguide layout to be inspected, a grating image including a grating structure is extracted from the optical waveguide layout, and the trained machine learning model is used to process the grating image to output the grating information therein. Based on the grating information output by the machine learning model, it can be determined whether there is an abnormality in the grating image. The inspection method of the optical waveguide layout does not require manual intervention and can achieve efficient automatic inspection of the optical waveguide layout; in addition, since the machine learning model is trained with a large amount of data, the inspection accuracy is high; and the machine learning model can process large-scale optical waveguide layout data, improving the comprehensiveness of the inspection.

[0074] Specifically, when training the machine learning model, first, a sample optical waveguide layout is obtained. Since the amount of data of the optical waveguide layout is very large, it is usually necessary to perform fragmentation processing on it first to obtain a sample grating image including its microscopic structure - the grating structure. Figure 5 Shows an exemplary schematic diagram of performing fragmentation processing on the optical waveguide layout to extract the grating image. As Figure 5 shown, by performing fragmentation processing on a certain position of the coupled-in grating, a GDS fragment is obtained, and the GDS fragment can be captured through an application programming interface (API) to obtain a grating image. In this way, a large number of grating images can be obtained as sample grating images.

[0075] After obtaining the sample grating images, the grating information in the sample grating images can be annotated to obtain an annotated image for constructing a training set. Among them, depending on the inspection desired for the layout, different grating information can be selected for annotation. The grating information to be annotated can train the machine learning model to output the corresponding grating information for the grating image.

[0076] In the embodiments of the present application, depending on the layout position of the extracted grating image, the extracted grating image may include one grating partition (i.e., at this time, the grating image only includes the grating information within one grating partition), as Figure 6 shown; it may also include more than one grating partition (i.e., at this time, the grating image not only includes the grating information within different grating partitions respectively, but also includes the grating information between different grating partitions, such as grating splicing information, etc.), as Figure 7 and Figure 8 shown, where Figure 7 the grating image in Figure 8 includes 4 partitions (partition A, partition B, partition C, and partition D respectively),

[0077] Therefore, in the embodiments of the present application, the grating information may include at least one of the following: the grating parameter information of each grating partition, the grating type information of each grating partition, the information on whether the grating within each grating partition is abnormal, and the information on whether the grating between different grating partitions is abnormal.

[0078] Among them, the grating parameter information may include at least one of the following: grating period, grating angle, grating line width, grating duty cycle, grating depth, and grating refractive index. After annotating the grating parameter information for the sample grating image, the machine learning model trained with the annotated image can output the grating parameter information for the newly acquired grating image.

[0079] The grating parameter information output by the machine learning model can be compared with the preset grating parameter information to determine whether the newly acquired grating image is abnormal. For example, when the grating parameter information output by the machine learning model is inconsistent with the preset grating parameter information, it indicates that the grating image may be abnormal. At this time, a first prompt message can be output to prompt that the grating parameters of the current grating image are abnormal. Further, the specific parameter type of the abnormality can also be prompted, such as abnormal grating angle, abnormal grating line width, etc. Further, the specific abnormal situation of the parameter can also be prompted, such as the grating angle is smaller than the preset value (even the specific value smaller than the preset value can be prompted). Further, when the grating parameter information output by the machine learning model is inconsistent with the preset grating parameter information, the optical waveguide layout can also be modified based on the preset grating parameter information. Specifically, the grating with abnormalities in the current grating image can be modified to make its parameters consistent with the preset parameters. For example, when the grating line width is smaller than the preset line width, the layout can be automatically modified to make the grating line width consistent with the preset value.

[0080] The grating type information may include the grating dimension and / or the grating shape. Among them, the grating dimension includes, for example, one-dimensional gratings and two-dimensional gratings. For exampleFigure 9 the one-dimensional grating shown and Figure 10 the left-tilted one-dimensional grating shown. The grating shape can include a parallelogram, a single diamond, a double diamond, etc., as Figure 11 , Figure 12 and Figure 13 shown respectively. After labeling the grating type information for the sample grating image, the machine learning model trained with the labeled image can output the grating type information for the newly acquired grating image.

[0081] The grating type information output by the machine learning model can be compared with the preset grating type information to determine whether there is an abnormality in the newly acquired grating image. For example, when the grating type information output by the machine learning model is inconsistent with the preset grating type information, it indicates that the grating image may be abnormal. At this time, a second prompt message can be output to prompt that there is an abnormality in the grating type of the current grating image. Further, the specific type of abnormality can also be prompted, such as grating dimension abnormality, grating shape abnormality, etc. Further, the specific abnormal situation can also be prompted, such as the grating dimension is a one-dimensional grating, but the preset grating type is a two-dimensional grating, etc.

[0082] The information on whether the grating is abnormal in each grating partition includes the grating normal information and the grating abnormal information in each grating partition, where the grating abnormal information includes the abnormal splicing of the gratings within the partition and / or the missing of the gratings within the partition. Figure 14 Fig. shows an example diagram of the abnormal splicing of the gratings within the partition. Figure 15 Fig. shows an example diagram of the missing of the gratings within the partition. After labeling the information on whether the grating is abnormal in the grating partition for the sample grating image, the machine learning model trained with the labeled image can output the information on whether the grating is abnormal in the grating partition for the newly acquired grating image.

[0083] The information on whether the grating is abnormal between different grating partitions includes the grating normal information and the grating abnormal information between different grating partitions, where the grating abnormal information includes the abnormal splicing of the gratings between the partitions. Figure 16 Fig. shows an example diagram of the abnormal splicing of the gratings between the partitions. Figure 17 Fig. shows an example diagram of the normal splicing of the gratings between the partitions. After labeling the information on whether the grating is abnormal between the grating partitions for the sample grating image, the machine learning model trained with the labeled image can output the information on whether the grating is abnormal between the grating partitions for the newly acquired grating image.

[0084] Further, in addition to the above-mentioned graphical information on the grating abnormality within the partition and between the partitions, the information on the position coordinates of the abnormality can also be provided at the same time. This position coordinate information is also output as the grating abnormal information to better prompt the user of the specific abnormal situation. Figure 18 Fig. shows an example diagram of the abnormal splicing of the gratings within the partition and the corresponding position coordinates.Figure 19 An example diagram showing the absence of gratings within a partition and the corresponding position coordinates. Figure 20 An example diagram showing the abnormal splicing of gratings between partitions and the corresponding position coordinates. Figure 21 An example diagram showing the normal splicing of gratings between partitions and the corresponding position coordinates (for cases without abnormalities, the diagram information and coordinate information may not be displayed).

[0085] In an embodiment of the present application, the above-mentioned annotations of grating parameter information, grating type information, and information on whether the grating is abnormal can be marked on the same sample grating image. In this way, a large number of annotated sample grating images can train a machine learning model, which can output grating parameter information, grating type information, and information on whether the grating is abnormal for newly acquired grating images. Alternatively, the above-mentioned annotations of grating parameter information, grating type information, and information on whether the grating is abnormal can be marked on different sample grating images. For example, some sample grating images only mark the grating parameter information, and such sample grating images are used to train a machine learning model that only outputs the grating parameter information for newly acquired grating images. Similarly, some sample grating images only mark the grating type information, and such sample grating images are used to train a machine learning model that only outputs the grating type information for newly acquired grating images. Similarly, some sample grating images only mark the information on whether the grating is abnormal, and such sample grating images are used to train a machine learning model that only outputs the information on whether the grating is abnormal for newly acquired grating images. Here, taking the example of separately annotating and separately training different machine learning models for illustration, it should be understood that the same machine learning model can also be trained to have the ability to output multiple grating information.

[0086] In one embodiment, the machine learning model used in step S420 may include a first machine learning model, a second machine learning model, and a third machine learning model, where: the first machine learning model includes a regressor for outputting grating parameter information; the second machine learning model includes a first classifier for outputting grating type information; the third machine learning model includes a second classifier for outputting information on whether the grating is abnormal. In this embodiment, different machine learning models can be trained to output different grating information.

[0087] Among them, the training of the first machine learning model may include: annotating the grating parameters in the sample grating image to obtain the annotated image, constructing a training set and a validation set based on the annotated image; training the first machine learning model based on the training set to obtain the first machine learning model to be verified; verifying the first machine learning model to be verified based on the validation set to obtain the trained first machine learning model. Among them, it is also possible to not construct a validation set, only construct a training set, and train the first machine learning model based on the training set to obtain the trained first machine learning model.

[0088] Specifically, the entire training process can be as follows: Extract the grating arrangements at different positions from the full version map and perform image processing to obtain the sample grating image, which can be automatically implemented by calling the API; Specify the image paths of the training set and the validation set, and both the training set and the validation set need to contain the corresponding sample grating images with various different grating parameters; Normalize the images in the training set and the validation set (for example, scale the pixel values to between 0 and 1); Generate the data streams of the training set and the validation set from the specified directory; Construct a convolutional neural network, including convolutional layers, pooling layers, fully connected layers, and Dropout layers, to extract image features. Among them, different filters are used in the convolutional layers for feature extraction; The pooling layer reduces the spatial size of the feature map through downsampling; The Flatten layer flattens the two-dimensional feature map into one dimension; The fully connected layer connects features at different levels through a neural network; The Dropout layer randomly discards a certain proportion of neurons to prevent overfitting; Compile the model, specify the loss function, optimizer, and evaluation metrics, and compile the model into a trainable state; Add EarlyStopping, use the Early Stopping callback function to monitor the loss, and when the loss stagnates, stop training and restore the model weights with the best validation set performance; Train the model, input the training data, validation data, and some training parameters, such as batch size, number of training steps, etc.; Save the trained model; Use the validation set to check the accuracy of the trained model. As Figure 22 shows the training loss and the validation loss, Figure 23 shows the training accuracy and the validation accuracy. It can be seen that the training loss and the validation loss get closer and closer to 0 as the number of training batches increases, and the training accuracy and the validation accuracy get closer and closer to 1 as the number of training batches increases, reflecting the good performance of the trained model. Figure 24 shows the prediction results after the model is applied, and it can be seen that the model can accurately predict the grating parameters.

[0089] In an embodiment of the present application, the training of the second machine learning model may include: obtaining sample optical waveguide layouts with different grating dimensions to obtain a plurality of sample grating images; annotating the grating types in the sample grating images to obtain annotated images, and constructing a training set and a validation set based on the annotated images; training the second machine learning model based on the training set to obtain a second machine learning model to be verified; verifying the second machine learning model to be verified based on the validation set to obtain a trained second machine learning model. Among them, it is also possible to not construct a validation set, only construct a training set, and train the second machine learning model based on the training set to obtain a trained second machine learning model.

[0090] Specifically, the entire training process may be as follows: Extract the grating arrangements at different positions from the full version layout and perform image processing to obtain sample grating images, and this process can be automatically implemented by calling the API; Specify the image paths of the training set and the validation set, and both the training set and the validation set need to contain the corresponding sample grating images of various different grating types; Normalize the images in the training set and the validation set (for example, scale the pixel values to between 0 and 1); Generate data streams for the training set and the validation set from the specified directory; Construct a convolutional neural network, including a convolutional layer, a pooling layer, a fully connected layer, and a Dropout layer, to extract image features. Among them, the convolutional layer uses different filters for feature extraction; The pooling layer reduces the spatial size of the feature map through downsampling; The Flatten layer flattens the two-dimensional feature map into one dimension; The fully connected layer connects features at different levels through a neural network; The Dropout layer randomly discards a certain proportion of neurons to prevent overfitting; Compile the model, specify the loss function, optimizer, and evaluation metrics, and compile the model into a trainable state; Add EarlyStopping, use the Early Stopping callback function to monitor the loss, and when the loss stagnates, stop training and restore the model weights with the best validation set performance; Train the model, input the training data, validation data, and some training parameters, such as batch size, number of training steps, etc.; Save the trained model; Use the validation set to check the accuracy of the trained model.

[0091] In an embodiment of the present application, the training of the third machine learning model may include: obtaining a normal sample optical waveguide layout and an abnormal sample optical waveguide layout to obtain a plurality of sample grating images; annotating whether the gratings in the sample grating images are abnormal and the abnormal types in the case of abnormalities to obtain an annotated image, and constructing a training set based on the annotated image; training the third machine learning model based on the training set to obtain a third machine learning model to be verified; verifying the third machine learning model to be verified based on a validation set to obtain a trained third machine learning model. Among them, it is also possible to not construct a validation set, only construct a training set, and train the third machine learning model based on the training set to obtain a trained third machine learning model.

[0092] Specifically, the entire training process may be as follows: Extract the grating arrangements at different positions from the full version layout and perform image processing, and this process can be automatically implemented by calling the API. The normal and abnormal splicings need to be saved in different folders. For example, according to different abnormal types, the pictures are respectively saved in four folders: "normal splicing", "intra-partition splicing abnormality", "grating missing", and "inter-partition splicing abnormality", corresponding to four common situations: normal splicing, intra-partition splicing abnormality, grating missing, and inter-partition splicing abnormality; specify the image paths of the training set and the validation set, and both the training set and the validation set need to include the folders of normal layouts and abnormal layouts and the corresponding pictures; perform normalization processing on the images in the training set and the validation set (scale the pixel values to between 0 and 1); generate data streams for the training set and the validation set from the specified directory; construct a convolutional neural network, including a convolutional layer, a pooling layer, a fully connected layer, and a Dropout layer, to extract image features. Among them, the convolutional layer uses different filters for feature extraction; the pooling layer reduces the spatial size of the feature map through downsampling; the Flatten layer flattens the two-dimensional feature map into one dimension; the fully connected layer connects features at different levels through a neural network; the Dropout layer randomly discards a certain proportion of neurons to prevent overfitting; compile the model, specify the loss function, optimizer, and evaluation metrics, and compile the model into a trainable state; add Early Stopping, use the Early Stopping callback function to monitor the loss, and when the loss stagnates, stop training and restore the model weights with the best validation set performance; train the model, input the training data, validation data, and some training parameters, such as batch size, number of training steps, etc.; save the trained model; use the validation set to check the accuracy of the trained model. As Figure 25 shows the training loss and the validation loss, Figure 26 shows the training accuracy and the validation accuracy. It can be seen that the training loss and the validation loss get closer and closer to 0 as the training batches increase, and the training accuracy and the validation accuracy get closer and closer to 1 as the training batches increase, reflecting the good performance of the trained model.

[0093] After obtaining the above-trained machine learning model, obtain the optical waveguide layout to be inspected, extract the grating image containing the grating structure from it, and then the trained machine learning model can be used to process the grating image and output the above grating information, such as the grating parameter information of each grating partition, the grating type information of each grating partition, the information on whether the grating in each grating partition is abnormal, and the information on whether the grating between different grating partitions is abnormal.

[0094] Among them, in one embodiment, extracting the grating image containing the grating structure from the optical waveguide layout may include: fragmenting the optical waveguide layout to obtain multiple grating images containing the grating structure. In this embodiment, the grating images at different positions of the entire layout can be obtained by fragmenting the optical waveguide layout. In another embodiment, extracting the grating image containing the grating structure from the optical waveguide layout may include: obtaining user input, and extracting the grating image containing the grating structure from the optical waveguide layout based on the user input, where the user input indicates the position coordinate range of the grating image to be extracted in the optical waveguide layout. In this embodiment, the grating image at the position specified by the user can be obtained, and the grating image can be extracted according to the user's needs for layout inspection. Generally, the layout file can be opened through the graphic design system software, and after magnifying to obtain the grating image at a certain position, the grating image can be saved by taking a screenshot for subsequent inspection.

[0095] The inspection method 400 of the optical waveguide layout according to the embodiment of the present application is described in detail above. Based on the above description, after obtaining the optical waveguide layout to be inspected, the inspection method 400 of the optical waveguide layout according to the embodiment of the present application extracts the grating image containing the grating structure from the optical waveguide layout, processes the grating image through the trained machine learning model, and outputs the grating information therein. Based on the grating information output by the machine learning model, it can be determined whether the grating image is abnormal. This inspection method of the optical waveguide layout does not require manual intervention and can achieve efficient and automatic inspection of the optical waveguide layout; in addition, since the machine learning model is trained with a large amount of data, the inspection accuracy is high; and the machine learning model can process large-scale optical waveguide layout data, improving the comprehensiveness of the inspection.

[0096] The following will be combined with Figure 27 Describe the inspection device 2700 of the optical waveguide layout provided according to another aspect of the present application. As Figure 27As shown in the figure, the inspection device 2700 for the optical waveguide layout includes a memory 2710 and a processor 2720. A computer program is stored on the memory 2710 and run by the processor 2720. When the computer program is run by the processor 2720, it causes the processor 2720 to execute the above-described inspection method 400 for the optical waveguide layout according to the embodiments of the present application. The method 400 has been described in detail above. Those skilled in the art can understand the structure and operation of the inspection device 2700 for the optical waveguide layout in combination with the foregoing description. For the sake of brevity, the details will not be repeated here, and only some main operations will be described.

[0097] In an embodiment of the present application, when the above computer program is run by the processor 2720, it causes the processor 2720 to perform the following steps: obtaining an optical waveguide layout to be inspected, extracting a grating image including a grating structure from the optical waveguide layout, where the grating image includes one grating partition or at least two grating partitions; inputting the grating image into a trained machine learning model, and outputting grating information in the grating image by the machine learning model, where the grating information includes at least one of the following: grating parameter information of each grating partition, grating type information of each grating partition, information on whether the grating in each grating partition is abnormal, and information on whether the grating between different grating partitions is abnormal; wherein the training of the machine learning model includes: obtaining a sample optical waveguide layout, performing fragmentation processing on the sample optical waveguide layout to obtain a plurality of sample grating images; annotating the grating information in the sample grating images to obtain an annotated image, and constructing a training set based on the annotated image; and training the machine learning model based on the constructed training set to obtain a trained machine learning model.

[0098] In an embodiment of the present application, the grating parameter information includes at least one of the following: grating period, grating angle, grating line width, grating duty cycle, grating depth, and grating refractive index.

[0099] In an embodiment of the present application, when the above computer program is run by the processor 2720, it further causes the processor 2720 to perform the following steps: comparing the grating parameter information output by the machine learning model with preset grating parameter information, and when the grating parameter information output by the machine learning model is inconsistent with the preset grating parameter information, outputting a first prompt message.

[0100] In an embodiment of the present application, when the above computer program is run by the processor 2720, it further causes the processor 2720 to perform the following steps: when the grating parameter information output by the machine learning model is inconsistent with the preset grating parameter information, modifying the optical waveguide layout based on the preset grating parameter information.

[0101] In an embodiment of the present application, the grating type information includes grating dimension and / or grating shape.

[0102] In an embodiment of the present application, when the above computer program is run by the processor 2720, it further causes the processor 2720 to perform the following steps: comparing the raster type information output by the machine learning model with the preset raster type information, and when the raster type information output by the machine learning model is inconsistent with the preset raster type information, outputting a second prompt message.

[0103] In an embodiment of the present application, the information on whether the gratings in each grating partition are abnormal includes the grating normal information and grating abnormal information in each grating partition, where the grating abnormal information includes abnormal grating splicing within the partition and / or missing gratings within the partition.

[0104] In an embodiment of the present application, the information on whether the gratings between different grating partitions are abnormal includes the grating normal information and grating abnormal information between different grating partitions, where the grating abnormal information includes abnormal grating splicing between partitions.

[0105] In an embodiment of the present application, the grating abnormal information further includes information indicating the coordinates of the location of the abnormality.

[0106] In an embodiment of the present application, the machine learning model includes a first machine learning model, a second machine learning model, and a third machine learning model, where: the first machine learning model includes a regressor for outputting grating parameter information; the second machine learning model includes a first classifier for outputting grating type information; the third machine learning model includes a second classifier for outputting information on whether the gratings are abnormal.

[0107] In an embodiment of the present application, the training of the first machine learning model includes: annotating the grating parameters in the sample grating image to obtain an annotated image, constructing a training set and a validation set based on the annotated image; training the first machine learning model based on the training set to obtain a first machine learning model to be verified; and verifying the first machine learning model to be verified based on the validation set to obtain a trained first machine learning model.

[0108] In an embodiment of the present application, the training of the second machine learning model includes: obtaining sample optical waveguide layouts of different grating dimensions to obtain a plurality of sample grating images; annotating the grating types in the sample grating images to obtain an annotated image, constructing a training set and a validation set based on the annotated image; training the second machine learning model based on the training set to obtain a second machine learning model to be verified; and verifying the second machine learning model to be verified based on the validation set to obtain a trained second machine learning model.

[0109] In an embodiment of the present application, the training of the third machine learning model includes: obtaining a normal sample optical waveguide layout and an abnormal sample optical waveguide layout to obtain a plurality of sample grating images; annotating whether the gratings in the sample grating images are abnormal and the abnormal types in the case of abnormalities to obtain an annotated image, and constructing a training set based on the annotated image; training the third machine learning model based on the training set to obtain a third machine learning model to be verified; and verifying the third machine learning model to be verified based on a verification set to obtain a trained third machine learning model.

[0110] In an embodiment of the present application, the processor 2720 extracts a grating image including a grating structure from the optical waveguide layout, including: fragmenting the optical waveguide layout to obtain a plurality of grating images including a grating structure; or, obtaining user input and extracting a grating image including a grating structure from the optical waveguide layout based on the user input, where the user input indicates the position coordinate range of the grating image to be extracted in the optical waveguide layout.

[0111] Based on the above description, after obtaining the optical waveguide layout to be inspected, the inspection device 2700 of the optical waveguide layout according to the embodiment of the present application extracts a grating image including a grating structure from the optical waveguide layout, processes the grating image through a trained machine learning model, outputs the grating information therein, and based on the grating information output by the machine learning model, it can be determined whether the grating image is abnormal. The inspection device of the optical waveguide layout can realize efficient automatic inspection of the optical waveguide layout without manual intervention; in addition, since the machine learning model is trained with a large amount of data, the inspection accuracy is high; and the machine learning model can process large-scale optical waveguide layout data, improving the comprehensiveness of the inspection.

[0112] In addition, according to an embodiment of the present application, there is also provided a storage medium on which program instructions are stored, and when the program instructions are run by a computer or a processor, they are used to execute the corresponding steps of the inspection method of the optical waveguide layout according to the embodiment of the present application. The storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0113] In addition, according to an embodiment of the present application, there is also provided a computer program, which can be stored on a storage medium in the cloud or locally. When the computer program is run by a computer or a processor, it is used to execute the corresponding steps of the inspection method of the optical waveguide layout according to the embodiment of the present application.

[0114] Based on the above description, after obtaining the optical waveguide layout to be inspected, the inspection method and device for the optical waveguide layout according to the embodiments of the present application extract a grating image including a grating structure from the optical waveguide layout, process the grating image through a trained machine learning model, output the grating information therein, and based on the grating information output by the machine learning model, it can be determined whether there is an abnormality in the grating image. The inspection method and device for the optical waveguide layout do not require manual intervention and can achieve efficient and automatic inspection of the optical waveguide layout; in addition, since the machine learning model is trained with a large amount of data, the inspection accuracy is high; and the machine learning model can process large-scale optical waveguide layout data, improving the comprehensiveness of the inspection.

[0115] Although example embodiments have been described herein with reference to the drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Those of ordinary skill in the art can make various changes and modifications therein without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as claimed in the appended claims.

[0116] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraint conditions of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

[0117] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0118] In the specification provided herein, a large number of specific details are set forth. However, it can be understood that the embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures, and technologies have not been shown in detail so as not to obscure the understanding of this specification.

[0119] Similarly, it should be understood that, for the sake of streamlining the present invention and aiding in the understanding of one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the methods of the present invention should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, the inventive point lies in that the corresponding technical problems can be solved with features less than all the features of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into the detailed description, where each claim by itself serves as a separate embodiment of the present invention.

[0120] Those skilled in the art will appreciate that, except where features are mutually exclusive, any combination can be used to combine all the features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all the processes or units of any method or device so disclosed. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) can be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0121] Furthermore, those skilled in the art will be able to understand that, although some embodiments herein include certain features included in other embodiments but not others, the combination of features of different embodiments means that it is within the scope of the present invention and forms different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0122] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some of the modules in the article analysis device according to the embodiments of the present invention. The present invention can also be implemented as a device program (such as a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0123] It should be noted that the above embodiments are illustrative of the present invention rather than restrictive of the present invention, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim listing several devices, several of these devices may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words may be interpreted as names.

[0124] The above are only specific embodiments of the present invention or descriptions of specific embodiments. The protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of changes or substitutions within the technical scope disclosed by the present invention, and all such changes or substitutions should be covered within the protection scope of the present invention. The protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for inspecting an optical waveguide layout, characterized in that, The method includes: Obtain a layout of an optical waveguide to be inspected, and extract a grating image containing a grating structure from the layout of the optical waveguide. The grating image includes one grating partition or at least two grating partitions; Input the grating image into a trained machine learning model, and the machine learning model outputs grating information in the grating image. The grating information includes at least one of the following: grating parameter information of each grating partition, grating type information of each grating partition, information on whether the gratings in each grating partition are abnormal, and information on whether the gratings between different grating partitions are abnormal; Among them, the training of the machine learning model includes: Obtain a sample layout of an optical waveguide, perform fragmentation processing on the sample layout of the optical waveguide to obtain a plurality of sample grating images; Annotate the grating information in the sample grating images to obtain an annotated image, and construct a training set based on the annotated image; Train the machine learning model based on the constructed training set to obtain a trained machine learning model.

2. The inspection method according to claim 1, characterized in that, The grating parameter information includes at least one of the following: grating period, grating angle, grating line width, grating duty cycle, grating depth, and grating refractive index.

3. The inspection method according to claim 1 or 2, characterized in that, The inspection method further includes: Compare the grating parameter information output by the machine learning model with preset grating parameter information. When the grating parameter information output by the machine learning model is inconsistent with the preset grating parameter information, output a first prompt message.

4. The inspection method according to claim 3, characterized in that, The inspection method further includes: When the grating parameter information output by the machine learning model is inconsistent with the preset grating parameter information, modify the layout of the optical waveguide based on the preset grating parameter information.

5. The inspection method according to claim 1, wherein The grating type information includes grating dimension and / or grating shape.

6. The inspection method according to claim 1 or 5, characterized in that The inspection method further includes: Compare the grating type information output by the machine learning model with preset grating type information. When the grating type information output by the machine learning model is inconsistent with the preset grating type information, output a second prompt message.

7. The inspection method according to claim 1, wherein The information on whether the gratings in each grating partition are abnormal includes grating normal information and grating abnormal information in each grating partition. Among them, the grating abnormal information includes abnormal splicing of gratings within the partition and / or missing gratings within the partition.

8. The inspection method according to claim 1, characterized in that, The information on whether the gratings between different grating partitions are abnormal includes grating normal information and grating abnormal information between different grating partitions. Among them, the grating abnormal information includes abnormal splicing of gratings between partitions.

9. The inspection method according to claim 7 or 8, characterized in that, The grating abnormal information further includes information indicating the coordinates of the abnormal location.

10. The inspection method according to claim 1, characterized in that, The machine learning model includes a first machine learning model, a second machine learning model, and a third machine learning model, where: The first machine learning model includes a regressor, and the regressor is used to output the grating parameter information; The second machine learning model includes a first classifier, and the first classifier is used to output the grating type information; The third machine learning model includes a second classifier, and the second classifier is used to output the information on whether the gratings are abnormal.

11. The inspection method according to claim 10, characterized in that, The training of the first machine learning model includes: Label the grating parameters in the sample grating image to obtain the labeled image, and construct a training set and a validation set based on the labeled image; Train the first machine learning model based on the training set to obtain the first machine learning model to be verified; Verify the first machine learning model to be verified based on the validation set to obtain the trained first machine learning model.

12. The inspection method according to claim 10, characterized in that, The training of the second machine learning model includes: Obtain sample optical waveguide layouts with different grating dimensions to obtain multiple sample grating images; Label the grating types in the sample grating images to obtain the labeled images, and construct a training set and a validation set based on the labeled images; Train the second machine learning model based on the training set to obtain the second machine learning model to be verified; Verify the second machine learning model to be verified based on the validation set to obtain the trained second machine learning model.

13. The inspection method according to claim 10, characterized in that, The training of the third machine learning model includes: Obtain normal sample optical waveguide layouts and abnormal sample optical waveguide layouts to obtain multiple sample grating images; Label whether the gratings in the sample grating images are abnormal and the abnormal types in the case of abnormality to obtain the labeled images, and construct a training set based on the labeled images; Train the third machine learning model based on the training set to obtain the third machine learning model to be verified; Verify the third machine learning model to be verified based on the validation set to obtain the trained third machine learning model.

14. The inspection method according to claim 1, wherein The extraction of the grating image including the grating structure from the optical waveguide layout includes: Fragment the optical waveguide layout to obtain multiple grating images including the grating structure; or, obtain user input, and extract the grating image including the grating structure from the optical waveguide layout based on the user input, where the user input indicates the position coordinate range of the grating image to be extracted in the optical waveguide layout.

15. The inspection method according to claim 1, characterized in that, The extraction operation of the grating image is based on the graphic design system software.

16. An inspection device for an optical waveguide layout, characterized in that, The device includes a memory and a processor, and a computer program run by the processor is stored on the memory. When the computer program is run by the processor, the processor performs the following steps: Obtain the optical waveguide layout to be inspected, and extract the grating image including the grating structure from the optical waveguide layout. The grating image includes one grating partition or at least two grating partitions; Input the grating image into the trained machine learning model, and the machine learning model outputs the grating information in the grating image. The grating information includes at least one of the following: the grating parameter information of each grating partition, the grating type information of each grating partition, the information on whether the grating in each grating partition is abnormal, and the information on whether the gratings between different grating partitions are abnormal; Among them, the training of the machine learning model includes: Obtain the sample optical waveguide layout, and fragment the sample optical waveguide layout to obtain multiple sample grating images; Annotate the grating information in the sample grating image to obtain an annotated image, and construct a training set based on the annotated image; Train the machine learning model based on the constructed training set to obtain a trained machine learning model.

17. The inspection device according to claim 16, characterized in that, The grating parameter information includes at least one of the following: grating period, grating angle, grating line width, grating duty cycle, grating depth, and grating refractive index.

18. The inspection device according to claim 16 or 17, characterized in that, When the computer program is run by the processor, it also causes the processor to perform the following steps: Compare the grating parameter information output by the machine learning model with the preset grating parameter information, and when the grating parameter information output by the machine learning model is inconsistent with the preset grating parameter information, output a first prompt message.

19. The inspection device according to claim 18, characterized in that, When the computer program is run by the processor, it also causes the processor to perform the following steps: When the grating parameter information output by the machine learning model is inconsistent with the preset grating parameter information, modify the optical waveguide layout based on the preset grating parameter information.

20. The inspection device according to claim 16, characterized in that, The grating type information includes grating dimension and / or grating shape.

21. The inspection device according to claim 16 or 20, characterized in that, When the computer program is run by the processor, it also causes the processor to perform the following steps: Compare the grating type information output by the machine learning model with the preset grating type information, and when the grating type information output by the machine learning model is inconsistent with the preset grating type information, output a second prompt message.

22. The inspection device according to claim 16, characterized in that, The information on whether the grating in each grating partition is abnormal includes the normal information and abnormal information of the grating in each grating partition, where the abnormal information of the grating includes abnormal splicing of the grating within the partition and / or missing of the grating within the partition.

23. The inspection device according to claim 16, characterized in that, The information on whether the grating is abnormal between different grating partitions includes the normal information and abnormal information of the grating between different grating partitions, where the abnormal information of the grating includes abnormal splicing of the grating between partitions.

24. The inspection device according to claim 22 or 23, characterized in that, The abnormal information of the grating also includes information indicating the coordinates of the location of the abnormality.

25. The inspection device according to claim 16, characterized in that, The machine learning model includes a first machine learning model, a second machine learning model, and a third machine learning model, where: The first machine learning model includes a regressor, and the regressor is used to output the grating parameter information; The second machine learning model includes a first classifier, and the first classifier is used to output the grating type information; The third machine learning model includes a second classifier, and the second classifier is used to output the information on whether the grating is abnormal.

26. The inspection device according to claim 25, characterized in that, The training of the first machine learning model includes: Annotate the grating parameters in the sample grating image to obtain an annotated image, and construct a training set and a validation set based on the annotated image; Train the first machine learning model based on the training set to obtain a first machine learning model to be verified; Verify the first machine learning model to be verified based on the validation set to obtain a trained first machine learning model.

27. The inspection device according to claim 25, characterized in that, The training of the second machine learning model includes: Obtain sample optical waveguide layouts with different grating dimensions to obtain a plurality of sample grating images; Label the grating types in the sample grating image to obtain the labeled image, and construct a training set and a validation set based on the labeled image; Train the second machine learning model based on the training set to obtain a second machine learning model to be verified; Verify the second machine learning model to be verified based on the validation set to obtain a trained second machine learning model.

28. The inspection device according to claim 25, characterized in that, The training of the third machine learning model includes: Obtain a normal sample optical waveguide layout and an abnormal sample optical waveguide layout to obtain a plurality of sample grating images; Label whether the grating in the sample grating image is abnormal and the abnormal type in the case of abnormality to obtain the labeled image, and construct a training set based on the labeled image; Train the third machine learning model based on the training set to obtain a third machine learning model to be verified; Verify the third machine learning model to be verified based on the validation set to obtain a trained third machine learning model.

29. The inspection device according to claim 16, characterized in that, The processor extracts a grating image including a grating structure from the optical waveguide layout, including: Fragment the optical waveguide layout to obtain a plurality of grating images including a grating structure; or, obtain user input, and extract a grating image including a grating structure from the optical waveguide layout based on the user input, where the user input indicates the position coordinate range of the grating image to be extracted in the optical waveguide layout.

30. A storage medium, characterized in that, A computer program run by a processor is stored on the storage medium. When the computer program is run by the processor, the processor is caused to execute the optical waveguide layout inspection method according to any one of claims 1-15.

31. A computer program, characterized in that, When the computer program is run by the processor, the processor is caused to execute the optical waveguide layout inspection method according to any one of claims 1-15.

Images