Crystal visual identification processing method

Abstract

The embodiment of the invention provides a crystal visual identification processing method. The method comprises the following steps: acquiring a to-be-identified image related to one or more crystals; image recognition is carried out on the to-be-recognized image, the recognition result of one or more crystals is determined, and the recognition result comprises at least one of size data, position data and defect data of one or more crystals; and determining a processing scheme of the one or more crystals based on the identification result.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a crystal visual recognition processing method. Background Art

[0002] Due to its unique physical, chemical and electrical properties, crystals are widely used in optical devices, electronic devices, medical equipment and other fields. During the growth and use of crystals, it is often necessary to manually judge the size, position, defects and other data of the crystals to determine the further processing plan of the crystals. However, manual judgment may result in errors, low efficiency and high cost.

[0003] Based on this, we hope to propose a crystal visual recognition processing method, which can determine the crystal processing plan through visual recognition, improve the efficiency of crystal processing, and reduce costs. Summary of the invention

[0004] One or more embodiments of the present specification provide a crystal visual recognition processing method, which includes: acquiring an image to be recognized related to one or more crystals; performing image recognition on the image to be recognized to determine the recognition results of the one or more crystals, wherein the recognition results include at least one of the size data, position data, and defect data of the one or more crystals; and determining a processing plan for the one or more crystals based on the recognition results.

[0005] In some embodiments, acquiring the image to be identified related to one or more crystals includes: acquiring the image to be identified of a tray containing the one or more crystals; the position data includes first position data of the one or more crystals on the tray; performing image recognition on the image to be identified and determining the identification result of the one or more crystals includes: performing image recognition on the image to be identified and determining the size data and the first position data of the one or more crystals; the processing plan includes a first adjustment plan for adjusting the one or more crystals placed on the tray; and determining the processing plan for the one or more crystals based on the recognition result includes: determining the first adjustment plan based on the configuration target of the tray, the size data of the one or more crystals and the first position data.

[0006] In some embodiments, the defect data includes first defect data characterizing defects of the crystal itself, and performing image recognition on the image to be identified to determine the identification results of the one or more crystals includes: performing image recognition on the image to be identified to determine the size data and the first defect data of the one or more crystals; the processing plan includes a cutting plan for cutting the one or more crystals, and determining the processing plan for the one or more crystals based on the recognition result includes: obtaining a first cutting target for the one or more crystals; for each crystal, determining the cutting plan for the crystal based on the size data, the first defect data and the first cutting target of the crystal.

[0007] In some embodiments, determining the cutting scheme of the crystal based on the size data of the crystal, the first defect data and the first cutting target includes: determining multiple candidate cutting schemes based on the size data of the crystal, the first defect data and the first cutting target; for each candidate cutting scheme, determining the unit score of each cut crystal unit based on the candidate cutting scheme, the size data of the crystal and the first defect data; determining the scheme score of the candidate cutting scheme based on the unit score of each crystal unit; and determining the cutting scheme of the crystal based on the scheme score of each candidate cutting scheme.

[0008] In some embodiments, determining the scheme score of the candidate cutting scheme based on the unit scores of each crystal unit includes: obtaining a second cutting target for cutting the crystal; determining the quantity weight and quality weight of the crystal unit based on the second cutting target; and determining the scheme score of the candidate cutting scheme based on the quantity weight, the quality weight, the quantity of the crystal unit and the unit score.

[0009] In some embodiments, the acquiring of the image to be identified related to one or more crystals includes: acquiring the image to be identified related to a crystal array composed of multiple crystal units, wherein at least two adjacent crystal units in the crystal array have a reflective structure; the position data includes second position data characterizing the relative positions of the multiple crystal units in the crystal array, the defect data includes second defect data and third defect data, the second defect data is data characterizing the defects of the crystal units themselves in the crystal array, and the third defect data is data characterizing the defects of the crystal array setting, the performing of image recognition on the image to be identified and determining the identification results of the one or more crystals includes: performing image recognition on the image to be identified and determining the second position data, the second defect data and the third defect data; the processing scheme includes a second adjustment scheme for adjusting the crystal units and / or the reflective structure in the crystal array, and the determining of the processing scheme for the one or more crystals based on the recognition result includes: determining the second adjustment scheme based on the second position data, the second defect data and the third defect data.

[0010] In some embodiments, the method further includes: based on the second adjustment scheme, adjusting the crystal unit and / or the reflective structure in the crystal array by a robotic arm to complete the packaging of the crystal array.

[0011] In some embodiments, the acquiring of images to be identified related to one or more crystals includes: acquiring the images to be identified of the one or more crystals under a plurality of light sources of different colors; the performing of image recognition on the images to be identified and determining the identification results of the one or more crystals includes: performing image recognition on the images to be identified corresponding to a plurality of light sources of different colors and determining a plurality of candidate identification results of the one or more crystals; and determining the identification results of the one or more crystals based on the plurality of candidate identification results.

[0012] In some embodiments, the acquiring of images to be identified related to one or more crystals includes: for each crystal, determining the crystal color of the crystal; using the complementary color corresponding to the crystal color as a light source to illuminate the crystal, and acquiring the image to be identified corresponding to the crystal.

[0013] In some embodiments, the acquiring of images to be identified related to one or more crystals includes: for each of the one or more crystals, acquiring a crystal image of the crystal under an initial light source; performing image recognition on the crystal image to obtain a crystal segmentation result of the crystal; when the crystal segmentation result indicates that the crystal is displayed completely, determining the crystal image as the image to be identified of the crystal; when the crystal segmentation result indicates that the crystal is not displayed completely, repeatedly adjusting the light source angle, acquiring a crystal image of the crystal under the adjusted light source, performing image recognition on the crystal image under the adjusted light source to obtain a new crystal segmentation result, and fusing the previously obtained crystal segmentation results until the fused crystal segmentation result indicates that the crystal is displayed completely, and then determining the image to be identified based on the previously obtained crystal image of the crystal. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0015] Figure 1 is a schematic diagram of an application scenario of a crystal visual recognition processing method according to some embodiments of this specification;

[0016] Figure 2 is an exemplary flow chart of a crystal visual recognition processing method according to some embodiments of this specification;

[0017] Figure 3 is an exemplary flow chart for determining a crystal cutting plan according to some embodiments of this specification;

[0018] Figure 4 is another exemplary flow chart for determining a crystal cutting plan according to some embodiments of this specification;

[0019] Figure 5 is an exemplary flow chart of determining a first adjustment scheme for a crystal according to some embodiments of this specification;

[0020] Figure 6 is an exemplary flow chart of determining a second adjustment scheme for a crystal according to some embodiments of this specification;

[0021] Figure 7 This is an exemplary flowchart of determining an image to be recognized according to some embodiments of this specification. DETAILED DESCRIPTION

[0022] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0023] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0024] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0025] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0026] Figure 1 It is a schematic diagram of an application scenario of the crystal visual recognition processing method shown in some embodiments of this specification.

[0027] The application scenario 100 of the crystal visual recognition processing method (hereinafter referred to as the application scenario 100) can be used to perform image recognition on the crystal 110, determine the recognition result (for example, size data, position data, defect data, etc.) of the crystal 110, and further determine the processing scheme for processing the crystal 110. For more information about the recognition result, size data, position data, defect data and processing scheme, please refer to the relevant description below in this specification.

[0028] like Figure 1As shown, the application scenario 100 may include a crystal 110, an image acquisition device 120, a network 130, a processor 140, and a user terminal 150. The image acquisition device 120 may shoot one or more crystals 110 to obtain an image to be identified, the processor 140 may obtain the aforementioned image to be identified through the network 130, and perform image recognition on it to determine the identification results of one or more crystals 110, thereby further determining the identification scheme of one or more crystals 110, and the user may view the aforementioned identification results and processing schemes through the user terminal 150. For more information about the aforementioned description, please refer to the relevant description below in this specification.

[0029] The crystal 110 may refer to a crystal that needs to be processed. The types of the crystal 110 may include, but are not limited to, silicon carbide, germanium single crystal, etc. The application scenario 100 may include one or more crystals 110 .

[0030] The aforementioned processing of the crystal 110 may include, but is not limited to, cutting, adjustment, etc. For example, the crystal 110 may be a crystal in a natural state or an artificially grown state that has not yet completed processing or molding, and the application scenario 100 may be used to determine the size data and defect data of the crystal 110, thereby determining a cutting scheme for the crystal 110. For another example, the crystal 110 may also be a crystal unit after cutting, and the application scenario 100 may be used to determine the size data and position data of the crystal unit contained on the tray, thereby determining a first adjustment scheme for adjusting the crystal unit on the tray. For another example, the crystal 110 may also be a crystal array composed of multiple crystal units, and the application scenario 100 may be used to determine the position data and defect data of one or more crystals 110 in the crystal array, thereby determining a second adjustment scheme for the crystal array.

[0031] The image acquisition device 120 is a device for photographing the crystal 110. The image acquisition device 120 can photograph the crystal 110 to obtain one or more images to be identified. For more information about the images to be identified, please refer to the relevant description below in this specification.

[0032] In some embodiments, the application scenario 100 may include one image acquisition device 120, and may also include multiple image acquisition devices 120 at different angles. In some embodiments, the image acquisition device 120 may be fixed, or the shooting position and / or shooting angle may be adjusted. The image acquisition device 120 may include but is not limited to various devices that can shoot the crystal 110. For example, the image acquisition device 120 may be a camera, a video camera, etc. For another example, the image acquisition device 120 may also be an X-ray machine.

[0033] The network 130 can connect multiple components of the application scenario 100 and / or connect the application scenario 100 with external resources. The network 130 enables multiple components of the application scenario 100 and other components outside the application scenario 100 to communicate with each other, thereby facilitating the exchange of data and / or information. For example, the processor 140 can obtain the image to be recognized captured by the image acquisition device 120 through the network 130.

[0034] The network 130 may be any one or more of a wired network or a wireless network. For example, the network 130 may include a cable network, an optical fiber network, a telecommunication network, the Internet, a local area network, a wide area network, a wireless local area network, a Bluetooth network, a near field communication, a bus within a device, a line within a device, a cable connection, or any combination thereof. The network connection between the various parts may be in one of the above-mentioned ways or in multiple ways.

[0035] The processor 140 can process data and / or information obtained from other devices or components of the application scenario 100. The processor 140 can execute program instructions based on these data, information and / or processing results to perform one or more functions described in this specification. For example, the processor 140 can perform image recognition on the image to be identified, determine the recognition results of one or more crystals 110, and thus determine the processing scheme of one or more crystals 110. For more explanation of the above examples, please refer to the following in this specification. In some embodiments, the processor 140 may include one or more sub-processing devices (for example, a single-core processing device or a multi-core multi-core processing device). As an example only, the processor 140 may include a central processing unit, an application-specific integrated circuit, a dedicated instruction processor, a graphics processor, a physical processor, a digital signal processor, a controller, a microcontroller unit, a microprocessor, etc. or any combination thereof.

[0036] The user terminal 150 refers to one or more terminal devices or software used by the user. The aforementioned user may be the manager of the crystal 110. The user may query the identification result of the crystal 110 through the user terminal 150, that is, at least one of the size data, position data, and defect data of the crystal 110. The user terminal 150 may be used by one or more users, including users who directly use the service, and other related users. The user terminal 150 may be one or any combination of other devices with input and / or output functions, such as a mobile device, a tablet computer, a laptop computer, a desktop computer, etc.

[0037] In some embodiments, the application scenario 100 may further include other components. In some embodiments, the application scenario 100 may further include a tray, which may support and display the crystal 110. In some embodiments, the application scenario 100 may further include a weight detection device, and the crystal 110 may be placed in the detection area of ​​the weight detection device. The weight detection device may measure the crystal 110 to obtain the weight data of the crystal 110. For example, the weight detection device may directly measure the crystal 110 to obtain the weight data of the crystal 110. For another example, a tray may be placed in the detection area of ​​the weight detection device, and the crystal 110 may be placed on the tray. The weight detection device may detect the weight of the tray and the crystal 110, and the weight data of the crystal 110 may be obtained by removing the preset tray weight. The image acquisition device 120 may shoot the crystal 110 placed on the weight detection device to obtain an image to be identified.

[0038] It should be noted that application scenario 100 is provided only for the purpose of illustration and is not intended to limit the scope of this specification. For those of ordinary skill in the art, various modifications or variations can be made according to the description of this specification. For example, multiple components in application scenario 100 can be integrated into one device, such as processor 140 and user terminal 150 can be integrated together. For example, application scenario 100 can be implemented on other devices to achieve similar or different functions. However, the aforementioned changes and modifications will not deviate from the scope of this specification.

[0039] Figure 2 200 is an exemplary flow chart of a crystal visual recognition processing method according to some embodiments of this specification. In some embodiments, process 200 may be executed by a processor (eg, processor 140). Figure 2 As shown, process 200 includes the following steps.

[0040] Step 210, obtaining an image to be identified related to one or more crystals.

[0041] The image to be identified may be an image containing one or more crystals.

[0042] In some embodiments, the purpose of the processor's recognition is different, and the content in the image to be recognized may be different. For example, the image to be recognized may include an image of a tray containing multiple crystals, and the processor may recognize the aforementioned image to be recognized, thereby determining the size data and the first position data on the tray, thereby determining whether the aforementioned multiple crystals can be assembled in the next step, and whether they need to be adjusted through the first adjustment scheme. For another example, the image to be recognized may include an image of a crystal, and the processor may recognize the aforementioned image to be recognized, thereby determining the size data and the first defect data of the crystal, thereby determining the cutting scheme for cutting the crystal. For another example, the image to be recognized may also include an image to be recognized of a crystal array composed of multiple crystal units, and the processor may recognize the aforementioned image to be recognized, thereby determining the second position data, the second defect data, and the third defect data of the crystal array, thereby determining whether the aforementioned multiple crystal arrays meet the requirements and whether they need to be adjusted through the second adjustment scheme.

[0043] In some embodiments, the image to be identified can be obtained in a variety of ways. For example, the user can control the image acquisition device 120 to photograph one or more crystals, and transmit the aforementioned image to be identified to the processor through the user terminal (e.g., user terminal 150). For another example, the processor can also control the image acquisition device (e.g., image acquisition device 120) to photograph one or more crystals to obtain the image to be identified.

[0044] The type of the image to be identified can be various. For example, when the image acquisition device is a visual image acquisition device (such as a camera), the image to be identified can be a visual image. For another example, when the image acquisition device is an X-ray machine, the image to be identified can also be an X-ray film. The image to be identified can be a two-dimensional image or a three-dimensional image. When the image to be identified is a three-dimensional image, it can be obtained by image fusion of two-dimensional images of one or more crystals at different viewing angles.

[0045] The number of images to be identified may be one or more. For example, the shooting position and / or shooting angle of the image acquisition device may be adjusted to obtain images to be identified of multiple different viewing angles of one or more crystals. For another example, the placement state of one or more crystals may be adjusted (e.g., rotated), and the image acquisition device may shoot the adjusted one or more crystals to obtain images to be identified of one or more crystals in different placement states.

[0046] In some embodiments, one or more crystals can also be illuminated by multiple light sources of different colors, and the image acquisition device can shoot one or more crystals illuminated by light sources of different colors to obtain multiple images to be identified. The color of the aforementioned light source can be determined by preset, for example, preset to five colors. The color of the light source can be determined by the color of the crystal. For example, when the color of a certain crystal is yellow, it can be determined by the preset crystal-light source comparison table that the color irradiated on the crystal includes three colors. It is worth noting that the absorption of the crystal to different colors is different, so the shooting effect under different light sources is different. Some features of the crystal in the image to be identified (for example, size, defects, etc.) may be relatively more obvious. In addition, some defects have different appearance effects under different light sources. For example, it is difficult to be photographed in a certain light source, but it can be clearly shown in another light source. Therefore, some embodiments of this specification can improve the accuracy of crystal identification by shooting the image to be identified of the crystal under different color light sources for comprehensive judgment.

[0047] It is worth mentioning that many crystals (for example, silicon dioxide, fluorite, etc.) are transparent. When photographing transparent crystals, there may be problems with reflections and fusion with the background area.

[0048] In some embodiments, for each color of light source, the background color of the crystal in the to-be-identified image corresponding to the light source is the complementary color of the color corresponding to the light source. Transparent crystals may transmit light of the background color. By setting the background color and the light source color to complementary colors, the light of the background color can be additively mixed with the light source in the corresponding area of ​​the crystal, so that the color of this part is different from the color of other areas. Based on the additive mixing principle of colored light, when two colors at relative positions on the color wheel are added together, they can offset each other or produce neutrality, and finally form a visual white or gray tone, thereby emphasizing the crystal area and avoiding the fusion of the crystal and the background area.

[0049] In some embodiments, for each crystal, the processor may determine the crystal color of the crystal. The processor may determine the crystal color of the crystal in a variety of ways, such as user input, or by analyzing and determining after photographing the crystal.

[0050] In some embodiments, the processor can use the complementary color corresponding to the color of the crystal as a light source to illuminate the crystal and obtain an image to be identified corresponding to the crystal. For example, when a certain crystal (such as neodymium-doped yttrium aluminum garnet) is red, green can be selected as a light source to illuminate it and obtain an image to be identified. In some embodiments of this specification, this setting can enhance the color saturation and contrast of the crystal in the image to be identified, improve the contrast effect between the crystal and the background area, highlight the crystal structure, avoid the fusion of the crystal and the background area in the image to be identified, and ensure accurate identification of the crystal.

[0051] In some embodiments, the background color of the crystal in the image to be identified can be a complementary color of the crystal color, thereby highlighting the crystal structure, avoiding the fusion of the crystal and the background area in the image to be identified, and ensuring accurate identification of the crystal.

[0052] In some embodiments, the processor may further fuse one or more crystal images of the crystal. The fused crystal image may fully display the crystal. The fused crystal image is determined as the image to be identified. For more information about the above embodiments, see Figure 7 and its related description.

[0053] In some embodiments, the image to be identified may also include a two-dimensional code, which may be used to label one or more crystals in the image to be identified, so that the processor can distinguish and save the identification results of different crystals for subsequent query.

[0054] Step 220 , performing image recognition on the image to be recognized, and determining recognition results of one or more crystals, wherein the recognition results include at least one of size data, position data, and defect data of the one or more crystals.

[0055] The dimensional data may refer to data characterizing the crystal size, and may include but is not limited to the diameter, shape, crystal axis size, thickness, and grain size of the crystal.

[0056] The position data may refer to data characterizing the position of the crystal. In some embodiments, the position data may include first position data. In some embodiments, the position data may include second position data. For more information about the first position data and the second position data, please refer to the relevant description below in this specification.

[0057] Defect data may refer to data characterizing one or more crystal defects. In some embodiments, the defect data may include first defect data. In some embodiments, the defect data may include second defect data and third defect data. For more information about the first defect data, the second defect data, and the third defect data, please refer to the relevant description below in this specification.

[0058] In some embodiments, the processor can perform image recognition on the image to be recognized through a machine learning model (e.g., a size recognition model, a first position recognition model, a first defect recognition model, etc.) or other image recognition methods (e.g., a template matching method, feature extraction and matching, etc.) to determine the recognition results of one or more crystals. For more information about the aforementioned embodiments, please refer to the relevant description below in this specification.

[0059] In some embodiments, the processor may also determine defect data (e.g., first defect data, second defect data, etc.) in one or more crystals based on the size data and weight data of one or more crystals. For example, the processor may determine the density data of the crystal based on the weight data and size data of the crystal, and compare the density data with the density data of the corresponding type of crystal to determine the defect data of the aforementioned crystal. It is understandable that some crystals may contain other types of crystals, which are difficult to determine directly through image recognition. In some embodiments of this specification, the density data of the crystal can be determined through the weight data of the crystal and the size data determined by image recognition, thereby determining whether the crystal has defects, which can improve the accuracy of the recognition result.

[0060] In some embodiments, when the image to be identified includes images of one or more crystals under multiple light sources of different colors, the processor may perform image recognition on the image to be identified corresponding to the multiple light sources of different colors to determine multiple candidate identification results of the one or more crystals. The aforementioned candidate identification results may be identification results of the corresponding crystal candidates. Similar to the identification results, the candidate identification results may also include at least one of the size data, position data, and defect data of the corresponding crystal.

[0061] In some embodiments, the processor can perform image recognition on the image to be recognized by a machine learning model or other image recognition method, and determine candidate recognition results of one or more crystals. For more information about determining candidate recognition results, please refer to the relevant description of determining recognition results below in this specification.

[0062] In some embodiments, the candidate recognition result may further include a confidence level corresponding to the candidate recognition result. Exemplarily, when the processor processes the image to be recognized through a machine learning model, the output of the corresponding machine learning model may further include a confidence level.

[0063] In some embodiments, the processor may determine the recognition results of one or more crystals based on multiple candidate recognition results. For example, the processor may determine the recognition results of one or more crystals by fusing multiple candidate recognition results. For another example, when a certain feature (e.g., size) of a crystal includes multiple different candidate recognition results, the processor may determine the candidate recognition result with the highest confidence among the candidate recognition results as the recognition result of the feature.

[0064] Some embodiments of this specification can ensure the accuracy of the recognition result by recognizing the images to be recognized under different light sources and comprehensively judging the recognition result of the crystal. For example, the defects in the crystal may appear differently under different colors of light. Through the images to be recognized under different colors of light sources, the defects in the crystal can be better recognized.

[0065] Step 230 , based on the identification result, determine a treatment plan for one or more crystals.

[0066] A processing protocol may refer to a protocol for processing one or more crystals.

[0067] In some embodiments, the processing scheme may include a first adjustment scheme for adjusting one or more crystals on the tray. For more information on how the processor determines the first adjustment scheme based on the identification result, see Figure 5 and its related description.

[0068] In some embodiments, the processing plan may also include a cutting plan for cutting one or more crystals. For more information on how the processor determines the cutting plan of the crystal based on the recognition result, see Figure 3 , Figure 4 and its related description.

[0069] In some embodiments, the processing scheme may further include a second adjustment scheme for adjusting the crystal units and / or the reflective structures in the crystal array composed of one or more crystals. For more information on how the processor determines the second adjustment scheme for the crystal array based on the recognition result, see Figure 6 and its related description.

[0070] Some embodiments of the present specification can accurately determine a processing plan for a crystal by performing image recognition on one or more crystals, thereby realizing automated processing of the crystal, improving the efficiency of crystal processing, and reducing the production and processing costs of the crystal.

[0071] The crystals produced by the crystal growth equipment are of different sizes and shapes and may have defects, so they cannot be used directly. They need to be divided into crystal units of specified size, without defects or with fewer defects, so as to facilitate subsequent processing (for example, assembling into a crystal array and installing it in a scanning device). If the size and defects of the crystal are evaluated manually, on the one hand, the accuracy is not high, and on the other hand, the cost is high and the efficiency is low. Based on this, some embodiments of this specification can determine the size data and the first defect data of the crystal by performing image recognition on the images to be identified of one or more crystals, thereby determining the cutting plan of the crystal, which can improve the accuracy of crystal identification, reduce costs, and improve processing efficiency.

[0072] Figure 3 is an exemplary flow chart for determining a crystal cutting plan according to some embodiments of this specification. In some embodiments, process 300 may be executed by a processor (eg, processor 140). Figure 3 As shown, process 300 may include the following steps.

[0073] Step 310 , performing image recognition on the image to be recognized, and determining size data and first defect data of one or more crystals.

[0074] In some embodiments, the processor may perform image recognition on the image to be recognized through a size recognition model to determine the size data of one or more crystals. The processor may input the image to be recognized into the size recognition model, and the output of the size recognition model may be the size data of one or more crystals in the image to be recognized. The size recognition model may be one or a combination of a convolutional neural network or any other machine learning model that can realize this function.

[0075] The size recognition model can be obtained by training a first training sample with a first label. The first training sample may include a first sample image, and the first label may include sample size data corresponding to the crystal in the first sample image. The first training sample may be determined by photographing the crystal, and the first label may be determined by manually measuring the photographed crystal.

[0076] In some embodiments, the output of the size recognition model may also include the confidence of the size data. Correspondingly, when training the size recognition model, the first label may also include a first sample confidence, which may be determined by manual annotation (for example, annotated as 1). When the confidence of the size data output by the size recognition model is greater than a preset first confidence threshold, the size data output by the size recognition model may be determined as the size data of the crystal; when the confidence of the size data output by the size recognition model is less than or equal to the preset first confidence threshold, a new image to be identified may be obtained by adjusting the color of the light source or the shooting angle and position of the image acquisition device, and the new image to be identified may be processed by the size recognition model until the size data output by the size recognition model is greater than the first confidence threshold, thereby ensuring the accuracy of the size data in the recognition result.

[0077] The first defect data represents the first defect data of the crystal itself. For example, the first defect includes but is not limited to the location, size, etc. of point defects, line defects, surface defects, crystal lines, inclusions, and other defects of the crystal.

[0078] In some embodiments, the processor may perform image recognition on the image to be recognized through the first defect recognition model to determine the first defect data of one or more crystals. The processor may input the image to be recognized into the first defect recognition model, and the output of the first defect recognition model may be the first defect data of one or more crystals in the image to be recognized. The first defect recognition model is one or a combination of a convolutional neural network or any other machine learning model that can realize this function.

[0079] The first defect recognition model can be obtained by training a second training sample with a second label. The second training sample may include a second sample image, and the second label may include sample second defect data corresponding to the crystal in the second sample image. The aforementioned second training sample can be determined by photographing the crystal, and the second label can be determined by manually observing and marking the photographed crystal.

[0080] In some embodiments, the output of the first defect recognition model may also include the confidence of the first defect data. Correspondingly, when training the first defect recognition model, the second label may also include the second sample confidence, and the second sample confidence may be determined by manual annotation. When the confidence of the first defect data output by the first defect recognition model is greater than the preset second confidence threshold, the first defect data output by the first defect recognition model may be determined as the first defect data of the crystal; when the confidence of the first defect data output by the first defect recognition model is less than or equal to the preset second confidence threshold, a new image to be recognized may be obtained by adjusting the color of the light source or the shooting angle and position of the image acquisition device, and the new image to be recognized may be processed by the first defect recognition model until the first defect data output by the first defect recognition model is greater than the second confidence threshold, thereby ensuring the accuracy of the first defect data in the recognition result.

[0081] Step 320 , obtaining a first cutting target of one or more crystals.

[0082] The first cutting target may refer to the primary target for cutting one or more crystals. When cutting a crystal, first consideration should be given to satisfying the first cutting target.

[0083] In some embodiments, the first cutting target may include the size of one or more crystal units to be cut. In some embodiments, the size of the crystal unit may also be determined by a preset. For example, the size of the crystal unit may be determined by an order requirement.

[0084] In some embodiments, the first cutting target may include a cutting priority. For example, the first cutting target may be crystal quantity priority or quality priority.

[0085] In some embodiments, the first cutting target may include a specific number of cuts and / or cutting quality. For example, the first cutting target may include cutting 10 crystal units of a specified size. The aforementioned cutting quality may be related to the defects in the crystal unit after cutting. The fewer the defects in the crystal unit after cutting, the higher the cutting quality. The aforementioned cutting quality may be represented by a quality score, for example, 1 to 10 points. The higher the quality score, the higher the cutting quality. The aforementioned cutting quality may also be represented by a quality grade, for example, grades I to V. The higher the quality grade, the higher the cutting quality. Exemplarily, the first cutting target may include that the quality grade of the crystal unit after cutting is at least grade II.

[0086] In some embodiments, the first cutting target of one or more crystals can be obtained in a variety of ways. For example, the first cutting target of one or more crystals can be obtained through user input. For another example, the first cutting target can also be determined through the size data and the first defect data of one or more crystals and a preset correspondence table.

[0087] Step 330 , for each crystal, based on the size data of the crystal, the first defect data, and the first cutting target, determine a cutting plan for the crystal.

[0088] The cutting scheme may be a specific scheme for cutting the crystal. The cutting scheme may include but is not limited to the number of times the crystal is cut, the angle and depth of each cut, etc. The cutting machine may cut the crystal based on the cutting scheme, and the crystal unit obtained after cutting meets the first cutting target.

[0089] In some embodiments, for each crystal, the processor can perform modeling or use various data analysis algorithms, such as regression analysis, discriminant analysis, etc., to analyze and process the size data, first defect data, and first cutting target of the crystal to obtain a cutting plan for the crystal.

[0090] In some embodiments, for each crystal, the processor obtains multiple candidate cutting schemes based on the size data, the first defect data, and the first cutting target of the crystal, and screens the multiple candidate cutting schemes to determine the cutting scheme of the crystal. For more information about the above embodiments, see Figure 4 and its related description.

[0091] Some embodiments of the present specification can accurately determine crystal size data and first defect data by performing image recognition on one or more crystals, and determine a cutting plan for cutting the crystal in combination with a first cutting target, thereby realizing automated processing of the crystal, improving the cutting accuracy and consistency of the crystal, reducing crystal loss, and reducing labor costs.

[0092] In some embodiments, for each order, the processor may determine the crystal delivery amount corresponding to the order based on the order requirements and the cutting schemes of each crystal. Among them, the order requirements may include the size of the required crystal unit and the number of crystals required. For example, the order requirements may include two hundred crystal units of size A. The processor may determine the cutting scheme of each crystal when the crystal unit size corresponding to the aforementioned order requirements is met, thereby determining the number of crystal units of each crystal based on the cutting scheme, and based on the number of crystals in the order requirements and the number of crystal units of each crystal based on the cutting scheme, determine the crystal delivery amount corresponding to the order. It can be understood that the crystal unit size required for each order may be different. If the number of crystals delivered is too small when cutting, the number of crystal units obtained cannot meet the order requirements, which will lead to repeated cutting operations and cumbersome processes; if the number of crystals delivered is too large when cutting, the number of crystal units obtained will exceed the order requirements, resulting in waste of crystals and increased costs. Some embodiments of this specification can accurately determine the crystal delivery amount corresponding to each order based on the order requirements and the cutting schemes of each crystal, ensure that the number of crystal units obtained after cutting meets the order requirements, and avoid problems caused by too much or too little feeding.

[0093] Figure 4 FIG. 4 is another exemplary flow chart for determining a crystal cutting plan according to some embodiments of the present specification. In some embodiments, process 400 may be executed by a processor (eg, processor 140). Figure 4 As shown, process 400 may include the following steps.

[0094] Step 410 , determining a plurality of candidate cutting schemes based on the size data of the crystal, the first defect data, and the first cutting target.

[0095] The candidate cutting scheme may refer to a scheme for cutting the crystal to be evaluated. Similar to the cutting scheme, the candidate cutting scheme may include but is not limited to the number of times the crystal is cut, the angle and depth of each cut, etc. It is worth noting that the crystal units obtained after cutting the crystal based on each candidate cutting scheme can meet the first cutting target.

[0096] In some embodiments, multiple candidate cutting schemes can be determined through multiple schemes based on the size data of the crystal, the first defect data, and the first cutting target. For example, a user can generate multiple candidate cutting schemes based on the size data of the crystal, the first defect data, and the first cutting target, and the processor can obtain the candidate cutting schemes input by the user. For another example, the processor can randomly generate multiple candidate cutting schemes based on the size data of the crystal, the first defect data, and the first cutting target.

[0097] For each candidate cutting scheme, the processor may perform the following steps 420 and 430:

[0098] Step 420 , determining a unit score of each cut crystal unit based on the candidate cutting scheme, the size data of the crystal, and the first defect data.

[0099] The processor can generate a three-dimensional model of the crystal based on the size data of the crystal and the first defect data, and then simulate cutting the three-dimensional model of the crystal based on the candidate cutting scheme through the simulation cutting software to obtain multiple crystal units after cutting.

[0100] In some embodiments, for each cut crystal unit, the processor can analyze and process the crystal unit, and then determine the unit score of the crystal unit based on a preset score relationship table. The aforementioned unit score can be used to evaluate the quality of the crystal unit. The unit score can be represented by a score. For example, the unit score can be 0 to 100 points, and the higher the score, the better the corresponding crystal unit.

[0101] In some embodiments, the aforementioned unit score may be related to defects in the crystal unit. For example, the processor analyzes whether there are defects in the crystal unit and the specific size of the defects, and determines the unit score of the crystal unit through a preset score relationship table. In some embodiments, the aforementioned unit score may also be related to other parameter information in the crystal unit. For example, the processor may also determine the unit score of the crystal unit in combination with the size of the crystal unit. For another example, the processor may also determine the optical properties of each crystal unit, such as refractive index, transmittance, polarization, etc., through simulation analysis by optical simulation software (e.g., TracePro, CODE V, etc.), thereby determining the unit score of the crystal unit in combination with the optical properties of the aforementioned each crystal unit.

[0102] Step 430 : determining a plan score of the candidate cutting plan based on the unit scores of each crystal unit.

[0103] The scheme score can be used to evaluate the quality of the candidate cutting schemes. For example, the scheme score can be represented by a score, and a higher scheme score can represent a better corresponding candidate cutting scheme.

[0104] In some embodiments, the processor may sum the unit scores of the respective crystal units to determine the plan score of the candidate cutting plan.

[0105] In some embodiments, the processor may obtain a second cutting target for cutting the crystal. The aforementioned second cutting target may refer to a secondary target for cutting one or more crystals. The aforementioned second cutting target may be determined by user input. When cutting the crystal, on the premise of satisfying the first cutting target, further consideration may be given to satisfying the second cutting target. It is worth noting that, on the premise of satisfying the first cutting target, the candidate cutting scheme may not satisfy the second cutting target, or may satisfy the second cutting target, but satisfying the candidate cutting scheme while satisfying both the first cutting target and the second cutting target is more in line with user expectations.

[0106] In some embodiments, the second cutting target may include a cutting priority. For example, the first cutting target may be to cut 10 crystal units of a specified size, and the second cutting target may include that the quality of the aforementioned 10 crystal units of the specified size is as high as possible. For another example, the first cutting target may be to cut 10 crystal units of a specified size, and the second cutting target may include that the number of crystal units of the aforementioned specified size is as high as possible.

[0107] In some embodiments, the processor may determine the quantity weight and quality weight of the crystal unit based on the second cutting target. Exemplarily, the processor may determine the quantity weight and quality weight of the crystal unit based on the cutting priority in the second cutting target through a preset weight relationship table. For example, when the processor is based on the cutting priority in the second cutting target as quality priority, the quantity weight of the crystal unit may be determined to be 0.3 and the quality weight to be 0.7 through a preset weight relationship table.

[0108] In some embodiments, the processor may determine the solution score of the candidate cutting solution based on the quantity weight, the quality weight, the number of crystal units, and the unit score. For example, the processor may determine based on the following formula: Among them, S is the solution score of the candidate cutting solution, w1 is the quantity weight, N is the number of crystal units, w2 is the quality weight, s i Give the cell score for the i-th crystal cell.

[0109] In some embodiments of the present specification, the second cutting target can further refine the user's requirements for crystal cutting to ensure that the crystal unit obtained after cutting the crystal according to the determined cutting plan better meets the user's needs and optimizes the flexibility of crystal cutting.

[0110] In some embodiments, the processor can also determine the discarded portion after crystal cutting based on the candidate cutting scheme and the size data, determine the discard score of the candidate cutting scheme based on the aforementioned discarded portion, and determine the scheme score of the candidate cutting scheme in combination with the unit score of the crystal unit. The discard score can be used to evaluate the degree of waste of the crystal by the candidate cutting scheme. The aforementioned discard score can be related to the volume or weight of the waste generated by the crystal cutting after executing the candidate cutting scheme. The more waste, the more waste, and the lower the discard score. In some embodiments, the aforementioned discard score can be a negative value.

[0111] Some embodiments of this specification can combine the discard score to evaluate the candidate cutting schemes from multiple aspects, which can effectively reduce the consumption of crystals and save crystal production costs.

[0112] Step 440: Determine a cutting plan for the crystal based on the plan scores of each candidate cutting plan.

[0113] In some embodiments, the processor may determine the candidate cutting plan with the highest plan score as the cutting plan for the crystal.

[0114] Some embodiments of the present specification can screen candidate cutting plans to make the determined cutting plans more in line with the user's cutting needs, improve the degree of automation in crystal production, improve the flexibility and adaptability of crystal cutting, reduce the cost and error caused by manually designed cutting plans, and ensure the smooth progress of crystal cutting.

[0115] In some embodiments, the crystals (e.g., crystal units) on the tray can be assembled by a robotic arm to improve the degree of automation of crystal production, improve assembly efficiency and accuracy, and reduce assembly costs. It is understandable that the robotic arm can grab and assemble the crystals on the tray based on a preset program. If the crystal is not placed in the specified position on the tray, or the crystal placed in the corresponding position is not the right size, it may cause the crystal assembly to fail or have defects. Based on this, some embodiments of the present specification can determine the size data and the first position data of the crystal by performing image recognition on the image to be identified of one or more crystals on the tray, thereby determining a first adjustment scheme for adjusting one or more crystals on the tray, thereby improving the success rate of crystal assembly.

[0116] Figure 5 500 is an exemplary flow chart of determining a first adjustment scheme for a crystal according to some embodiments of this specification. In some embodiments, process 500 may be executed by a processor (eg, processor 140). Figure 5 As shown, process 500 may include the following steps.

[0117] Step 510 , obtaining an image to be identified of a tray containing one or more crystals.

[0118] Step 520 , performing image recognition on the image to be recognized, and determining size data and first position data of one or more crystals.

[0119] In some embodiments, the processor can perform image recognition on the image to be recognized through the size recognition model to determine the size data of one or more crystals. For more information about the size recognition model, see Figure 2 and its related description.

[0120] The first position data may refer to the position data of the crystals in the tray. For example, when the tray is a rectangular tray, the tray coordinate system may be constructed with the lower left corner of the tray as the coordinate origin, the width of the tray as the X-axis, and the length of the tray as the Y-axis. The first position data may refer to the position of each crystal in the tray in the aforementioned tray coordinate system. In some embodiments, the first position data may include the coordinate position and the tilt angle in the tray. The aforementioned coordinate position may be the coordinate position of the centroid of the crystal, and the aforementioned tilt angle may be the tilt angle in the extension direction of the crystal.

[0121] In some embodiments, the processor may analyze and process the image to be identified through a first position recognition model to determine the first position data of one or more crystals. The processor may input the image to be identified into the first position recognition model, and the output of the first position recognition model may be the first position data of one or more crystals. The first position recognition model may be one or a combination of a convolutional neural network or any other machine learning model that can achieve this function.

[0122] The first position recognition model can be obtained by training a third training sample with a third label. The third training sample may include a third sample image, and the third label may include sample first position data corresponding to the crystal in the third sample image. The third training sample may be determined by photographing the crystal, and the third label may be determined by manually measuring the position of the photographed crystal.

[0123] In some embodiments, the output of the first position recognition model may also include the confidence of the first position data. Correspondingly, when training the first position recognition model, the third label may also include a third sample confidence, and the third sample confidence may be determined by manual annotation. When the confidence of the first position data output by the first position recognition model is greater than the preset third confidence threshold, the first position data output by the first position recognition model may be determined as the first position data of the crystal; when the confidence of the first position data output by the first position recognition model is less than or equal to the preset third confidence threshold, a new image to be recognized may be obtained by adjusting the color of the light source or the shooting angle and position of the image acquisition device, and the new image to be recognized may be processed by the first position recognition model until the first position data output by the first position recognition model is greater than the third confidence threshold, thereby ensuring the accuracy of the first position data in the recognition result.

[0124] Step 530 : determining a first adjustment solution based on the configuration target of the tray, the size data of one or more crystals, and the first position data.

[0125] The configuration target may refer to the target for configuring the crystals on the tray. The aforementioned configuration target may include the number of crystals to be configured on the tray, the size of each crystal, and the position relative to the tray, etc.

[0126] In some embodiments, the processor can analyze the size data and the first position data of each crystal to evaluate whether each crystal meets the configuration target. When each crystal meets the configuration target, it can be determined that the first adjustment scheme does not require adjustment; when there are one or more crystals that do not meet the configuration target, the processor can determine the first adjustment scheme for adjusting the corresponding crystal.

[0127] For example, when a crystal on the tray does not match the position specified in the configuration target (for example, the position coordinates deviate or the angle deviates), the first adjustment plan may include adjusting the crystal to the corresponding position of the configuration target. For another example, when there are extra crystals in the tray, the first adjustment plan may include moving the extra crystals out of the tray. For another example, when the size data of a crystal on the tray does not match the crystal size specified for the corresponding position by the configuration target, the first adjustment plan may include replacing the crystal at the corresponding position, and the size data of the replaced crystal matches the crystal size specified for the corresponding position by the configuration target.

[0128] In some embodiments, the processor may send the first adjustment solution to a user terminal for the user to view.

[0129] In some embodiments, the processor may also generate a control instruction based on the first adjustment scheme and send it to the robotic arm. After receiving the control instruction, the robotic arm may adjust the crystal on the tray.

[0130] Some embodiments of the present specification perform image recognition on one or more crystals on a tray and determine a first adjustment plan for adjusting the crystals on the tray based on the recognition results, thereby ensuring that the crystals are located at designated positions on the tray, ensuring subsequent smooth assembly of the crystals on the tray, and improving the assembly efficiency and quality of the crystals.

[0131] Some embodiments of the present specification also perform image recognition on the image to be identified of a crystal array composed of multiple crystal units, and determine whether it is necessary to adjust the crystals at various positions in the aforementioned crystal array through the size data of each crystal in the crystal array, so as to ensure the appearance consistency of the crystal array, improve the assembly quality of the crystal array, and improve the detection efficiency.

[0132] In some embodiments, the processor can also analyze the performance data of each crystal and the first position data to evaluate whether each crystal meets the performance target. When each crystal meets the performance target, it can be determined that the third adjustment scheme does not require adjustment; when there are one or more crystals that do not meet the performance target, the processor can determine a third adjustment scheme to adjust the corresponding crystal.

[0133] The aforementioned performance target may refer to a target that one or more performance parameters of the crystal on the tray need to meet. The aforementioned performance parameter may refer to a parameter that characterizes the performance of the crystal, and the performance parameter may include but is not limited to crystal surface roughness, light yield, energy resolution, decay time, etc. The performance parameter can be obtained by testing the crystal using performance testing equipment.

[0134] The performance target may include a range that one or more performance parameters of the crystals to be configured on the tray must meet. For example, the aforementioned performance target may include, but is not limited to, one or more of a crystal surface roughness target, a light yield target, an energy resolution target, a decay time target, etc. The performance target may be determined by the application requirements of the crystal array.

[0135] For example, the processor can also obtain one or more performance parameters of the crystals at each position on the tray, and determine whether the data of the crystals at each position meet the performance target. When the performance parameters of the crystal at a certain position meet the configuration target, the crystal at the corresponding position is retained. When the performance parameters of the crystal at a certain position do not meet the configuration target, the crystal at the corresponding position is replaced, and the data of the replaced crystal meets the configuration target.

[0136] In some embodiments, the processor may send the third adjustment scheme to the user terminal to prompt the user to adjust the crystal array. After receiving the third adjustment scheme through the user terminal, the user may choose whether to adjust the aforementioned crystal array as needed.

[0137] In some embodiments, the processor may also generate a control instruction based on the third adjustment scheme and send it to the robotic arm. After receiving the control instruction, the robotic arm may adjust the crystals on the tray.

[0138] Some embodiments of the present specification also perform image recognition on the image to be identified of a crystal array composed of multiple crystal units, and determine whether it is necessary to adjust the crystals at various positions in the aforementioned crystal array through the performance parameters of each crystal in the crystal array, thereby ensuring the performance consistency of the crystal array, improving the assembly quality of the crystal array, and improving the detection efficiency.

[0139] Figure 6 600 is an exemplary flow chart of determining a second adjustment scheme for a crystal according to some embodiments of this specification. In some embodiments, process 600 may be executed by a processor (eg, processor 140). Figure 6 As shown, process 600 may include the following steps.

[0140] Step 610 , obtaining an image to be identified related to a crystal array composed of a plurality of crystal units, wherein a reflection structure is provided between at least two adjacent crystal units in the crystal array.

[0141] A crystal array may refer to an array formed by combining multiple crystal units. The aforementioned crystal array may be used in nuclear medicine such as X-ray tomography and positron emission tomography, and may also be used in nuclear detection technology such as industrial tomography, oil well exploration, nuclear physics, high energy physics, environmental testing, safety testing, weapon equipment fire control, guidance and other fields.

[0142] There is a reflective structure between at least two adjacent crystal units in the crystal array. The aforementioned reflective structure refers to a structure with light reflection capability inside the crystal array, which can enhance the optical performance of the crystal array and prevent crosstalk between target crystals. The reflective structure includes, but is not limited to, one or more of a reflective filling material (e.g., a barium compound, a titanium compound, or a mixture of a barium compound and a titanium compound), a reflective film (e.g., an ESR film, a polytetrafluoroethylene tape, an aluminum foil, a white polyester reflective film, etc.).

[0143] For information on how to obtain the image of the crystal array to be identified, please refer to Figure 2 and its related description.

[0144] Step 620: Perform image recognition on the image to be recognized to determine second position data, second defect data, and third defect data.

[0145] The second position data characterizes the relative positions of the plurality of crystal units in the crystal array, for example, the arrangement position of each crystal unit in the crystal array (eg, a certain position in a certain row).

[0146] In some embodiments, the processor may perform image recognition on the image to be recognized based on the second position recognition model to determine the second position data. The processor may input the image to be recognized into the second position recognition model, and the output of the second position recognition model may be the second position data of each crystal unit in the crystal array. The second position recognition model may be one or a combination of a convolutional neural network or any other machine learning model that can realize this function.

[0147] The second position recognition model can be obtained by training a fourth training sample with a fourth label. The fourth training sample may include a fourth sample image, and the fourth label may include sample second position data corresponding to the crystal in the fourth sample image. The aforementioned fourth training sample can be determined by photographing a crystal array, and the fourth label can be determined by manually marking the position of the crystal unit in the photographed crystal array.

[0148] In some embodiments, the output of the second position recognition model may also include the confidence of the second position data. Correspondingly, when training the second position recognition model, the fourth label may also include the fourth sample confidence, and the fourth sample confidence may be determined by manual annotation. When the confidence of the second position data output by the second position recognition model is greater than the preset fourth confidence threshold, the second position data output by the second position recognition model may be determined as the second position data of the crystal; when the confidence of the second position data output by the second position recognition model is less than or equal to the preset fourth confidence threshold, a new image to be recognized may be obtained by adjusting the color of the light source or the shooting angle and position of the image acquisition device, and the new image to be recognized may be processed by the second position recognition model until the second position data output by the second position recognition model is greater than the fourth confidence threshold, thereby ensuring the accuracy of the second position data in the recognition result.

[0149] The second defect data may refer to data characterizing defects of the crystal unit itself in the crystal array. For example, the second defect data may include but is not limited to the location, size, etc. of defects such as point defects, line defects, surface defects, crystal lines, inclusions, etc. of the crystal unit.

[0150] In some embodiments, the processor may perform image recognition on the image to be recognized through a second defect recognition model to determine the second defect data. The processor may input the image to be recognized into the second defect recognition model, and the output of the second defect recognition model may be the second defect data of the crystal unit in the image to be recognized. The second defect recognition model is one or a combination of a convolutional neural network or any other machine learning model that can realize this function.

[0151] The second defect recognition model can be obtained by training a fifth training sample with a fifth label. The fifth training sample may include a fifth sample image, and the fifth label may include sample second defect data corresponding to the crystal unit in the fifth sample image. The fifth training sample may be determined by photographing the crystal array, and the fifth label may be determined by manually observing and annotating the photographed crystal unit.

[0152] In some embodiments, the output of the second defect recognition model may also include the confidence of the second defect data. Correspondingly, when training the second defect recognition model, the fifth label may also include the fifth sample confidence, and the fifth sample confidence may be determined by manual annotation. When the confidence of the second defect data output by the second defect recognition model is greater than the preset fifth confidence threshold, the second defect data output by the second defect recognition model may be determined as the second defect data of the crystal; when the confidence of the second defect data output by the second defect recognition model is less than or equal to the preset fifth confidence threshold, a new image to be recognized may be obtained by adjusting the color of the light source or the shooting angle and position of the image acquisition device, and the new image to be recognized may be processed by the second defect recognition model until the second defect data output by the second defect recognition model is greater than the fifth confidence threshold, thereby ensuring the accuracy of the second defect data in the recognition result.

[0153] The third defect data is data characterizing the defects of the crystal array setting. In some embodiments, the third defect data may include defects in the position setting of the crystal unit in the crystal array. For example, the crystal unit at a certain position in a certain row is not aligned with other crystal units. In some embodiments, the third defect data may also include defects in the position setting of the reflective structure in the crystal array. For example, there is a lack of a reflective structure between two adjacent crystal units. For another example, the reflective structure is set in the wrong position.

[0154] In some embodiments, the processor may perform image recognition on the image to be recognized through a third defect recognition model to determine third defect data. The processor may input the image to be recognized and the target array image into the third defect recognition model, and the output of the third defect recognition model may be the third defect data of the crystal unit in the image to be recognized. The third defect recognition model is one or a combination of a convolutional neural network or any other machine learning model that can realize this function. The aforementioned target array image may be an image of a crystal array that needs to be set. The target array image may be determined by user input.

[0155] The third defect recognition model can be obtained by training a sixth training sample with a sixth label. The sixth training sample may include a sixth sample image and a sample array image, and the sixth label may include the sample third defect data corresponding to the crystal unit in the sixth sample image. The aforementioned sixth training sample can be determined by photographing the crystal array, and the sixth label can be determined by manually comparing the settings of the crystal unit and the reflective structure in the sixth sample image and the sample array image.

[0156] In some embodiments, the output of the third defect recognition model may also include the confidence of the third defect data. Correspondingly, when the third defect recognition model is trained, the sixth label may also include the sixth sample confidence, and the sixth sample confidence may be determined by manual annotation. When the confidence of the third defect data output by the third defect recognition model is greater than the preset sixth confidence threshold, the third defect data output by the third defect recognition model may be determined as the third defect data of the crystal; when the confidence of the third defect data output by the third defect recognition model is less than or equal to the preset sixth confidence threshold, a new image to be recognized may be obtained by adjusting the color of the light source or the shooting angle and position of the image acquisition device, and the new image to be recognized may be processed by the third defect recognition model until the third defect data output by the third defect recognition model is greater than the sixth confidence threshold, thereby ensuring the accuracy of the third defect data in the recognition result.

[0157] Step 630: Determine a second adjustment solution based on the second position data, the second defect data, and the third defect data.

[0158] The processor can analyze and process the second position data, the second defect data, the third defect data, and the target array requirements to determine the second adjustment scheme. The target array requirements can be the requirements for the crystal array input by the user. For example, the allowed defect types and sizes. Another example is the specific arrangement of the crystal array.

[0159] For example, when the second position data indicates that the second row includes only three crystal units, but the target array requirement indicates that the second row includes four crystal units, the processor may determine that the second adjustment solution includes adding one crystal unit to the second row.

[0160] For another example, when the second defect data indicates that the volume of the second crystal unit from the left in the second row is 0.02 mm 3 However, the maximum volume of defects allowed by the target array characterization is 0.01mm 3 , the processor may determine that the second adjustment scheme includes supplementing a reflection structure between the second crystal unit and the third crystal unit from the left in the second row.

[0161] For another example, when the third defect data indicates that no reflective structure is provided between the second crystal unit from the left and the third crystal unit in the second row, the processor may determine that the second adjustment scheme includes adding a reflective structure between the second crystal unit from the left and the third crystal unit in the second row.

[0162] For another example, when the second position data, the second defect data, and the third defect data all meet the target array requirement, the processor may determine that the second adjustment solution does not require adjustment.

[0163] Some embodiments of the present specification can determine a second adjustment scheme for adjusting the crystal units and / or reflective structures in the crystal array by performing image recognition on an image to be identified that includes a crystal array, thereby improving the detection efficiency of the crystal array, ensuring the yield of the crystal array, and enhancing the degree of automation in crystal production.

[0164] In some embodiments, the processor may send the second adjustment scheme to the user terminal to prompt the user to adjust the crystal array. After receiving the second adjustment scheme through the user terminal, the user may choose whether to adjust the crystal array as needed.

[0165] In some embodiments, the crystal unit and / or the reflective structure in the crystal array can be adjusted by the mechanical arm based on the second adjustment scheme to complete the packaging of the crystal array. The processor can generate a control instruction based on the second adjustment scheme and send it to the mechanical arm. The mechanical arm can adjust the crystal unit and / or the reflective structure in the crystal array based on the received control instruction, thereby completing the packaging of the crystal array, improving the packaging efficiency of the crystal, improving the packaging accuracy, ensuring the continuous operation capability of the crystal packaging, and reducing costs.

[0166] It is understandable that when a transparent crystal is photographed, the crystal in the crystal image may reflect light or merge with the background area, resulting in an incomplete display of the crystal. If image recognition is performed directly based on the crystal image as the image to be recognized, the accuracy of the recognition result will be reduced. Based on this, some embodiments of this specification can judge the integrity of the crystal in the crystal image to ensure that the crystal can be fully displayed in the image to be recognized, so as to ensure the accuracy of image recognition of the image to be recognized.

[0167] Figure 7 is an exemplary flow chart of determining an image to be identified according to some embodiments of this specification. In some embodiments, for each of the one or more crystals, a processor (e.g., processor 140) may execute process 700 to determine the integrity of the crystal in the image to be identified. Figure 7 As shown, process 700 may include the following steps.

[0168] Step 710, acquiring a crystal image of the crystal under an initial light source.

[0169] The aforementioned initial light source may refer to a pre-set light source.

[0170] Step 720, performing image recognition on the crystal image to obtain a crystal segmentation result of the crystal.

[0171] The crystal segmentation result refers to the result of segmenting the crystals in the crystal image. The crystal segmentation result only includes the crystals in the crystal image.

[0172] In some embodiments, the processor may perform image recognition on the crystal image based on the segmentation model to obtain a crystal segmentation result of the crystal in the crystal image. The processor may input the crystal image into the segmentation model to obtain a crystal segmentation result of the crystal in the crystal image of the segmentation model. The segmentation model may be one or more of a convolutional neural network or any other machine learning model that can achieve this function.

[0173] The segmentation model can be obtained by training a seventh training sample with a seventh label. The seventh training sample may include a seventh sample image, and the seventh label may include a crystal segmentation result corresponding to the crystal in the seventh sample image. The aforementioned seventh training sample can be determined by photographing the crystal, and the seventh label can be determined by manually segmenting the crystal in the seventh sample image.

[0174] Step 730, determining whether the crystal segmentation result indicates that the crystal is complete.

[0175] The processor can evaluate the crystal segmentation result through the completeness evaluation model to determine whether the crystal segmentation result displays the corresponding crystal completely. The input of the completeness evaluation model can be the crystal segmentation result, and the output can be the judgment result of whether the crystal segmentation result displays the corresponding crystal completely. The completeness evaluation model can be one or more of a convolutional neural network or any other machine learning model that can realize this function.

[0176] The completeness evaluation model can be obtained by training an eighth training sample with an eighth label. The eighth training sample may include the first sample segmentation result, and the eighth label may include a sample judgment result of whether the aforementioned sample segmentation result displays the corresponding crystal completely. The aforementioned eighth training sample may be determined by segmenting the crystal after photographing it, and the eighth label may be determined by manually partially blocking or not blocking the eighth training sample. When the eighth training sample is not blocked, the corresponding eighth label is displayed completely; when the eighth training sample is partially blocked, the corresponding eighth label is not displayed completely.

[0177] Step 740: When the crystal segmentation result indicates that the crystal is complete, the crystal image is determined as the crystal image to be identified.

[0178] Step 750: When the crystal segmentation result indicates that the crystal is not complete, adjust the light source angle to obtain a crystal image of the crystal under the adjusted light source.

[0179] In some embodiments, the processor can adjust the light source angle based on a variety of methods. For example, the processor can adjust the light source based on a preset adjustment angle.

[0180] In some embodiments, the processor may also analyze and process the crystal segmentation result based on the light source adjustment model to determine an adjusted light source angle.

[0181] In some embodiments, the light source adjustment model may include a missing position judgment layer and an angle adjustment layer. The aforementioned missing position judgment layer can be used to judge the missing crystal position in the crystal segmentation result. The input of the missing position judgment layer may include the crystal segmentation results of each crystal, and the output may include the crystal position of the corresponding crystal missing. The aforementioned angle adjustment layer can be used to adjust the light source angle to supplement the crystal position of the missing crystal. The input of the angle adjustment layer may include the crystal position of the missing crystal, and the output may include the adjusted light source angle. The missing position judgment layer and the angle adjustment layer can be one or more of a convolutional neural network or any other machine learning model that can implement this function.

[0182] In some embodiments, the light source adjustment model can be obtained by jointly training the missing position judgment layer and the angle adjustment layer. The ninth training sample may include the second sample segmentation result, and the ninth label may include the adjusted sample light source angle. The ninth training sample may be input into the initial missing position judgment layer, and the output of the initial missing position judgment layer may be input into the initial angle adjustment layer. A loss function may be constructed based on the output and label of the initial angle adjustment layer, and the parameters of the initial missing position judgment layer and the initial angle adjustment layer may be iteratively updated based on the loss function until the preset conditions are met, the parameters in the missing position judgment layer and the angle adjustment layer are determined, and a trained light source adjustment model is obtained. The preset conditions may include but are not limited to the convergence of the loss function, the training cycle reaching a threshold, etc.

[0183] In some embodiments, a sample crystal can be photographed at a sample light source angle to obtain an image. In the aforementioned image, the sample crystal has reflections or merges with the background area. The aforementioned image is processed to obtain a second sample segmentation result. The second sample segmentation result indicates that the sample crystal is not displayed completely. The second sample segmentation result is used as the ninth training sample. The aforementioned sample light source angle is manually adjusted, and the adjusted sample crystal is photographed to obtain an image. A new sample crystal segmentation result can be obtained by processing the aforementioned image. The aforementioned new sample crystal segmentation result can supplement the missing part of the sample crystal in the second sample segmentation result. The adjusted sample light source is used as the ninth label. It is worth noting that the new sample crystal segmentation result can display the sample crystal completely or incompletely.

[0184] Step 760: Perform image recognition on the crystal image under the adjusted light source to obtain a new crystal segmentation result.

[0185] Performing image recognition on the crystal image under the adjusted light source to obtain a new crystal segmentation result can refer to the relevant description of performing image recognition on the crystal image to obtain the crystal segmentation result of the crystal in step 720.

[0186] Step 770, merging the previously obtained crystal segmentation results until the fused crystal segmentation result represents a complete display of the crystal, and determining an image to be identified based on the previously obtained crystal image of the crystal.

[0187] In some embodiments, after obtaining a new crystal segmentation result, the processor may fuse the previously obtained crystal segmentation results, and determine whether the fused crystal segmentation result represents a complete display of the crystal based on the completeness evaluation model. When the fused crystal segmentation result represents that the crystal is completely displayed, an image fusion may be performed on the crystal image based on the previously obtained crystal, and the fused crystal image may be determined as the image to be identified. When the fused crystal segmentation result represents that the crystal is not completely displayed, steps 750 to 770 may be repeatedly executed until the fused crystal segmentation result represents that the crystal is completely displayed.

[0188] In some embodiments, when it is necessary to determine processing solutions for multiple crystals, the processor may fuse the fully displayed images of the multiple crystals to obtain images to be identified corresponding to the multiple crystals.

[0189] Some embodiments of the present specification determine the integrity of the crystal in the crystal image to ensure that the crystal can be completely displayed in the image to be identified, so as to ensure the accuracy of image recognition of the image to be identified.

[0190] It should be noted that the description of the above-mentioned various processes is only for example and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the various processes under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0191] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0192] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0193] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0194] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0195] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0196] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.

[0197] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A crystal visual recognition processing method, characterized in that: include: Acquiring an image to be identified related to one or more crystals; Performing image recognition on the image to be recognized to determine recognition results of the one or more crystals, wherein the recognition results include at least one of size data, position data, and defect data of the one or more crystals; Based on the identification result, a treatment plan for the one or more crystals is determined.

2. The method according to claim 1, characterized in that The step of acquiring images to be identified related to one or more crystals comprises: Acquire the image to be identified of the tray containing the one or more crystals; The position data includes first position data of the one or more crystals on the tray, and performing image recognition on the image to be recognized to determine the recognition results of the one or more crystals includes: Performing image recognition on the image to be recognized to determine the size data and the first position data of the one or more crystals; The processing scheme includes a first adjustment scheme for adjusting the one or more crystals placed on the tray, and the processing scheme for determining the one or more crystals based on the recognition result includes: The first adjustment scheme is determined based on the configuration target of the material tray, the size data of the one or more crystals, and the first position data.

3. The method according to claim 1, characterized in that The defect data includes first defect data characterizing defects of the crystal itself, and performing image recognition on the image to be recognized to determine the recognition results of the one or more crystals includes: Performing image recognition on the image to be recognized to determine the size data and the first defect data of the one or more crystals; The processing scheme includes a cutting scheme for cutting the one or more crystals, and the determining the processing scheme for the one or more crystals based on the recognition result includes: obtaining a first cutting target for the one or more crystals; For each crystal, a cutting plan for the crystal is determined based on the size data of the crystal, the first defect data, and the first cutting target.

4. The method according to claim 3, characterized in that The determining of a cutting scheme of the crystal based on the size data of the crystal, the first defect data, and the first cutting target includes: Determine a plurality of candidate cutting schemes based on the size data of the crystal, the first defect data, and the first cutting target; For each candidate cutting solution, Determine a unit score of each cut crystal unit based on the candidate cutting scheme, the size data of the crystal, and the first defect data; Determine a solution score of the candidate cutting solution based on the unit scores of each crystal unit; Based on the plan scores of the candidate cutting plans, the cutting plan for the crystal is determined.

5. The method according to claim 4, characterized in that Determining the scheme score of the candidate cutting scheme based on the unit score of each crystal unit includes: Obtaining a second cutting target for cutting the crystal; Based on the second cutting target, determining the quantity weight and the quality weight of the crystal unit; The plan score of the candidate cutting plan is determined based on the quantity weight, the quality weight, the number of the crystal units, and the unit score.

6. The method according to claim 1, characterized in that The step of acquiring images to be identified related to one or more crystals comprises: Acquire the image to be identified related to a crystal array composed of a plurality of crystal units, wherein at least two adjacent crystal units in the crystal array have a reflection structure between them; The position data includes second position data characterizing the relative positions of the plurality of crystal units in the crystal array, the defect data includes second defect data and third defect data, the second defect data is data characterizing defects of the crystal units themselves in the crystal array, and the third defect data is data characterizing defects in the crystal array, and the image recognition is performed on the image to be recognized to determine the recognition results of the one or more crystals, including: Performing image recognition on the image to be recognized to determine the second position data, the second defect data and the third defect data; The processing scheme includes a second adjustment scheme for adjusting the crystal unit and / or the reflective structure in the crystal array, and the processing scheme for determining the one or more crystals based on the identification result includes: The second adjustment solution is determined based on the second position data, the second defect data and the third defect data.

7. The method according to claim 6, characterized in that The method further comprises: Based on the second adjustment scheme, the crystal units and / or the reflective structures in the crystal array are adjusted by a robotic arm to complete the packaging of the crystal array.

8. The method according to claim 1, characterized in that The step of acquiring images to be identified related to one or more crystals comprises: Acquire the to-be-identified images of the one or more crystals under light sources of multiple different colors; The performing image recognition on the image to be recognized and determining the recognition result of the one or more crystals comprises: Performing image recognition on the to-be-recognized images corresponding to the multiple light sources of different colors to determine multiple candidate recognition results of the one or more crystals; Based on the plurality of candidate identification results, identification results of the one or more crystals are determined.

9. The method according to claim 1, characterized in that The step of acquiring images to be identified related to one or more crystals comprises: For each crystal, determine the crystal color of the crystal; The complementary color corresponding to the color of the crystal is used as a light source to illuminate the crystal, and an image to be identified corresponding to the crystal is obtained.

10. The method according to claim 1, characterized in that The step of acquiring images to be identified related to one or more crystals comprises: For each of the one or more crystals, acquiring a crystal image of the crystal under an initial light source; Performing image recognition on the crystal image to obtain a crystal segmentation result of the crystal; When the crystal segmentation result indicates that the crystal is complete, determining the crystal image as the image to be identified of the crystal; When the crystal segmentation result indicates that the crystal is not completely displayed, Repeat the process of adjusting the light source angle, obtaining a crystal image of the crystal under the adjusted light source, performing image recognition on the crystal image under the adjusted light source, obtaining a new crystal segmentation result, and fusing the previously obtained crystal segmentation results until the fused crystal segmentation result represents a complete display of the crystal, and then determining the image to be identified based on the previously obtained crystal image of the crystal.

Images