A processing method and device for glass detection and a terminal device

Abstract

The application is suitable for the technical field of machine vision, and provides a processing method and device for glass detection and terminal equipment, the method comprising: in response to a detection instruction of a user on a to-be-detected glass, acquiring original image data of the to-be-detected glass, wherein the original image data comprises red-green-blue image data and / or ultraviolet-visible light image data, and the to-be-detected glass is photovoltaic glass in a molten state; performing feature extraction on the original image data to obtain target feature data of the to-be-detected glass; performing analysis on the target feature data to obtain a detection result of the to-be-detected glass, and the detection result is used to represent whether there is a foreign matter in the to-be-detected glass. The method can penetrate the surface layer of the glass liquid to realize foreign matter detection of the glass liquid by performing feature extraction on the red-green-blue image data and / or ultraviolet-visible light image data of the to-be-detected glass, avoid missed detection caused by visual fatigue, improve the efficiency and accuracy of glass detection, and improve product quality.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of machine vision, and particularly relates to a glass detection processing method and device and a terminal device. BACKGROUND

[0002] Photovoltaic glass is a material with high transparency, and the glass liquid maintains the property of being clear and transparent at an extremely high temperature, so that relevant personnel cannot directly detect it at close range.

[0003] The existing glass detection processing method mainly adds monitoring equipment at the front end, and the detection personnel observes through the display screen at the back end.

[0004] However, the above glass detection processing method mainly relies on artificial vision for observation and judgment, and is prone to cause missed detection under the influence of visual fatigue, especially when facing complex defects, which is more difficult to deal with, resulting in low detection efficiency and high missed detection rate, and seriously affecting product quality. SUMMARY

[0005] The embodiments of the present application provide a glass detection processing method, device and terminal device, which can avoid missed detection caused by visual fatigue, improve the efficiency and accuracy of glass detection, and thus improve product quality.

[0006] In a first aspect, the embodiments of the present application provide a glass detection processing method, comprising:

[0007] In response to a detection instruction of a to-be-detected glass of a user, original image data of the to-be-detected glass is acquired, wherein the original image data includes red-green-blue image data and / or ultraviolet-visible light image data, and the to-be-detected glass is photovoltaic glass in a molten state;

[0008] Feature extraction is performed on the original image data to obtain target feature data of the to-be-detected glass;

[0009] The target feature data is analyzed to obtain a detection result of the to-be-detected glass, and the detection result is used to represent whether there is a foreign object in the to-be-detected glass.

[0010] In a possible implementation manner of the first aspect, the detection instruction carries a target algorithm identifier, the feature extraction is performed on the original image data to obtain the target feature data of the to-be-detected glass, and includes:

[0011] If the target algorithm identifier is detected as a first algorithm identifier, an improved YOLO target detection model is used to perform feature extraction on the original image data to obtain the target feature data of the to-be-detected glass, wherein the image fusion processing module in the improved YOLO target detection model includes a plurality of transformer networks, and the image fusion processing module is used for image data fusion processing.

[0012] If the target algorithm identifier is detected as the second algorithm identifier, feature extraction is performed on the original image data according to historical image data of the glass to be detected to obtain target feature data of the glass to be detected, wherein the historical image data and the original image data are image data of the glass to be detected at different time nodes.

[0013] In a possible implementation manner of the first aspect, the original image data includes red-green-blue image data and ultraviolet-visible light image data, and the feature extraction is performed on the original image data by using the improved YOLO target detection model to obtain the target feature data of the glass to be detected, and the feature extraction includes:

[0014] The red-green-blue image data and the ultraviolet-visible light image data are respectively preprocessed to obtain first processed image data and second processed image data, wherein the preprocessing at least includes cropping processing.

[0015] The improved YOLO target detection model is used to perform feature extraction on the first processed image data and the second processed image data to obtain extracted image data.

[0016] The extracted image data is subjected to format conversion processing to obtain the target feature data of the glass to be detected, and the format conversion processing is used to convert the format of the extracted image data into a preset format.

[0017] In a possible implementation manner of the first aspect, the improved YOLO target detection model is used to perform feature extraction on the first processed image data and the second processed image data to obtain extracted image data, and the feature extraction includes:

[0018] The first processed image data and the second processed image data are respectively subjected to convolution processing to obtain third processed image data and fourth processed image data.

[0019] The third processed image data and the fourth processed image data are input into an image fusion processing module to obtain first fusion image data and second fusion image data, wherein the first processed image data and the first fusion image data have the same spectral value, and the second processed image data and the second fusion image data have the same spectral value.

[0020] XOR calculation is performed on the third processed image data and the first fusion image data to obtain fifth processed image data, and XOR calculation is performed on the fourth processed image data and the second fusion image data to obtain sixth processed image data.

[0021] The extracted image data is calculated according to the fifth processed image data and the sixth processed image data, and the extracted image data at least includes a gray value.

[0022] In a possible implementation manner of the first aspect, the third processed image data and the fourth processed image data are input into the image fusion processing module to obtain the first fused image data and the second fused image data, including:

[0023] The third processed image data and the fourth processed image data are spliced to obtain spliced image data;

[0024] The spliced image data is XOR calculated with the preset position encoding to obtain first calculation data;

[0025] The first calculation data is sequentially input into the plurality of transformer networks in the image fusion processing module to obtain target fused image data;

[0026] The target fused image data is split to obtain the first fused image data and the second fused image data.

[0027] In a possible implementation manner of the first aspect, each transformer network includes a first normalization layer, a multi-head attention mechanism unit, a second normalization layer, and a multi-layer perception machine, and the first calculation data is sequentially input into the plurality of transformer networks in the image fusion processing module to obtain the target fused image data, including:

[0028] The first calculation data is normalized based on the first normalization layer to obtain second calculation data;

[0029] The second calculation data is processed by the multi-head attention mechanism unit to obtain third calculation data;

[0030] The first calculation data is XOR calculated with the third calculation data to obtain fourth calculation data;

[0031] The fourth calculation data is normalized based on the second normalization layer to obtain fifth calculation data;

[0032] The fifth calculation data is processed by the multi-layer perception machine to obtain sixth calculation data;

[0033] The fifth calculation data is XOR calculated with the sixth calculation data to obtain seventh calculation data;

[0034] The seventh calculation data is taken as the first calculation data, and the step of normalizing the first calculation data based on the first normalization layer to obtain the second calculation data is returned to be executed until the first calculation data is input into all transformer networks to obtain the target fused image data.

[0035] In one possible implementation of the first aspect, the original image data includes red-green-blue image data or ultraviolet-visible light image data, and the target feature data includes the number of regions in the target detection area, as well as the moving distance of the target pixels in the target detection area and the number of successful pixel movements;

[0036] Based on historical image data of the glass to be inspected, feature extraction is performed on the original image data to obtain the target feature data of the glass to be inspected, including:

[0037] Calculate the grayscale difference between the original pixels at the same position in the original image data and the historical image data to obtain the difference image;

[0038] Based on the grayscale difference of the original pixels in the difference image, the original pixels in the difference image are merged to generate candidate detection regions of the original image data.

[0039] The candidate detection regions of the original image data are matched with the candidate detection regions of the historical image data, and the candidate detection regions that intersect are identified as the target detection regions.

[0040] Determine the number of regions in the target detection area, as well as the pixel coordinates and movement distance of the target pixels within the target detection area;

[0041] For each target pixel within the target detection area, determine whether each target pixel conforms to the movement law of the foreign object based on the pixel coordinates of each target pixel;

[0042] The number of target pixels that conform to the motion law of the foreign object is determined as the number of successful pixel motions.

[0043] In one possible implementation of the first aspect, after analyzing the target feature data to obtain the detection result of the glass to be detected, the method further includes:

[0044] If the test results indicate that there is a foreign object inside the glass to be tested, an alarm message is generated for the glass to be tested. The alarm message is used to prompt the user to check the glass to be tested.

[0045] Foreign objects in the original image data are marked based on target feature data to obtain marked image data;

[0046] A video recording command is sent to the image acquisition device, which instructs the image acquisition device to record a video of the glass to be inspected.

[0047] Secondly, embodiments of this application provide a glass inspection processing apparatus, comprising:

[0048] The acquisition module is used to acquire the original image data of the glass to be tested in response to the user's test command. The original image data includes red, green and blue image data and / or ultraviolet and visible light image data. The glass to be tested is photovoltaic glass in a molten state.

[0049] The feature extraction module is used to extract features from the original image data to obtain the target feature data of the glass to be detected;

[0050] The analysis module is used to analyze the target feature data to obtain the detection results of the glass to be tested. The detection results are used to characterize whether there are foreign objects inside the glass to be tested.

[0051] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any of the first aspects.

[0052] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as described in any of the first aspects.

[0053] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute any of the methods described in the first aspect above.

[0054] This application provides a glass inspection processing method, apparatus, and terminal device. The method includes: responding to a user's inspection command for the glass to be inspected, acquiring original image data of the glass to be inspected, wherein the original image data includes red-green-blue image data and / or ultraviolet-visible light image data, and the glass to be inspected is photovoltaic glass in a molten state; extracting features from the original image data to obtain target feature data of the glass to be inspected; and analyzing the target feature data to obtain an inspection result for the glass to be inspected, the inspection result being used to characterize whether foreign objects exist inside the glass to be inspected. Using the above technical solution, by extracting features from the red-green-blue image data and / or ultraviolet-visible light image data of the glass to be inspected to obtain target feature data, it is possible to penetrate the surface of the molten glass to detect foreign objects in the molten glass. This avoids missed detections due to visual fatigue while improving the efficiency and accuracy of glass inspection, thereby improving product quality. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a schematic flowchart of a glass inspection processing method provided in an embodiment of this application;

[0057] Figure 2 This is a schematic flowchart of a glass inspection processing method provided in another embodiment of this application;

[0058] Figure 3 This is a schematic diagram of the overall process of feature extraction provided in an embodiment of this application;

[0059] Figure 4 This is a schematic diagram of the overall process of an image fusion processing module provided in one embodiment of this application;

[0060] Figure 5 This is a flowchart illustrating a detection algorithm based on a non-network model provided in an embodiment of this application;

[0061] Figure 6 This is a schematic flowchart of a multi-threaded processing method provided in an embodiment of this application;

[0062] Figure 7 This is a structural block diagram of a glass inspection processing device provided in one embodiment of this application;

[0063] Figure 8 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0064] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0065] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0066] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0067] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0068] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0069] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0070] The glass detection processing method provided in this application embodiment can be applied to terminal devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application embodiment does not impose any restrictions on the specific type of terminal device.

[0071] It can be argued that current glass inspection methods mainly rely on manual visual observation and judgment, which is prone to missed detection due to visual fatigue. This is especially true when dealing with complex defects, resulting in low inspection efficiency, a high rate of missed detection, and difficulty in adapting to high-speed production lines. Ultimately, this leads to defects not being detected and handled in a timely manner.

[0072] Meanwhile, current glass inspection methods can only observe larger surface defects and are difficult to capture inclusions inside molten glass; the algorithms involved are highly complex and lack real-time performance; they are sensitive to lighting conditions and are easily affected by the fluidity of molten glass and surface ripples.

[0073] Based on this, the embodiments of this application provide a glass inspection processing method. By introducing a visual inspection algorithm, it realizes automatic identification and alarm of foreign objects generated by photovoltaic glass liquid. This not only improves the accuracy and efficiency of inspection, but also greatly reduces the reliance on manual vision, ensuring the continuous and stable operation of the production line.

[0074] Figure 1 This is a schematic flowchart of a glass inspection processing method provided in one embodiment of this application. It is intended as an example and not a limitation. This method can be applied to terminal devices, such as... Figure 1 As shown, the method includes:

[0075] S101. In response to the user's inspection command for the glass to be inspected, acquire the original image data of the glass to be inspected.

[0076] The original image data includes red, green and blue image data and / or ultraviolet and visible light image data. The glass to be tested is photovoltaic glass in a molten state, that is, the glass to be tested can be understood as molten glass in a molten state at high temperature.

[0077] In this embodiment, the terminal device can be designed with a simple and intuitive UI to provide users with a convenient and efficient operating experience. For example, the entire UI can adopt a simple and clear layout style, mainly divided into three functional areas: a display box area, an input operation area, and a mode selection and control area. In the UI, users can enter a URL, select a mode, and then click the "Start" button. This triggers a detection command for the glass to be inspected, and the terminal device can respond to this command by acquiring the original image data of the glass. Subsequently, it can start the defect detection steps of the glass liquid in the video stream according to the selected mode. If the user wants to stop the detection, they can click the "Stop" button in the UI.

[0078] S102. Extract features from the original image data to obtain the target feature data of the glass to be detected.

[0079] Target feature data can be used to characterize the specific features of the glass to be tested, and different target feature data can characterize different features of the glass to be tested.

[0080] After obtaining the original image data of the glass to be inspected, feature extraction can be performed on the original image data to obtain the target feature data of the glass to be inspected. The means and process of feature extraction in this embodiment are not limited. For example, feature extraction can be performed on the original image data according to the algorithm specified by the user. Different algorithms may extract different target feature data. Alternatively, the terminal device can perform feature extraction on the original image data according to a preset strategy. The preset strategy can be a pre-configured feature extraction strategy. For example, feature extraction on the original image data can be performed directly through a neural network model, or the target feature data of the glass to be inspected can be obtained by performing certain calculations on the original image data. This embodiment does not limit this.

[0081] S103. Analyze the target feature data to obtain the detection results of the glass to be tested. The detection results are used to characterize whether there are foreign objects inside the glass to be tested.

[0082] This step analyzes the target feature data to obtain the detection results of the glass to be tested. For example, it can detect whether there are foreign objects such as bubbles or impurities in the glass. The specific analysis process can be achieved by comparing the target feature data with preset standards to determine whether there are foreign objects in the glass. Different target feature data can correspond to different preset standards.

[0083] Furthermore, this embodiment can also correspond to different processing procedures for different detection results. For example, when the detection result indicates that there are no foreign objects in the glass to be detected, it means that there are currently no defects in the glass to be detected. In this case, the process can return to the step of obtaining the original image data of the glass to be detected and continue to detect and process the next original image data of the glass to be detected.

[0084] In some embodiments, after analyzing the target feature data to obtain the detection result of the glass to be detected, the method further includes:

[0085] If the test results indicate that there is a foreign object inside the glass to be tested, an alarm message is generated for the glass to be tested. The alarm message is used to prompt the user to check the glass to be tested.

[0086] Foreign objects in the original image data are marked based on target feature data to obtain marked image data;

[0087] A video recording command is sent to the image acquisition device, which instructs the image acquisition device to record a video of the glass to be inspected.

[0088] In a specific implementation, if the detection result indicates the presence of foreign objects within the glass to be tested, a corresponding alarm mechanism can be executed. For example, a voice alarm can be triggered to remind relevant personnel to handle the situation; the target location can be marked on the video screen to obtain marked image data; video recording can also be started to save the video of the detection result, such as sending a video recording command to the image acquisition device to instruct the image acquisition device to record the video of the glass to be tested.

[0089] This embodiment provides a glass inspection processing method. In response to a user's inspection command for the glass to be inspected, the method acquires the original image data of the glass, including red-green-blue image data and / or ultraviolet-visible light image data. The glass to be inspected is photovoltaic glass in a molten state. Feature extraction is performed on the original image data to obtain target feature data of the glass to be inspected. The target feature data is analyzed to obtain the inspection result of the glass to be inspected. The inspection result is used to characterize whether foreign objects exist within the glass. Using this method, by extracting features from the red-green-blue image data and / or ultraviolet-visible light image data of the glass to be inspected, the method obtains the target feature data of the glass to be inspected. This allows for the detection of foreign objects in the molten glass by penetrating the surface layer of the molten glass. It avoids missed detections due to visual fatigue, improves the efficiency and accuracy of glass inspection, and thus enhances product quality.

[0090] Figure 2 This is a flowchart illustrating a glass detection processing method according to another embodiment of this application. In this embodiment, the detection instruction carries a target algorithm identifier, and the feature extraction of the original image data to obtain the target feature data of the glass to be detected is further optimized as follows: If the target algorithm identifier is detected as a first algorithm identifier, an improved YOLO target detection model is used to extract features from the original image data to obtain the target feature data of the glass to be detected. The improved YOLO target detection model includes an image fusion processing module comprising multiple transformer networks, which is used to perform image data fusion processing. If the target algorithm identifier is detected as a second algorithm identifier, features are extracted from the original image data based on the historical image data of the glass to be detected to obtain the target feature data of the glass to be detected. The historical image data and the original image data are image data of the glass to be detected at different time points, respectively. Figure 2 As shown, the method includes:

[0091] S201. In response to the user's inspection command for the glass to be inspected, acquire the original image data of the glass to be inspected.

[0092] S202. If the target algorithm identifier is detected as the first algorithm identifier, the improved YOLO target detection model is used to extract features from the original image data to obtain the target feature data of the glass to be detected.

[0093] The improved YOLO object detection model includes an image fusion processing module comprising multiple transformer networks, which is used to fuse image data.

[0094] In this embodiment, the detection instruction may carry a target algorithm identifier, which can be an identifier of a user-specified processing algorithm. For example, the target algorithm identifier could be a first algorithm identifier or a second algorithm identifier. The first algorithm identifier can be used to characterize a network-based detection algorithm. A network-based detection algorithm can be understood as an algorithm that uses a deep neural network to extract features from image data through a feature extraction module. For example, if the target algorithm identifier is detected as the first algorithm identifier, this embodiment can use an improved YOLO target detection model to extract features from the original image data to obtain the target feature data of the glass to be detected. This embodiment optimizes the network structure, specifically by redesigning the YOLO feature extraction network, aiming to improve operational efficiency and performance by simplifying the network structure. In the newly designed YOLO feature extraction network, the core module is the image fusion module, which mainly consists of three transformer networks. Based on this, while simplifying the network structure, the efficiency and accuracy of feature extraction are significantly improved, effectively reducing computational load and runtime.

[0095] More specifically, the following is a specific embodiment of feature extraction from raw image data using an improved YOLO object detection model.

[0096] In some embodiments, the original image data includes red-green-blue image data and ultraviolet-visible image data. An improved YOLO target detection model is used to extract features from the original image data to obtain target feature data of the glass to be detected, including:

[0097] The red-green-blue image data and the ultraviolet-visible light image data are preprocessed respectively to obtain the first processed image data and the second processed image data, wherein the preprocessing includes at least cropping.

[0098] An improved YOLO object detection model is used to extract features from the first and second processed image data to obtain the extracted image data.

[0099] The extracted image data is processed by format conversion to obtain the target feature data of the glass to be detected. The format conversion process is used to convert the format of the extracted image data into a preset format.

[0100] In a specific implementation, red-green-blue image data and ultraviolet-visible light images can be used as input data for preprocessing. For these two different modalities of image data, a cropping operation can be performed first. The purpose is to remove irrelevant edge parts in the image so that subsequent processing can focus on key areas. This also helps to reduce the amount of data processed and improve processing efficiency.

[0101] Furthermore, an improved YOLO object detection model can be used to extract features from the cropped RGB and UV-Vis images. The specific feature extraction process can be determined based on the structure of the improved YOLO object detection model.

[0102] In some embodiments, an improved YOLO object detection model is used to extract features from the first processed image data and the second processed image data to obtain extracted image data, including:

[0103] The first and second processed image data are convolved respectively to obtain the third and fourth processed image data.

[0104] The third processed image data and the fourth processed image data are input into the image fusion processing module to obtain the first fused image data and the second fused image data. The first processed image data and the first fused image data have the same spectral values, and the second processed image data and the second fused image data have the same spectral values.

[0105] The third processed image data and the first fused image data are XORed to obtain the fifth processed image data, and the fourth processed image data and the second fused image data are XORed to obtain the sixth processed image data.

[0106] Based on the image data processed in the fifth and sixth processes, the extracted image data is calculated, and the extracted image data includes at least grayscale values.

[0107] In a specific implementation, the improved YOLO object detection model may include convolutional layers and an image fusion processing module. For example, the first processed image data and the second processed image data may be convolved by the convolutional layers to obtain the third processed image data and the fourth processed image data. Then, the third processed image data and the fourth processed image data may be fused by the image fusion processing module to obtain the corresponding first fused image data and the second fused image data. Finally, the first fused image data and the second fused image data may be calculated by the convolutional layers to obtain the extracted image data, wherein the extracted image data includes at least grayscale values.

[0108] Figure 3This is a schematic diagram of the overall process of feature extraction provided in an embodiment of this application, as shown below. Figure 3 As shown, the RGB image and the UV-Vis image are first cropped separately. The cropped RGB image and UV-Vis image then enter their respective independent convolutional layer processing flows to obtain third and fourth processed image data. These third and fourth processed image data are then input into the image fusion module (i.e., the image fusion processing module) to obtain corresponding first and second fused image data. Subsequently, the third processed image data and the first fused image data are XORed to obtain fifth processed image data, and the fourth processed image data and the second fused image data are XORed to obtain sixth processed image data. The fifth and sixth processed image data again enter their respective independent convolutional layer processing flows. The image data obtained after convolutional calculations are XORed and then input into the output layer. The output layer performs final processing and transformation on the processed feature map, outputting it in a suitable form to form the final feature map (i.e., the extracted image data). This feature map contains rich feature information after the fusion of the RGB image and the UV-Vis image, which can be used for subsequent target detection, recognition, and other tasks.

[0109] Figure 4 This is a schematic diagram of the overall flow of an image fusion processing module provided in an embodiment of this application, as shown below. Figure 4 As shown, the specific processing flow of the image fusion processing module may include: firstly, deforming and stitching the third processed image data (Figure A) and the fourth processed image data (Figure B) to obtain stitched image data; then, performing an XOR operation between the stitched image data and a preset position code to obtain first calculated data; subsequently, the first calculated data can be sequentially input into multiple transformer networks in the image fusion processing module to obtain target fused image data; finally, the obtained target fused image data can be split to obtain first fused image data corresponding to the first processed image data (Figure A'), and second fused image data corresponding to the second processed image data (Figure B'). The preset position code can be a pre-configured position code.

[0110] More specifically, each transformer network includes a first normalization layer, a multi-head attention mechanism unit, a second normalization layer, and a multilayer perceptron. The first computational data is sequentially input into multiple transformer networks within the image fusion processing module. The process of obtaining the target fused image data can include: normalizing the first computational data based on the first normalization layer to obtain second computational data; performing attention mechanism processing on the second computational data using the multi-head attention mechanism unit to obtain third computational data; performing an XOR operation on the first and third computational data to obtain fourth computational data; normalizing the fourth computational data based on the second normalization layer to obtain fifth computational data; performing perceptual processing on the fifth computational data using the multilayer perceptron to obtain sixth computational data; performing an XOR operation on the fifth and sixth computational data to obtain seventh computational data; using the seventh computational data as the first computational data, and returning to execute the step of normalizing the first computational data based on the first normalization layer to obtain second computational data, until the first computational data is input into all transformer networks to obtain the target fused image data.

[0111] S203. If the target algorithm identifier is detected as the second algorithm identifier, then feature extraction is performed on the original image data based on the historical image data of the glass to be detected to obtain the target feature data of the glass to be detected.

[0112] Among them, historical image data and original image data are image data of the glass to be tested at different time points, respectively.

[0113] The second algorithm identifier can be used to characterize detection algorithms based on non-network models. Detection algorithms based on non-network models can include specially designed difference calculation and filtering algorithms. For example, the detection algorithm can be understood as an algorithm that extracts features by subtracting the current frame image data from the reference frame image data, and then removes interference (such as video jitter due to heat waves) through a filtering algorithm, thereby obtaining the target feature data.

[0114] Specifically, if the target algorithm identifier is detected as the second algorithm identifier, this embodiment can extract features from the original image data based on the historical image data of the glass to be tested, obtaining the target feature data of the glass to be tested. The historical image data and the original image data represent image data of the glass to be tested at different time points, respectively. Based on this, the difference between the current frame image data and the reference frame image data can be quickly calculated, accurately extracting the feature data that most likely represents internal defects in the molten glass. Simultaneously, through a built-in filtering algorithm, interference factors such as video jitter caused by heat waves can be effectively removed, ensuring that the extracted feature data truly reflects the internal defects of the molten glass, providing clean feature data for subsequent defect analysis.

[0115] In some embodiments, the raw image data includes red-green-blue image data or ultraviolet-visible image data, and the target feature data includes the number of regions in the target detection area, as well as the moving distance of the target pixels in the target detection area and the number of successful pixel movements;

[0116] Based on historical image data of the glass to be inspected, feature extraction is performed on the original image data to obtain the target feature data of the glass to be inspected, including:

[0117] Calculate the grayscale difference between the original pixels at the same position in the original image data and the historical image data to obtain the difference image;

[0118] Based on the grayscale difference of the original pixels in the difference image, the original pixels in the difference image are merged to generate candidate detection regions of the original image data.

[0119] The candidate detection regions of the original image data are matched with the candidate detection regions of the historical image data, and the candidate detection regions that intersect are identified as the target detection regions.

[0120] Determine the number of regions in the target detection area, as well as the pixel coordinates and movement distance of the target pixels within the target detection area;

[0121] For each target pixel within the target detection area, determine whether each target pixel conforms to the movement law of the foreign object based on the pixel coordinates of each target pixel;

[0122] The number of target pixels that conform to the motion law of the foreign object is determined as the number of successful pixel motions.

[0123] Figure 5 This is a flowchart illustrating a detection algorithm based on a non-network model, as provided in an embodiment of this application. Figure 5 As shown, the process begins by reading a single frame of image data, such as continuously reading frames from a video stream. The frame preceding the current frame is used as a reference frame. Parameters are initialized, and a detection region is created, covering only the surface of the molten glass. An interference mask is used to eliminate interference. Then, inter-frame subtraction is performed, such as calculating the difference in brightness (Y channel) of the same pixel within the detection region between the current and reference frames, resulting in a difference image. The difference is larger at the stone-like areas. Subsequently, pixels with large differences can be merged into blocks. Specifically, the difference sum can be calculated and graded in blocks, excluding blocks from the detection area. Adjacent pixels with the same difference intensity are then merged to generate candidate regions.

[0124] Secondly, search and matching can be performed, such as using linked list dynamic tracking technology for multi-frame matching. This involves matching the detection blocks corresponding to multiple frames to obtain a series of data on the changes of the detected target over time, which can then be used for subsequent motion analysis. The specific search and matching process can include traversing the linked list (i.e., candidate detection regions of the original image data and candidate detection regions of historical image data), comparing adjacent frame blocks based on the intersection condition. If a match is successful, the node information is updated, such as recording the size of the foreign object block (i.e., the number of regions) and the pixel position (i.e., pixel coordinates). If a match fails, a new node is created, which can be considered a false alarm, and the reading of the next frame continues.

[0125] Finally, motion analysis and judgment can be performed. For example, when the relevant data reaches a certain amount, it can be judged whether each pixel point conforms to the stone movement law (that is, based on the pixel coordinates of each target pixel point, it can be judged whether each target pixel point conforms to the foreign object movement law). For example, it can be judged whether the target is descending at a constant speed. If it does not conform to the stone movement law, the node can be processed, that is, it is considered a false alarm; if it conforms to the stone movement law, the success count can be increased, and after a certain number of times, it can be preliminarily judged as a stone.

[0126] Correspondingly, when analyzing target feature data, the number of matching attempts, the number of successful movements, and the descent distance can be used to make a judgment. If the preset standard requirements are met, it is determined to be a stone and an alarm is triggered, including voice broadcast, marking the stone, and saving the video.

[0127] S204. Analyze the target feature data to obtain the detection results of the glass to be tested. The detection results are used to characterize whether there are foreign objects inside the glass to be tested.

[0128] This embodiment provides a glass inspection processing method that extracts features from the original image data based on the different algorithms used by the user according to different production environments. This ensures that the target feature data is adaptable to different production environments, providing an accurate data foundation for subsequent analysis and further improving the accuracy and efficiency of glass inspection.

[0129] Figure 6 This is a flowchart illustrating a multi-threaded processing method according to an embodiment of this application, as shown below. Figure 6 The diagram illustrates the collaborative process of the main thread, decoding thread, and encoding thread, thereby achieving real-time detection, alerting, and recording of the video stream. Hardware encoding and other methods improve processing efficiency and response speed. The main thread handles the execution of the core detection algorithm and triggering of the alarm mechanism, primarily coordinating data interaction between the various modules.

[0130] The main thread can first be used to input URLs, such as when a user enters a URL, which is a crucial step in obtaining the video source. The main thread can also read a frame of image data, such as from the decoding queue, for subsequent detection operations. It can also detect the read image frames to determine if a target (such as a kidney stone or other object to be detected) exists. If a target is detected, the main thread can determine whether it has been detected. If no target is detected, it continues to read the next frame for detection; if a target is detected, it initiates an alarm and recording process. The main thread can also trigger alarms and record; for example, when a target is detected, the main thread can trigger an alarm and start the encoding thread to record the video.

[0131] The decoding thread can be used to capture video streams, which is the source of image data. The decoding thread can also be used to decode the captured video streams, converting the video data into processable image frames. The decoding thread can also be used to store the decoded image frames in a decoding queue, waiting for the main thread to read them. Each time the main thread reads the latest frame, it will clear the queue to ensure that the latest image data is processed each time, ensuring the smoothness and real-time performance of video frame transmission.

[0132] When a defect is detected, this embodiment will automatically start the encoding thread. The encoding thread can be used to mark the target after receiving the start signal from the main thread so that the target position can be clearly displayed in the video. The encoding thread can also be used to encode the video containing the target using hardware encoding, taking advantage of the high efficiency of hardware to reduce encoding latency. The encoding thread can also be used to save the encoded video for later viewing and analysis.

[0133] In addition, the glass inspection processing method in this embodiment involves several key components, each with unique functions and characteristics, providing strong support for the implementation of the visual inspection invention. Among them, the camera in the image acquisition module is a basic component. The high-resolution camera is responsible for capturing clear images and has high frame rate characteristics, which can meet the image acquisition needs of high-speed moving objects. Different types of cameras, such as industrial cameras and infrared cameras, can adapt to various environments and inspection targets, providing rich image information.

[0134] After receiving image data, the edge computing server uses a multimodal data fusion algorithm to extract features. Then, the decision-making module of the edge computing server can analyze and judge based on the extracted features, compare them with preset standards, and obtain detection results, such as determining whether bubbles or impurities exist in the molten glass.

[0135] The core of the edge computing server is a fusion algorithm chip, which possesses powerful computing capabilities and can quickly process data from multiple sensors. This chip can employ an advanced architecture design to optimize data processing flows, ensuring accurate fusion of different data and improving information integrity and accuracy.

[0136] Meanwhile, deep neural network chips can be used in network model-based detection algorithms, where they play a crucial role. These chips integrate a large number of neurons, possessing high-speed computing capabilities, enabling deep analysis of fused multimodal data and rapid, accurate extraction of key features from target objects. Furthermore, the chips support flexible programming, allowing users to easily adjust the structure and parameters of the neural network for different glass melt defect detection tasks, achieving efficient extraction of various complex defect features and significantly improving detection accuracy and adaptability.

[0137] The processor in the edge computing server can analyze and judge based on the extracted features, possessing intelligent decision-making algorithms that can quickly compare features with preset standards and make accurate judgments. The processor also supports a real-time feedback mechanism to output detection results promptly, ensuring the efficient operation of the detection process.

[0138] Furthermore, the implementation steps of the visual inspection invention can also cover both the operation process and parameter settings. In terms of the operation process, the image acquisition module can first be activated, and a water-cooled high-temperature camera with ≥8 megapixels can be deployed on-site according to the inspection requirements. After activation, the camera can continuously acquire image data at a set frame rate and transmit it to the edge computing server.

[0139] In terms of parameter settings, the image acquisition module needs to configure parameters such as camera resolution, frame rate, and exposure time to adapt to different detection scenarios; the edge computing server needs to adjust the weight parameters of the fusion algorithm to ensure the characteristics of the target object, and set parameters such as the number of layers and neurons in the neural network to optimize feature extraction; the decision-making module needs to set judgment thresholds to ensure the accuracy and reliability of the detection results. By reasonably setting these parameters, efficient and accurate visual detection can be achieved.

[0140] As described above, the glass inspection processing method of this embodiment overcomes the limitations of surface reflection through multispectral fusion technology, achieving online detection of internal defects in molten glass for the first time. The dual-mode selection allows it to adapt to various scenarios; for example, the lightweight model can adapt to complex environments and improve detection accuracy, while the non-network model detection algorithm can adapt to general environments, offering higher detection speed and lower latency.

[0141] Secondly, the glass inspection processing method in this embodiment combines high precision and low latency, such as lightweight model GPU acceleration, which meets the requirements of high-speed production lines (≥1m / s); the dynamic optical compensation algorithm reduces the influence of environmental temperature drift (accuracy ±0.1℃) and has strong anti-interference ability.

[0142] Meanwhile, in terms of video recording, the glass detection processing method in this embodiment fully utilizes the parallel computing power of the GPU by adopting GPU-based hardware encoding technology, further reducing encoding latency and enabling fast and high-quality saving of the detection video, providing strong support for subsequent data analysis and problem tracing.

[0143] Furthermore, the glass detection processing method in this embodiment adopts a multi-threaded parallel processing architecture, which greatly improves the overall operating efficiency and enables real-time detection, alarm and recording of video streams, effectively meeting the stringent requirements for real-time performance and high efficiency in industrial production.

[0144] Applying the glass inspection processing method of this embodiment to actual production applications yields the following quantitative indicators: defect detection rate ≥99.9%, false alarm rate ≤0.2%; detection speed up to 200 frames / second, supporting ≥5000 points per meter of glass plate; and system power consumption reduced by 40%. Qualitative effects include: the ability to identify defects as small as 0.1mm. 2 The following minor defects reduce the workload of personnel, decrease the probability of equipment damage, and assist in process optimization.

[0145] Therefore, the glass inspection method in this embodiment, through real-time online vision processing at the front end, can promptly detect foreign objects in the molten glass and issue an alarm. Based on the system prompts, timely human intervention and handling are carried out. Early detection and early handling not only improve the continuity of product production and product quality, but also prevent foreign objects in the molten glass from damaging subsequent production equipment, thereby saving production costs.

[0146] Corresponding to the glass inspection processing method in the above embodiments, Figure 7 This is a structural block diagram of a glass inspection processing device provided in one embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0147] Reference Figure 7 The device includes:

[0148] The acquisition module 301 is used to acquire the original image data of the glass to be tested in response to the user's test command. The original image data includes red, green and blue image data and / or ultraviolet and visible light image data. The glass to be tested is photovoltaic glass in a molten state.

[0149] Feature extraction module 302 is used to extract features from the original image data to obtain target feature data of the glass to be detected;

[0150] The analysis module 303 is used to analyze the target feature data to obtain the detection results of the glass to be tested. The detection results are used to characterize whether there are foreign objects inside the glass to be tested.

[0151] This embodiment provides a glass inspection processing device. In response to a user's inspection command for the glass to be inspected, an acquisition module acquires the original image data of the glass, including red, green, and blue (RGB) image data and / or ultraviolet-visible (UV-Vis) image data. The glass to be inspected is photovoltaic glass in a molten state. A feature extraction module extracts features from the original image data to obtain target feature data of the glass to be inspected. An analysis module analyzes the target feature data to obtain the inspection result of the glass to be inspected. The inspection result is used to characterize whether foreign objects exist within the glass. Using this device, by extracting features from the RGB and / or UV-Vis image data of the glass to be inspected to obtain target feature data, it can penetrate the surface of the molten glass to detect foreign objects. This avoids missed detections due to visual fatigue, improves the efficiency and accuracy of glass inspection, and thus enhances product quality.

[0152] Optionally, the detection instruction carries a target algorithm identifier, and the feature extraction module includes:

[0153] The first feature extraction unit is used to extract features from the original image data using the improved YOLO target detection model if the target algorithm identifier is detected as the first algorithm identifier, so as to obtain the target feature data of the glass to be detected. The image fusion processing module in the improved YOLO target detection model includes multiple transformer networks and is used to perform image data fusion processing.

[0154] The second feature extraction unit is used to extract features from the original image data based on the historical image data of the glass to be detected if the target algorithm identifier is detected as the second algorithm identifier, so as to obtain the target feature data of the glass to be detected. The historical image data and the original image data are image data of the glass to be detected at different time points, respectively.

[0155] Optionally, the original image data includes red-green-blue image data and ultraviolet-visible image data, and the first feature extraction unit includes:

[0156] The preprocessing subunit is used to preprocess the red-green-blue image data and the ultraviolet-visible image data respectively to obtain the first processed image data and the second processed image data, wherein the preprocessing includes at least cropping processing;

[0157] The extraction subunit is used to extract features from the first processed image data and the second processed image data using an improved YOLO object detection model to obtain the extracted image data.

[0158] The format conversion subunit is used to perform format conversion processing on the extracted image data to obtain the target feature data of the glass to be detected. The format conversion processing is used to convert the format of the extracted image data into a preset format.

[0159] Optionally, extracting sub-units is specifically used for:

[0160] The first and second processed image data are convolved respectively to obtain the third and fourth processed image data.

[0161] The third processed image data and the fourth processed image data are input into the image fusion processing module to obtain the first fused image data and the second fused image data. The first processed image data and the first fused image data have the same spectral values, and the second processed image data and the second fused image data have the same spectral values.

[0162] The third processed image data and the first fused image data are XORed to obtain the fifth processed image data, and the fourth processed image data and the second fused image data are XORed to obtain the sixth processed image data.

[0163] Based on the image data processed in the fifth and sixth processes, the extracted image data is calculated, and the extracted image data includes at least grayscale values.

[0164] Optionally, the third processed image data and the fourth processed image data are input into the image fusion processing module to obtain the first fused image data and the second fused image data, including:

[0165] The third and fourth processed image data are stitched together to obtain stitched image data;

[0166] The stitched image data is XORed with the preset position code to obtain the first calculated data;

[0167] The first calculation data is sequentially input into multiple transformer networks in the image fusion processing module to obtain the target fused image data;

[0168] The target fused image data is split to obtain the first fused image data and the second fused image data.

[0169] Optionally, each transformer network includes a first normalization layer, a multi-head attention mechanism unit, a second normalization layer, and a multilayer perceptron. The first computational data is sequentially input into multiple transformer networks in the image fusion processing module to obtain target fused image data, including:

[0170] The first calculation data is normalized based on the first normalization layer to obtain the second calculation data;

[0171] The second computational data is processed using a multi-head attention mechanism unit to obtain the third computational data;

[0172] Perform an XOR operation on the first and third calculated data to obtain the fourth calculated data;

[0173] The fourth calculation data is normalized based on the second normalization layer to obtain the fifth calculation data;

[0174] A multilayer perceptron is used to process the fifth computational data to obtain the sixth computational data.

[0175] Perform an XOR operation on the fifth and sixth calculated data to obtain the seventh calculated data;

[0176] The seventh computational data is used as the first computational data, and the process of normalizing the first computational data based on the first normalization layer is returned to obtain the second computational data, until the first computational data is input into all transformer networks to obtain the target fused image data.

[0177] Optionally, the original image data includes red-green-blue image data or ultraviolet-visible image data, and the target feature data includes the number of regions in the target detection area, as well as the moving distance of the target pixels in the target detection area and the number of successful pixel movements;

[0178] The second feature extraction unit includes:

[0179] Calculate the grayscale difference between the original pixels at the same position in the original image data and the historical image data to obtain the difference image;

[0180] Based on the grayscale difference of the original pixels in the difference image, the original pixels in the difference image are merged to generate candidate detection regions of the original image data.

[0181] The candidate detection regions of the original image data are matched with the candidate detection regions of the historical image data, and the candidate detection regions that intersect are identified as the target detection regions.

[0182] Determine the number of regions in the target detection area, as well as the pixel coordinates and movement distance of the target pixels within the target detection area;

[0183] For each target pixel within the target detection area, determine whether each target pixel conforms to the movement law of the foreign object based on the pixel coordinates of each target pixel;

[0184] The number of target pixels that conform to the motion law of the foreign object is determined as the number of successful pixel motions.

[0185] Optionally, the glass inspection processing apparatus provided in this embodiment further includes:

[0186] The alarm generation module is used to generate alarm information for the glass under test after analyzing the target feature data and obtaining the detection result of the glass under test. If the detection result indicates that there is a foreign object in the glass under test, the alarm information is used to prompt the user to check the glass under test.

[0187] The marking module is used to mark foreign objects in the original image data based on the target feature data after analyzing the target feature data and obtaining the detection result of the glass to be detected, so as to obtain marked image data.

[0188] The recording module is used to send a video recording command to the image acquisition device after analyzing the target feature data and obtaining the detection result of the glass to be inspected. The video recording command is used to instruct the image acquisition device to record the video of the glass to be inspected.

[0189] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0190] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0191] This application also provides a terminal device. Figure 8 This is a schematic diagram of the structure of a terminal device provided in one embodiment of this application, as shown below. Figure 8 As shown, the terminal device includes: at least one processor 401, a memory 402, an input device 403, an output device 404, and a computer program stored in the memory 402 and executable on at least one processor 401. When the processor 401 executes the computer program, it implements the steps in any of the above-described method embodiments.

[0192] Input device 403 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the terminal device. Output device 404 may include display devices such as a display screen.

[0193] This application also provides a computer-readable storage medium storing a computer program, which, when executed by processor 401, can implement the steps in the above-described method embodiments.

[0194] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments.

[0195] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 401, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable storage media cannot be electrical carrier signals or telecommunication signals.

[0196] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0197] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0198] In the embodiments provided in this application, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0199] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0200] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for glass inspection, characterized in that, include: In response to a user's inspection command for the glass to be inspected, the system acquires the original image data of the glass to be inspected, wherein the original image data includes red, green and blue image data and / or ultraviolet and visible light image data, and the glass to be inspected is photovoltaic glass in a molten state; Feature extraction is performed on the original image data to obtain the target feature data of the glass to be detected. The target feature data is determined based on the target algorithm identifier carried in the detection instruction. The target algorithm identifier includes a first algorithm identifier or a second algorithm identifier. The first algorithm identifier is used to identify the algorithm that uses an improved YOLO target detection model to extract features from the image data. The second algorithm identifier is used to identify the algorithm that extracts features by performing difference calculation on the image data and then removes interference by a filtering algorithm. The target feature data is analyzed to obtain the detection result of the glass to be tested. The detection result is used to characterize whether there are foreign objects inside the glass to be tested.

2. The glass testing method as described in claim 1, characterized in that, The detection instruction carries a target algorithm identifier, and the step of extracting features from the original image data to obtain the target feature data of the glass to be detected includes: If the target algorithm identifier is detected as the first algorithm identifier, the improved YOLO target detection model is used to extract features from the original image data to obtain the target feature data of the glass to be detected. The image fusion processing module in the improved YOLO target detection model includes multiple transformer networks, and the image fusion processing module is used to perform image data fusion processing. If the target algorithm identifier is detected as the second algorithm identifier, then feature extraction is performed on the original image data based on the historical image data of the glass to be detected to obtain the target feature data of the glass to be detected, wherein the historical image data and the original image data are image data of the glass to be detected at different time points.

3. The glass testing method as described in claim 2, characterized in that, The original image data includes red-green-blue image data and ultraviolet-visible light image data. The improved YOLO target detection model is used to extract features from the original image data to obtain the target feature data of the glass to be detected, including: The red-green-blue image data and the ultraviolet-visible image data are preprocessed respectively to obtain first processed image data and second processed image data, wherein the preprocessing includes at least cropping. An improved YOLO object detection model is used to extract features from the first processed image data and the second processed image data to obtain the extracted image data. The extracted image data is subjected to format conversion processing to obtain the target feature data of the glass to be detected. The format conversion processing is used to convert the format of the extracted image data into a preset format.

4. The glass testing method as described in claim 3, characterized in that, The improved YOLO object detection model is used to extract features from the first processed image data and the second processed image data to obtain extracted image data, including: The first processed image data and the second processed image data are convolved respectively to obtain the third processed image data and the fourth processed image data. The third processed image data and the fourth processed image data are input into the image fusion processing module to obtain first fused image data and second fused image data, wherein the first processed image data and the first fused image data have the same spectral values, and the second processed image data and the second fused image data have the same spectral values. The third processed image data and the first fused image data are XORed to obtain the fifth processed image data, and the fourth processed image data and the second fused image data are XORed to obtain the sixth processed image data. Based on the fifth processed image data and the sixth processed image data, the extracted image data is calculated, and the extracted image data includes at least grayscale values.

5. The glass testing method as described in claim 4, characterized in that, The step of inputting the third processed image data and the fourth processed image data into the image fusion processing module to obtain the first fused image data and the second fused image data includes: The third processed image data and the fourth processed image data are stitched together to obtain stitched image data; The stitched image data is XORed with a preset position code to obtain the first calculated data; The first calculated data is sequentially input into multiple transformer networks in the image fusion processing module to obtain the target fused image data; The target fused image data is split to obtain first fused image data and second fused image data.

6. The glass testing method as described in claim 5, characterized in that, Each transformer network includes a first normalization layer, a multi-head attention mechanism unit, a second normalization layer, and a multilayer perceptron. The first computational data is sequentially input into multiple transformer networks in the image fusion processing module to obtain target fused image data, including: The first calculated data is normalized based on the first normalization layer to obtain the second calculated data; The second computational data is processed using the multi-head attention mechanism unit to obtain the third computational data; Perform an XOR operation between the first calculated data and the third calculated data to obtain the fourth calculated data; The fourth calculation data is normalized based on the second normalization layer to obtain the fifth calculation data; The multilayer perceptron is used to process the fifth calculation data to obtain the sixth calculation data; Perform an XOR operation on the fifth and sixth calculated data to obtain the seventh calculated data; The seventh computational data is used as the first computational data, and the process of normalizing the first computational data based on the first normalization layer to obtain the second computational data is repeated until the first computational data is input into all transformer networks to obtain the target fused image data.

7. The glass testing method as described in claim 2, characterized in that, The original image data includes red-green-blue image data or ultraviolet-visible light image data, and the target feature data includes the number of regions in the target detection area, as well as the moving distance and the number of successful pixel movements in the target detection area. The step of extracting features from the original image data based on the historical image data of the glass to be tested to obtain the target feature data of the glass to be tested includes: Calculate the grayscale difference between the original image data and the original pixels at the same position in the historical image data to obtain a difference image; Based on the grayscale difference of the original pixels in the difference image, the original pixels in the difference image are merged to generate candidate detection regions of the original image data; The candidate detection regions of the original image data are matched with the candidate detection regions of the historical image data, and the candidate detection regions that intersect are determined as the target detection regions. Determine the number of regions in the target detection area, as well as the pixel coordinates and moving distance of the target pixels within the target detection area; For each target pixel within the target detection area, determine whether each target pixel conforms to the movement law of the foreign object based on the pixel coordinates of each target pixel; The number of target pixels that conform to the motion law of the foreign object is determined as the number of successful pixel motions.

8. The glass testing method as described in claim 1, characterized in that, After analyzing the target feature data to obtain the detection result of the glass to be detected, the method further includes: If the detection result indicates that there is a foreign object inside the glass to be tested, an alarm message is generated for the glass to be tested, and the alarm message is used to prompt the user to check the glass to be tested; Based on the target feature data, foreign objects in the original image data are marked to obtain marked image data; A video recording command is sent to the image acquisition device, which instructs the image acquisition device to record a video of the glass to be inspected.

9. A glass inspection processing device, characterized in that, include: The acquisition module is used to acquire the original image data of the glass to be tested in response to the user's test command. The original image data includes red, green and blue image data and / or ultraviolet and visible light image data. The glass to be tested is photovoltaic glass in a molten state. The feature extraction module is used to extract features from the original image data to obtain target feature data of the glass to be detected. The target feature data is determined based on the target algorithm identifier carried in the detection instruction. The target algorithm identifier includes a first algorithm identifier or a second algorithm identifier. The first algorithm identifier is used to identify the algorithm that uses an improved YOLO target detection model to extract features from the image data. The second algorithm identifier is used to identify the algorithm that extracts features by performing difference calculation on the image data and then removes interference by a filtering algorithm. The analysis module is used to analyze the target feature data to obtain the detection result of the glass to be tested, and the detection result is used to characterize whether there are foreign objects inside the glass to be tested.

10. A terminal device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the terminal device to implement the method as described in any one of claims 1-8.

Images