Screen scratch detection method and apparatus, system, electronic device, and storage medium

Abstract

A screen scratch detection method and apparatus, a system, an electronic device, and a storage medium, relating to the technical field of display. The method comprises: receiving a plurality of screen images transmitted by an image acquisition device, the plurality of screen images being obtained by the image acquisition device triggered, in response to an inspection start signal, to continuously photograph a screen to be inspected, said screen moving in a specified direction during the continuous photographing process; storing the plurality of screen images; and sequentially performing scratch detection on the plurality of screen images to obtain a detection result of the screen to be inspected, the photographing of the screen images and the scratch detection being asynchronously performed, and the storage of the screen images and the scratch detection being performed in parallel. The present method can improve the the accuracy and efficiency of detecting scratches on screens, thereby improving production efficiency.

Description

Screen scratch detection method and device, system, electronic device and storage medium Technical Field

[0001] The present disclosure relates to the field of display technology, and in particular to a screen scratch detection method and device, a system, an electronic device, and a storage medium. Background Art

[0002] During the production process of display screens, scratch defects may occur on the products due to various reasons. To ensure product quality, such defects are currently mainly detected through manual screening. However, this detection method not only has a high missed detection rate but also has low detection efficiency, which affects production efficiency.

[0003] Summary of the Invention

[0004] According to a first aspect of the present disclosure, a screen scratch detection method is provided, comprising:

[0005] receiving a plurality of screen images transmitted by an image acquisition device, wherein the plurality of screen images are obtained by continuously photographing a screen to be inspected by the image acquisition device in response to a detection start signal; the screen to be inspected moves along a specified direction during the continuous photographing process;

[0006] storing a plurality of the screen images;

[0007] Performing scratch detection on the plurality of screen images in sequence to obtain detection results of the screen to be detected;

[0008] The capturing of the screen image and the scratch detection are performed asynchronously, and the storing of the screen image and the scratch detection are performed in parallel.

[0009] Optionally, storing the plurality of screen images includes: storing the plurality of screen images into an image queue;

[0010] The sequentially performing scratch detection on the plurality of screen images includes:

[0011] In response to the first screen image being stored in the image queue, the scratch detection is triggered.

[0012] Optionally, the screen image includes a first image, and storing multiple screen images transmitted by the image acquisition device includes: storing the first image in a pre-applied target storage space in a target format, wherein the first image is multiple screen images taken for the first area of ​​the screen to be detected.

[0013] Optionally, before receiving the multiple screen images transmitted by the image acquisition device, the method further includes:

[0014] Acquire the size of the first area, the moving speed of the screen to be detected, and the shooting frequency of the image acquisition device;

[0015] determining the number of screen images in the first image according to the size of the first area, the moving speed of the screen to be detected, and the shooting frequency of the image acquisition device;

[0016] determining a storage space required for the first image according to the number of the screen images and the size of the screen image;

[0017] Apply for a memory corresponding to the storage space size as the target storage space.

[0018] Optionally, before storing the first image in the pre-applied target storage space in the target format, the method further includes: compressing the first image.

[0019] Optionally, storing the first image in a pre-applied target storage space in a target format includes:

[0020] For the plurality of screen images in the first image, sequentially determining the image size of the compressed image corresponding to each screen image;

[0021] storing the compressed images and the corresponding image sizes in the same target file in sequence;

[0022] In response to the last screen image captured for the first area being stored in the target file, determining information on the number of images in the target file;

[0023] The image quantity information is added to the header byte position of the target file.

[0024] Optionally, before continuously acquiring a plurality of screen images of the screen to be detected, the method further includes: performing direction correction on the screen to be detected.

[0025] Optionally, performing orientation correction on the screen to be detected includes:

[0026] Acquire a second image of a second area of ​​the screen to be detected;

[0027] Matching the second image with a candidate template to determine a matching template;

[0028] Determining the offset angle of the screen to be detected according to the matching template;

[0029] The offset angle is sent to a control end of the mobile platform so that the control end controls the rotation of the screen to be detected according to the offset angle.

[0030] Optionally, matching the second image with a candidate template to determine a matching template includes:

[0031] Get the original template of the screen to be detected;

[0032] The original template is rotated multiple times along the center point at certain angle intervals to obtain multiple candidate templates;

[0033] The second image is matched with each of the candidate templates respectively, and the candidate template that overlaps with the second area of ​​the second image is determined as the matching template.

[0034] Optionally, the detection start signal is generated in response to the image acquisition device acquiring identification information of the screen to be detected, and the identification information of the screen to be detected is acquired using a code reading platform.

[0035] Optionally, the sequentially performing scratch detection on the plurality of screen images includes: inputting the plurality of screen images into a target detection model for scratch detection, and outputting the detection results, wherein the target detection model is a trained image semantic segmentation network.

[0036] Optionally, inputting the plurality of screen images into an object detection model for scratch detection includes:

[0037] Performing feature extraction on the input screen image to obtain a shallow feature map and a deep feature map;

[0038] Performing dilated convolution and channel adjustment with different dilation rates on the deep feature map to obtain encoded output;

[0039] Performing feature adjustment based on an attention mechanism on the shallow feature map and the encoded output respectively to obtain a first feature map and a second feature map;

[0040] Performing feature fusion on the first feature map and the second feature map;

[0041] Perform feature adjustment based on the attention mechanism on the fusion result to obtain the third feature map;

[0042] The third feature map is convolved and upsampled to obtain the detection result.

[0043] Optionally, the feature adjustment based on the attention mechanism includes: performing channel attention adjustment on the target feature map; the target feature map is the shallow feature map, the encoding output or the fusion result; and performing spatial attention adjustment on the channel attention adjustment result.

[0044] Optionally, the method further includes: obtaining a training sample, wherein the training sample includes a screen image with scratches; inputting the training sample into an image semantic segmentation network to obtain a training output; using a dynamic scaling factor to determine the cross-entropy loss of the training output and the sample label; and adjusting the model parameters of the image semantic segmentation network based on the cross-entropy loss.

[0045] Optionally, the method further includes: when the detection result indicates that scratches exist on the screen to be detected, determining a scratch area; and performing a secondary detection on the scratch area to determine a final detection result.

[0046] According to a second aspect of the present disclosure, a scratch detection system is provided, including a first detection platform, wherein the first detection platform includes:

[0047] The mobile platform is loaded with the screen to be inspected and moves along a specified direction to transport the screen to be inspected;

[0048] an image acquisition device, in response to a detection start signal, continuously acquiring a plurality of screen images of the screen to be detected on the mobile platform;

[0049] The detection sub-platform receives and stores the multiple screen images transmitted by the image acquisition device; and performs scratch detection on the multiple screen images in sequence to obtain the detection results of the screen to be detected; wherein the storage of the screen images and the scratch detection are performed in parallel.

[0050] Optionally, the system includes: a first detection platform for performing scratch detection on a group of opposite edges of the screen to be detected; and a second detection platform for performing scratch detection on another group of opposite edges of the screen to be detected.

[0051] Optionally, the system further includes: a third detection platform, configured to perform a secondary detection on the scratch area determined by the first detection platform or the second detection platform.

[0052] Optionally, the system further includes: a screen transfer device for transporting and transferring the screen to be inspected between the first inspection platform, the second inspection platform and the third inspection platform.

[0053] According to a third aspect of the present disclosure, a screen scratch detection device is provided, comprising:

[0054] a trigger module configured to trigger the image acquisition device to continuously acquire multiple screen images of the screen to be detected in response to a detection start signal; the screen to be detected moves along a specified direction during the image acquisition process;

[0055] a storage module, configured to receive and store the plurality of screen images transmitted by the image acquisition device;

[0056] The detection module is configured to perform scratch detection on the plurality of screen images in sequence to obtain detection results of the screen to be detected; wherein the storage of the screen images and the scratch detection are performed in parallel.

[0057] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned methods by executing the executable instructions.

[0058] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any one of the methods described above is implemented.

[0059] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0061] FIG1 schematically shows a flow chart of a scratch detection method in an exemplary embodiment of the present disclosure;

[0062] FIG2 schematically shows a schematic diagram of the position of the second area on the screen in an exemplary embodiment of the present disclosure;

[0063] FIG3 schematically illustrates the time consumption of simultaneously capturing and storing a screen image and detecting scratches in an exemplary embodiment of the present disclosure;

[0064] FIG4 schematically illustrates a training process diagram of a target detection model in an exemplary embodiment of the present disclosure;

[0065] FIG5 schematically illustrates a process of detecting scratches on a screen image using an object detection model in an exemplary embodiment of the present disclosure;

[0066] FIG6 schematically illustrates a schematic diagram of a scratch detection system in an exemplary embodiment of the present disclosure;

[0067] FIG7 schematically illustrates a schematic diagram of the information interaction process between hardware and software in a scratch detection system in an exemplary embodiment of the present disclosure;

[0068] FIG8 schematically shows a structural diagram of a scratch detection device in an exemplary embodiment of the present disclosure;

[0069] FIG9 schematically shows an electronic device for implementing the above method in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0070] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0071] It should be noted that, in the present disclosure, the terms "including", "configured with", and "set to" are used to express an open-ended inclusion, and mean that in addition to the listed elements / components / etc., there may be additional elements / components / etc.; the terms "first", "second", etc. are only used as labels and are not limitations on the number or order of their objects.

[0072] In the actual production process of display screens, there is a process of cutting large screens into small ones. During this cutting process, scratches are easily generated due to stress, especially near the cutting line. The scratch incidence rate is relatively high. Therefore, it is necessary to focus on scratch detection at this location.

[0073] The scratch detection method of the embodiment of the present disclosure can be applied to a terminal device or a server. Referring to FIG1 , the scratch detection method may include the following steps:

[0074] S110, receiving multiple screen images transmitted by an image acquisition device, wherein the multiple screen images are obtained by triggering the image acquisition device to continuously photograph the screen to be detected in response to a detection start signal; the screen to be detected moves along a specified direction during the continuous photographing process.

[0075] S120: Store a plurality of the screen images.

[0076] S130, performing scratch detection on multiple screen images in sequence to obtain detection results of the screen to be detected; wherein, the shooting of the screen images and the scratch detection are performed asynchronously, and the storage of the screen images and the scratch detection are performed in parallel.

[0077] The beneficial effects of the embodiment of Figure 1 above are that, on the one hand, the scratch detection process of the screen image can be carried out in parallel with the storage of the screen image, thereby improving data processing efficiency and thus improving detection efficiency; on the other hand, the screen to be detected moves along a specified direction, and multiple screen images are continuously captured during the movement of the screen to be detected, thereby realizing a streaming dynamic detection process of the screen, ensuring detection accuracy while further improving detection efficiency; combined with the asynchronous capture of screen images and scratch detection, it is possible to fully utilize production resources, avoid waiting problems between different processing flows, further improve processing efficiency, and thus realize a pipeline detection process, which is convenient for integration with existing production processes and improves production efficiency.

[0078] In order to make the technical solution of the present disclosure clearer, the steps of the screen scratch detection method are described below.

[0079] In step S110, multiple screen images transmitted by an image acquisition device are received, where the multiple screen images are obtained by continuously photographing the screen to be detected by the image acquisition device in response to a detection start signal; the screen to be detected moves along a specified direction during the continuous photographing process.

[0080] In this exemplary embodiment, the detection start signal can be generated based on the user's specified operation or in response to the identification of the screen to be detected entering the detection system, or it can be generated in response to the detection instruction received from other devices in the system, and this example does not limit this. For example, the code reading platform can be used to obtain the identification information of the screen to be detected; in response to obtaining the identification information of the screen to be detected, a detection start signal is generated, and the image capture process is triggered by the detection signal, which facilitates the interaction between the terminal side and the control side, and ensures the immediate start and accurate control of the detection process. In this example, after the code reading platform identifies the ID of the screen to be detected, it sends the ID of the screen to be detected to the image acquisition device, and the image acquisition device receives the screen ID and generates a detection start signal. The detection end (such as a PC or server, etc.) can generate a reception preparation signal in response to the detection start signal, and receive multiple screen images (flying images) transmitted by the image acquisition device.

[0081] In this exemplary embodiment, the image acquisition device may include a camera for capturing screen images, which may have a corresponding control terminal (such as a PLC). For example, a positioning camera may be used to capture screen images. The screen to be inspected may be moved in a specified direction during the image acquisition process. For example, the screen to be inspected may be placed on a platform that can move along a guide rail, and the screen may be moved by moving the platform along the guide rail. In this way, images of different positions of the screen can be obtained by continuous shooting with the camera, thereby realizing the detection of all positions to be inspected on the screen.

[0082] For example, two cameras can be placed on the same vertical line. When the two ends of the screen reach below the cameras, a high-speed trigger can be used to trigger the camera to take a picture until the screen moves out of the camera's field of view. This high-speed trigger can reduce the response delay of the photo (less than 1us), and the pixel error of the captured image is less than 10 pixels, ensuring image quality.

[0083] In some embodiments, in order to ensure the accuracy of the position of the captured screen, the direction of the screen to be detected is corrected before continuously capturing multiple screen images of the screen to be detected. For example, the direction correction can be used to make the edge of the screen to be detected parallel to the direction of screen movement. This can ensure the accuracy of the position of the captured screen during the screen movement shooting process. At the same time, it can ensure that each frame of the image captured is in an aligned state, which is convenient for the subsequent identification and detection process.

[0084] For example, the orientation of the screen to be detected may be corrected by template matching.

[0085] The first step is to obtain a second image of a second area of ​​the screen to be detected;

[0086] The second step is to match the second image with the candidate template to determine the matching template;

[0087] The third step is to determine the offset angle of the screen to be detected based on the matching template;

[0088] The fourth step is to send the offset angle to the control end of the mobile platform so that the control end can control the rotation of the screen to be detected according to the offset angle.

[0089] In this exemplary embodiment, the second area is used to calibrate the screen's orientation. For example, as shown in Figure 2, the second area is composed of markers at the four corners of a rectangular screen. These markers are arranged completely symmetrically, and the positions of two of these markers can be used to determine whether the screen's angle is level. Two alignment cameras capture images of the two markers, respectively. In real-world scenarios, the screen's angle often shifts slightly as it moves between different platforms or devices. Therefore, this example requires that the screen be oriented before capturing the image. A candidate template is determined based on a standard image of the screen to be inspected. For example, an image obtained by rotating or fine-tuning this standard image can serve as a candidate template. The second image is matched against the candidate template, primarily in terms of orientation. If the two images coincide, the match is considered successful. The offset angle of the matched template relative to the standard image is used as the screen offset angle and is sent to the mobile platform controller, which controls the mobile platform to rotate the corresponding angle to achieve screen correction. The markers in the corrected image coincide with those in the original template. This example uses a template matching method to correct the screen's orientation. The matching process is simple, the algorithm is efficient, and it can be adapted to the actual streaming process in production, ensuring production efficiency.

[0090] Exemplarily, the second image is matched with the candidate template, and the process of determining the matching template may include the following steps: obtaining the original template of the screen to be detected; rotating the original template multiple times along the center point at a certain angle interval to obtain multiple candidate templates; matching the second image with each of the candidate templates respectively, and determining that the candidate template that coincides with the second area of ​​the second image is the matching template.

[0091] In this exemplary embodiment, the original template is a standard image taken before the direction of the screen to be detected is shifted. First, the original template (standard mark point image) of the screen to be detected is obtained, and then the original template is rotated clockwise or counterclockwise along the center point at a certain interval angle multiple times. Each rotation obtains a corresponding template as a candidate template. For example, from -10 to 10 degrees, it is rotated multiple times at intervals of 1 degree to obtain 20 candidate templates. Then, the mark point image (second image) of the screen to be detected is matched with each candidate template respectively, and the template whose mark point of the template coincides with the mark point of the second image is used as the matching template. The rotation angle and rotation interval of the candidate template can be set according to actual conditions, and this example does not limit this. This example presets various screen offset conditions in actual conditions in advance by rotating the original template, and then obtains the matching template corresponding to the actual screen offset through a simple template matching process. By pre-setting the results, a foundation is laid for the subsequent rapid matching of the offset angle, and the processing process is carried out in advance, saving processing time in actual production and improving processing efficiency.

[0092] By using the above pre-built template method, a matching template can be successfully matched, and a fast rotation template matching process can be achieved. At the same time, the matching process is simple, the algorithm is efficient, and the positioning accuracy is high.

[0093] In step S120 , a plurality of screen images are stored.

[0094] In this example embodiment, multiple screen images taken continuously (flying images) can be stored in memory. For example, multiple screen images can be stored in a message queue to achieve instant storage of screen images, facilitate streaming processing of the captured images, and improve production efficiency.

[0095] Exemplarily, the first image is stored in a pre-applied target storage space in a target format, where the first image is a plurality of screen images captured for a first area of ​​the screen to be detected.

[0096] In this example embodiment, the first area refers to the area where scratch detection is required. For example, the first area can be the area near the four sides of the matrix screen, where scratches may be generated due to the cutting process. The target format refers to a preset storage format. For example, the storage content of each subsection can be customized. For each flying process (i.e., the process of capturing images during the moving screen process), memory space can be applied in advance to store the captured images corresponding to this flying process. This memory only needs to be applied once, and there is no need to apply / release memory for each storage. This example adopts a space-for-efficiency method. By applying for the memory for the current flying process in advance, the memory application / release process in each image storage process is avoided, and the storage efficiency is higher.

[0097] Exemplarily, the process of pre-applying for target storage space includes the following steps: obtaining the size of the first area, the moving speed of the screen to be detected, and the shooting frequency of the image acquisition device; determining the number of screen images in the first image based on the size of the first area, the moving speed of the screen to be detected, and the shooting frequency of the image acquisition device; determining the storage space size required for the first image based on the number of screen images and the size of the screen image; and applying for memory corresponding to the storage space size as the target storage space.

[0098] In this example embodiment, applying for storage space in advance requires predetermining the amount of storage space required for this shooting. The moving speed of the screen to be detected can be obtained from the mobile platform, the shooting frequency can be obtained from the image acquisition device, and the size of the first area (such as length) can be input in advance. The shooting time can be determined based on the moving speed and the size of the first area. The number of screen images to be captured can be determined based on the shooting time and the shooting frequency. The required storage space size is determined based on the screen image data and the size of each screen image. For example, when determining the shooting screen range for each flight, the area for the same product to be shot is fixed, and the number of images required to be captured in this area can also be determined. Taking the shooting of 100 pictures as an example, 100*4096*2048*3 bytes (approximately 2.4G) of memory can be pre-applied as the target storage space. This example can more accurately determine the storage space required for each shooting, avoid wasting memory space, improve resource utilization, and at the same time, avoid data loss caused by applying for too little memory.

[0099] In some embodiments, before the first image is stored in the pre-applied target storage space in the target format, the first image is compressed.

[0100] In this exemplary embodiment, the first image may be compressed. For example, the first image may be compressed and encoded using jpg format, and the compressed and encoded image may be stored to save memory space.

[0101] Because images of the same size can have different sizes after compression, a storage format design is necessary. For multiple screen images in the first image, the image size of the compressed image corresponding to each screen image can be determined sequentially; each compressed image and its corresponding image size can be stored sequentially in the same target file; in response to the last screen image captured for the first area being stored in the target file, the number of images in the target file can be determined; and this image number information can be added to the header byte of the target file.

[0102] In this example embodiment, the storage format of the first image can be designed to include the image size and image content. Taking into account the frequent input / output operations caused by the image storage process one by one, this example stores the screen images corresponding to the same flying process in the same target file to avoid the disk usage problem caused by the image storage process. At this time, the image storage format can also include information on the total number of images. For example, as shown in Table 1, the image storage format can be to add an 8-byte (long long) data before each compressed data to record the byte content size of the current image + the compressed data; the ellipsis in Table 1 indicates that the storage of other images except the first two images is omitted.

[0103] Table 1

[0104] This example only requires a single input / output operation to store all first images during the flight. Specifically, 4 bytes are written first. For each subsequent image captured, 8 bytes are added to the target file to record the byte size of the compressed image. The compressed image data is then written, and subsequent images are written in sequence. When the last image is written, the first 4 bytes are modified to include the number of images captured in this flight. This image storage method not only reduces the number of input / output operations required for image storage during flight, but also reduces the number of image files actually stored in the system, significantly improving both disk and system performance.

[0105] In step S130, scratch detection is performed on multiple screen images in sequence to obtain detection results of the screen to be detected; wherein, the shooting of the screen images and the scratch detection are performed asynchronously, and the storage of the screen images and the scratch detection are performed in parallel.

[0106] In this example embodiment, scratch detection can be performed using dynamic threshold segmentation or various machine learning models, which is not limited in this example. Screen image storage and scratch detection can be performed separately in two queues. For example, in response to a detection start signal, two parallel tasks (such as a first thread and a second thread) can be triggered. The screen image is received and stored based on the first thread, and scratch detection of the screen image is performed based on the second thread. For example, in response to the first screen image being stored in the image queue, scratch detection of the screen image can be triggered. In this way, the scratch detection process of the stored image is performed while the image is stored, which can save processing time of the entire process.

[0107] In actual production processes, because screen capture speeds are faster than the corresponding scratch detection processing, some solutions reduce the speed of the fly-by-fly capture to achieve a simultaneous capture and processing process, which reduces process speed. Based on this, in this example, screen image capture and scratch detection processing are performed asynchronously to fully utilize production resources, avoid waiting between processing steps, and improve processing efficiency. For example, considering that the transfer of screens between different platforms also requires time, the complete capture time of a fly-by-fly capture can be calculated as the fly-by-fly capture time + the time it takes to transfer the screen to the next process. The screen transfer time is basically fixed at 1.5 seconds. As shown in Figure 3, taking the example of capturing 100 images along a screen edge (i.e., a single fly-by-fly capture), the minimum time to complete all 100 images is 3.33 seconds. The current scratch detection speed is 20 fps, and it takes 5 seconds to detect all images. According to the traditional solution, the time required for capture and recognition is at least 3.33 + 5 + 1.5 seconds = 9.33 seconds. This example approach performs scratch detection simultaneously with image capture and storage. Image capture and storage are queued in one queue, while scratch detection is performed in another. Starting with the first image stored in the queue, the scratch detection processing queue begins working synchronously. This allows images to be captured and processed simultaneously until the last image is processed. The total processing time of this example is determined only by the longest step, reducing it from the original 9.33 seconds to approximately 5 seconds, significantly improving processing efficiency.

[0108] For example, considering that the scratch area occupies a small area on the screen and is difficult to identify, this example can input multiple screen images into the target detection model for scratch detection and output the detection results. The target detection model is a trained image semantic segmentation network.

[0109] In this example embodiment, the image semantic segmentation network may include FCN, Unet, DeepLabV3+, Vnet, and the like. These networks may also be modified, and this example is not limited thereto. The image semantic segmentation network may be trained using training samples to obtain an object detection model. The image semantic segmentation network can directly segment objects in an image with high segmentation accuracy.

[0110] For example, based on the DeepLabV3+ network, as shown in FIG4 , the target detection model may include an encoding module and a decoding module. Before scratch detection is performed, the model needs to be trained. The training process includes the following steps:

[0111] The first step is to obtain training samples, which include screen images with scratches;

[0112] The second step is to input the training samples into the image semantic segmentation network to obtain the training output;

[0113] The third step is to determine the cross entropy loss between the training output and the sample label using a dynamic scaling factor.

[0114] The fourth step is to adjust the model parameters of the image semantic segmentation network based on cross entropy loss.

[0115] In this example implementation, a screen image with scratches can be first obtained as a training sample. The scratches in the image can be considered as negative samples, while other areas in the image can be considered as positive samples. The model parameters of the image semantic segmentation network are then initialized. Considering that the scratched area in the actual image accounts for a very small proportion compared to the background area, that is, there is a serious imbalance in the ratio of positive and negative samples, most of the parameter weights in the entire training process are learning background samples. A dynamic scaling factor γ is used to adjust the sample weights during the training process. For example, the dynamic scaling factor is used to reduce the weight of samples that are easy to distinguish during training, thereby quickly focusing on samples that are difficult to distinguish, thereby improving training effectiveness and efficiency. By calculating the cross entropy loss during training, the model parameters are updated based on the gradient descent algorithm until the model converges or reaches the specified training round, and a trained target detection model is obtained.

[0116] For example, the loss function based on the dynamic scaling factor can be expressed as: Focal Loss = -(1-pt) γ log(pt)

[0117] Where pt represents the probability of predicting the true sample label. When pt approaches 1, it indicates that the sample is easily distinguishable. In this case, the dynamic scaling factor γ approaches 0, reducing the loss ratio of easily distinguishable samples. When pt is very small (approaching 0), the probability of the sample being a foreground is extremely low, meaning that it is misclassified as a positive sample. In this case, the dynamic scaling factor γ approaches 1, reducing its impact on the loss function. The dynamic scaling factor γ can range from 0 to 5, with a preferred value of 1.5.

[0118] In the embodiment shown in FIG4 , taking into account the small proportion of scratch defects in the forward image, the model training parameters are adjusted through a dynamic scaling factor to dynamically reduce the weight of easily distinguishable samples during training, thereby enabling the model to quickly focus on learning difficult-to-distinguish samples, thereby improving the detection accuracy and efficiency of the model in scratch detection.

[0119] Furthermore, scratch detection is performed on the screen image using the trained object detection model, as shown in FIG5 , which may include the following steps:

[0120] In the first step, feature extraction is performed on the input screen image to obtain shallow feature maps and deep feature maps.

[0121] In the second step, dilated convolution and channel adjustment with different dilation rates are performed on the deep feature map to obtain the encoded output.

[0122] In the third step, feature adjustments based on the attention mechanism are performed on the shallow feature map and the encoding output to obtain the first feature map and the second feature map.

[0123] The fourth step is to perform feature fusion on the first feature map and the second feature map.

[0124] The fifth step is to perform feature adjustment on the fusion result based on the attention mechanism to obtain the third feature map.

[0125] In the sixth step, convolution and upsampling are performed on the third feature map to obtain the detection result.

[0126] In this example implementation, the input screen image enters the deep convolutional neural network DCNN for feature extraction, obtaining two effective feature layers, namely a deep feature map (smaller) and a shallow feature map (wider and higher); then, in the encoding module, the deep feature map is subjected to dilated convolution with different dilation rates (such as dilation rates of 6, 12, and 18) to improve the network's receptive field, so that the network has a sense of different features. After that, the extracted features are stacked, and the number of channels is adjusted by 1*1 convolution to obtain the encoded output. The shallow feature map extracted by DCNN enters the decoding module, and then undergoes feature adjustment based on the attention mechanism after 1*1 convolution. The high-level semantic information output by the encoding module is also feature adjusted and upsampled based on the attention mechanism, and then feature fused (such as feature splicing) with the feature adjustment result of the decoding module. The fusion result is then feature adjusted based on the attention mechanism, and then subjected to 3*3 convolution and upsampling to output the segmentation result.

[0127] In the embodiment shown in FIG5 , the shallow feature map and the high-level semantic features of the encoded output are respectively subjected to feature adjustment based on the attention mechanism, which can adjust the attention of features at different depths so that the network focuses on the important areas of the image (i.e., the areas where the scratches are located) at different depths, thereby improving the network's classification accuracy for scratches. Furthermore, attention adjustment can be performed on the fusion result after the fusion of the shallow feature map and the high-level semantic features to ensure that the fused features also focus on the features to be classified (scratches), thereby reducing the classification difficulty and low classification accuracy caused by the large difference in the proportion of the two samples to be classified in the screen image (the proportion of scratches is too small), thereby improving the classification accuracy of scratches.

[0128] For example, CBAM (Convolutional Block Attention Module) can be used to perform feature adjustment based on the attention mechanism. The processing process includes the following steps:

[0129] The first step is to perform channel attention adjustment on the target feature map; the target feature map is a shallow feature map, encoding output or fusion result.

[0130] In this example embodiment, channel attention adjustment can first perform feature map compression to aggregate the spatial information of the feature map. Specifically, global average pooling and global maximum pooling can be performed on the target feature map respectively to generate two different spatial context descriptions, namely average pooling features and maximum pooling features; the two features enter the shared network (such as multi-layer perceptron MLP) respectively to generate a channel attention map (channel attention weight matrix), and the channel attention weight matrix is ​​multiplied with the target feature map of the original input to perform adaptive refinement of the feature map based on channel attention, and output the corresponding channel attention feature map.

[0131] In the second step, spatial attention adjustment is performed on the channel attention adjustment results.

[0132] In this example implementation, the channel attention adjustment results can be subjected to maximum pooling and average pooling in the channel dimension respectively to fuse the information of different channels, and then the maximum pooling and average pooling results are connected and convolved in the channel dimension to obtain a spatial attention matrix diagram, which is activated by the Sigmoid function to obtain the weight matrix of spatial attention; the weight matrix of spatial attention is element-wise multiplied with the input channel attention adjustment result to perform adaptive refinement of the feature map based on spatial attention, and output the spatial attention adjustment.

[0133] In the above example, by performing the above-mentioned channel attention adjustment and spatial attention adjustment on the intermediate results of the image semantic segmentation network (such as shallow feature maps, encoding outputs and fusion results), the network increases its attention and transmission to important image areas and their positions, increases the network's recognition ability of important areas and their positions, and improves recognition accuracy.

[0134] In some embodiments, when the detection result indicates that there are scratches on the screen to be detected, a scratch area is determined; and a secondary detection is performed on the scratch area to determine a final detection result.

[0135] In this example embodiment, to avoid false detections, a secondary inspection can be performed on screens that have scratches detected, further improving detection accuracy. This secondary inspection allows for higher-precision imaging and detection of the scratched area from the initial inspection. Specifically, the scratch location can be determined based on the scratch detection results, and higher-precision images can be taken of that location. For example, 3*5=15 fine-inspection images can be taken of that location. Because the fine-inspection image has a smaller field of view, an autofocus function can be added to the fine-inspection camera to ensure image quality. To speed up the capture process, the scratched area can be moved back and forth to save time.

[0136] Experimental verification

[0137] Scratch detection was performed on a screen obtained by cutting a certain specification. The Deeplabv3+ network and the image semantic segmentation network of the present disclosure were trained using the same training data based on different dynamic scaling factors. The trained models were then used for scratch detection. The detection results are shown in Table 2. In Table 2, the mean intersection over union (MIoU) result was used as the semantic segmentation evaluation indicator, and its formula is:

[0138] Where P represents the predicted value, G represents the true value, k represents the total number of categories. In this example, k is 2. MIoU represents the average of the intersection of the predicted value and the true value of each category divided by the union.

[0139] Table 2

[0140] As can be seen from Table 2, the image semantic segmentation network disclosed in the present invention can improve the network recognition performance compared with the original Deeplabv3+, and the network recognition performance can also be improved through a suitable dynamic scaling factor.

[0141] For a common rectangular screen, the scratch detection process may be performed on one set of opposite sides of the screen first, and then on another set of opposite sides of the screen, to complete the screen scratch detection process.

[0142] 6 , an embodiment of the present disclosure further provides a screen scratch detection system, which may include: a first detection platform 610 , which may include:

[0143] The mobile platform 611 is loaded with the screen to be inspected and moves along a specified direction to transport the screen to be inspected;

[0144] The image acquisition device 612 continuously captures multiple screen images of the screen to be inspected on the mobile platform in response to the inspection start signal;

[0145] The detection sub-platform 613 receives and stores multiple screen images transmitted by the image acquisition device; and performs scratch detection on the multiple screen images in sequence to obtain the detection results of the screen to be detected; wherein, the storage of the screen images and the scratch detection are performed in parallel.

[0146] In this example embodiment, mobile platform 611 can be a screen loading platform mounted on a movable guide rail. For example, the mobile platform can be a vacuum platform that can adsorb the screen to be inspected onto the platform. Mobile platform 611 can also be configured with a corresponding controller to exchange information with other platforms and control the mobile platform's movement in a specified direction. Image acquisition device 612 can be a set of alignment cameras, or the alignment cameras can be equipped with a high-speed trigger and a PLC controller. Inspection sub-platform 613 can be a terminal or server with data processing capabilities, although this example does not limit this.

[0147] In some embodiments, the system includes:

[0148] The first inspection platform 610 is used to perform scratch inspection on a set of opposite edges of the screen to be inspected;

[0149] The second inspection platform 620 is used to perform scratch inspection on another set of opposite edges of the screen to be inspected.

[0150] In this exemplary embodiment, the second inspection platform can be constructed identically to the first inspection platform, and can include corresponding mobile platforms and image acquisition devices to form a subsystem for inspecting one set of opposite edges of the screen. The entire screen scratch inspection system can be comprised of two or more subsystems.

[0151] In some embodiments, the system further includes: a third detection platform 630 for performing a secondary detection on the scratch area determined by the first detection platform or the second detection platform.

[0152] In this example embodiment, a third detection platform for the scratch area can be set after the first detection platform and the second detection platform respectively. When the first detection platform or the second detection platform detects the presence of scratches, the corresponding third detection platform will perform a secondary detection on the detection result to ensure the accuracy of the detection result.

[0153] In some embodiments, the system further includes: a screen transfer device 640 for transporting and transferring the screen to be inspected between the first inspection platform, the second inspection platform, and the third inspection platform.

[0154] In this example embodiment, the screen transfer device may be a mechanical gripper or a vacuum rotatable mechanical gripper, which can transfer the screen to be inspected between different platforms or devices to achieve the connection between different processes.

[0155] As shown in Figure 6, the system can also include a code reading platform 650 and a conveying platform 660. In a complete inspection process, the screen transfer device 640 can transfer the screen to be inspected from the loading tray to the code reading platform 650, and then transfer it from the code reading platform 650 to the first inspection platform 610, from the first inspection platform 610 to the conveying platform 660, from the conveying platform 660 to the second inspection platform 620, and from the first inspection platform 610 or the second inspection platform 620 to the third inspection platform 630 or the unloading tray to complete the inspection process of the screen to be inspected.

[0156] In some embodiments, as shown in FIG7 , the information interaction process between the software processing process of each platform and the hardware device in the detection system may include the following steps:

[0157] In the first step, the controller of the code reading platform reads the ID of the screen to be detected and sends a detection trigger signal to each detection platform.

[0158] In the second step, each detection platform receives and records the screen ID and broadcasts the detection start signal.

[0159] In the third step, the mobile platform controls the moving guide rail based on the received detection start signal.

[0160] In the fourth step, the image acquisition device triggers continuous acquisition of screen images based on the received detection start signal.

[0161] In the fifth step, each detection sub-platform receives the screen image captured by the corresponding image acquisition device and stores it in the corresponding image queue; at the same time, the detection queue starts to perform scratch detection on the screen image and writes the detection results into the result queue.

[0162] Step 6: After the shooting is completed, each image acquisition device sends a shooting completion signal to the corresponding hardware device.

[0163] In the seventh step, each hardware device generates a detection end signal in response to receiving the shooting end signal.

[0164] In the eighth step, each detection sub-platform waits for the detection queue to complete detection, and writes the detection results into the result queue, for example, into the MYSQL database.

[0165] The ninth step is to send the detection results and detection end signal to the controller corresponding to each platform / device.

[0166] In step 10, the mobile platform ends the current round of detection in response to receiving the detection end signal, and returns the mobile platform to the starting position to prepare for the next round of detection.

[0167] The beneficial effect of the above embodiment is that information interaction can be carried out with each detection platform through the hardware controller PLC, realizing a convenient and fast detection control process combining software and hardware, so that the PLC focuses on the hardware control movement process, and each detection platform is committed to the algorithm processing process, realizing the decoupling of hardware operation and software processing process, and improving detection efficiency.

[0168] The present disclosure aims to solve the problem that cracks are likely to occur on the edge of the screen due to adsorption stress and other reasons during the process of cutting a substrate into screens using a laser process in actual screen production. A scratch detection method is designed for the edge of the screen. The mobile platform moves with the guide rail to realize the flying shooting of the screen to be detected by the alignment camera. Through the system layout design, the flying shooting process of two sets of opposite edges of the screen can be completed with one guide rail movement, thereby improving the detection efficiency. For the data processing process, the flying image storage is performed by pre-applying for memory and storing the same file, which improves the storage efficiency, reduces the disk usage, and improves the processing efficiency; and the image storage and scratch detection and identification are performed by two parallel queues, which saves processing time and further improves the processing efficiency. Furthermore, scratch detection is performed by an image semantic segmentation network based on the attention mechanism, which can improve the detection accuracy of important areas (scratch areas). The present disclosure can realize the automated assembly line processing process of screen scratch detection, monitor the process yield in real time, and improve the process quality.

[0169] Corresponding to the above method embodiment, the present disclosure further discloses a scratch detection device. Referring to FIG8 , FIG8 schematically shows a structural diagram 800 of the scratch detection device in an exemplary embodiment of the present disclosure, including:

[0170] The receiving module 810 is configured to receive a plurality of screen images transmitted by an image acquisition device, wherein the plurality of screen images are obtained by triggering the image acquisition device to continuously photograph the screen to be inspected in response to a detection start signal; the screen to be inspected moves along a specified direction during the continuous photographing process;

[0171] The storage module 820 is configured to store a plurality of screen images;

[0172] The detection module 830 is configured to perform scratch detection on the plurality of screen images in sequence to obtain a detection result of the screen to be detected;

[0173] The capturing of the screen image and the scratch detection are performed asynchronously, and the storing of the screen image and the scratch detection are performed in parallel.

[0174] In an exemplary embodiment of the present disclosure, the storage module 820 is further configured to store the plurality of screen images into an image queue;

[0175] The detection module 830 is further configured to trigger scratch detection in response to the first screen image being stored in the image queue.

[0176] In an exemplary embodiment of the present disclosure, the screen image includes a first image, and the storage module 820 is further configured to:

[0177] The first image is stored in a pre-applied target storage space in a target format, wherein the first image is a plurality of screen images shot for a first area of ​​the screen to be detected.

[0178] In an exemplary embodiment of the present disclosure, the storage module 820 is further configured to:

[0179] Before storing the first image in the pre-applied target storage space in the target format, the first image is compressed.

[0180] In an exemplary embodiment of the present disclosure, the storage module 820 is further configured to:

[0181] Acquire the size of the first area, the moving speed of the screen to be detected, and the shooting frequency of the image acquisition device;

[0182] determining the number of screen images in the first image according to the size of the first area, the moving speed of the screen to be detected, and the shooting frequency of the image acquisition device;

[0183] determining a storage space required for the first image according to the number of the screen images and the size of the screen image;

[0184] Apply for a memory corresponding to the storage space size as the target storage space.

[0185] In an exemplary embodiment of the present disclosure, the storage module 820 stores the first image in the target format into the pre-applied target storage space through the following steps:

[0186] For the multiple screen images in the first image, sequentially determining the image size of the compressed image corresponding to each screen image;

[0187] Store the compressed images and their corresponding image sizes into the same target file in sequence;

[0188] In response to the last screen image captured for the first area being stored in the target file, determining information on the number of images in the target file;

[0189] Add the image quantity information to the header byte of the target file.

[0190] In an exemplary embodiment of the present disclosure, the apparatus further includes:

[0191] The direction correction module is configured to perform direction correction on the screen to be detected before continuously acquiring multiple screen images of the screen to be detected.

[0192] In an exemplary embodiment of the present disclosure, the direction correction module corrects the direction of the screen to be detected by the following steps:

[0193] Acquire a second image of a second area of ​​the screen to be detected;

[0194] Matching the second image with the candidate template to determine a matching template;

[0195] Determine the offset angle of the screen to be detected based on the matching template;

[0196] According to the offset angle, the screen to be detected is rotated.

[0197] In an exemplary embodiment of the present disclosure, the direction correction module matches the second image with the candidate template to determine the matching template through the following steps:

[0198] Get the original template of the screen to be detected;

[0199] The original template is rotated multiple times along the center point at certain angle intervals to obtain multiple candidate templates;

[0200] The second image is matched with each of the candidate templates respectively, and the candidate template that overlaps with the second area of ​​the second image is determined as the matching template.

[0201] In an exemplary embodiment of the present disclosure, the detection start signal is generated in response to the image acquisition device acquiring identification information of the screen to be detected, and the identification information of the screen to be detected is acquired using a code reading platform.

[0202] In an exemplary embodiment of the present disclosure, the detection module 830 is further configured to:

[0203] Multiple screen images are input into the target detection model for scratch detection, and the detection results are output. The target detection model is a trained image semantic segmentation network.

[0204] In an exemplary embodiment of the present disclosure, the detection module 830 implements scratch detection by inputting multiple screen images into the target detection model through the following steps:

[0205] Perform feature extraction on the input screen image to obtain a shallow feature map and a deep feature map;

[0206] Perform dilated convolution and channel adjustment with different dilation rates on the deep feature map to obtain the encoded output;

[0207] Perform feature adjustments based on the attention mechanism on the shallow feature map and the encoding output to obtain the first feature map and the second feature map;

[0208] Perform feature fusion on the first feature map and the second feature map;

[0209] Perform feature adjustment based on the attention mechanism on the fusion result to obtain the third feature map;

[0210] The third feature map is convolved and upsampled to obtain the detection result.

[0211] In an exemplary embodiment of the present disclosure, the detection module 830 implements feature adjustment based on the attention mechanism through the following steps:

[0212] Perform channel attention adjustment on the target feature map; the target feature map is a shallow feature map, encoding output or fusion result;

[0213] Perform spatial attention adjustment on the channel attention adjustment results.

[0214] In an exemplary embodiment of the present disclosure, the apparatus further includes a training module, which is configured to:

[0215] Obtaining a training sample, where the training sample includes a screen image with scratches;

[0216] Input the training samples into the image semantic segmentation network to obtain the training output;

[0217] Determine the cross entropy loss between training output and sample labels using a dynamic scaling factor;

[0218] Adjust the model parameters of the image semantic segmentation network based on cross entropy loss.

[0219] In an exemplary embodiment of the present disclosure, the apparatus further includes:

[0220] The secondary detection module is configured to determine the scratch area when the detection result shows that the screen to be detected has scratches; and perform secondary detection on the scratch area to determine the final detection result.

[0221] The specific details of each module / unit in the above device have been described in detail in the embodiment of the method part, so they will not be repeated here.

[0222] In an exemplary embodiment of the present disclosure, an electronic device is further provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute all or part of the steps of the scratch detection method in this exemplary embodiment.

[0223] Figure 9 shows a schematic diagram of the structure of a computer system of an electronic device for implementing an embodiment of the present disclosure. It should be noted that the computer system 900 of the electronic device shown in Figure 9 is only an example and should not bring any limitation to the functions and scope of use of the embodiment of the present disclosure.

[0224] As shown in FIG9 , a computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for system operation are also stored in the RAM 903. The CPU 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0225] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, and the like; an output section 907 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 908 including a hard disk and the like; and a communication section 909 including a network interface card such as a local area network (LAN) card or a modem. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 910 as needed, so that a computer program read therefrom can be installed into the storage section 908 as needed.

[0226] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from a removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the various functions defined in the device of the present application are executed.

[0227] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, any one of the above methods is implemented.

[0228] It should be noted that the computer-readable storage media described in the present disclosure may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, radio frequency, or any suitable combination thereof.

[0229] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow from the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

[0230] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A screen scratch detection method, comprising: Receiving a plurality of screen images transmitted by an image acquisition device, wherein the plurality of screen images are obtained by triggering the image acquisition device to continuously photograph the screen to be detected in response to a detection start signal; The screen to be detected moves along a specified direction during continuous shooting; storing a plurality of the screen images; Performing scratch detection on the plurality of screen images in sequence to obtain detection results of the screen to be detected; The shooting of the screen image and the scratch detection are performed asynchronously, and the storage of the screen image and the scratch detection are performed in parallel.

2. The method according to claim 1, wherein: Storing a plurality of said screen images, including: storing a plurality of the screen images into an image queue; The sequentially performing scratch detection on the plurality of screen images comprises: In response to the first screen image being stored in the image queue, the scratch detection is triggered.

3. The method according to claim 1, wherein: The screen image includes a first image, and storing a plurality of the screen images transmitted by the image acquisition device includes: The first image is stored in a pre-applied target storage space in a target format, wherein the first image is a plurality of screen images captured for a first area of ​​the screen to be detected.

4. The method according to claim 3, wherein: Before receiving the multiple screen images transmitted by the image acquisition device, the method further includes: Acquire the size of the first area, the moving speed of the screen to be detected, and the shooting frequency of the image acquisition device; Determining the number of screen images in the first image according to the size of the first area, the moving speed of the screen to be detected, and the shooting frequency of the image acquisition device; Determining the storage space required for the first image according to the number of the screen images and the size of one of the screen images; Apply for a memory corresponding to the storage space size as the target storage space.

5. The method according to claim 3, wherein: Before storing the first image in the pre-applied target storage space in the target format, the method further includes: The first image is compressed.

6. The method according to claim 5, wherein: The first image is stored in a target format in a pre-applied target storage space, including: For the multiple screen images in the first image, sequentially determine the image size of the compressed image corresponding to each screen image; storing the compressed images and the corresponding image sizes in the same target file in sequence; In response to the last screen image captured for the first area being stored in the target file, determining information on the number of images in the target file; The image quantity information is added to the header byte position of the target file.

7. The method according to claim 1, wherein: Before continuously acquiring a plurality of screen images of the screen to be detected, the method further includes: Direction correction is performed on the screen to be detected.

8. The method according to claim 7, wherein: Direction correction is performed on the screen to be detected, including: Acquire a second image of a second area of ​​the screen to be detected; Matching the second image with a candidate template to determine a matching template; Determining the offset angle of the screen to be detected according to the matching template; The offset angle is sent to a control end of the mobile platform so that the control end controls the rotation of the screen to be detected according to the offset angle.

9. The method according to claim 8, wherein: Matching the second image with a candidate template to determine a matching template includes: Get the original template of the screen to be detected; The original template is rotated multiple times along the center point at certain angle intervals to obtain multiple candidate templates; The second image is matched with each of the candidate templates respectively, and the candidate template that overlaps with the second area of ​​the second image is determined as the matching template.

10. The method according to claim 1, wherein: The detection start signal is generated in response to the image acquisition device acquiring the identification information of the screen to be detected, and the identification information of the screen to be detected is acquired by using a code reading platform.

11. The method according to claim 1, wherein: The sequentially performing scratch detection on the plurality of screen images comprises: The plurality of screen images are input into a target detection model for scratch detection, and the detection result is output, wherein the target detection model is a trained image semantic segmentation network.

12. The method according to claim 11, wherein: The step of inputting the plurality of screen images into a target detection model for scratch detection comprises: Performing feature extraction on the input screen image to obtain a shallow feature map and a deep feature map; Performing dilated convolution and channel adjustment with different dilation rates on the deep feature map to obtain a coded output; Performing feature adjustment based on the attention mechanism on the shallow feature map and the encoding output respectively to obtain a first feature map and a second feature map; Performing feature fusion on the first feature map and the second feature map; Perform feature adjustment based on the attention mechanism on the fusion result to obtain the third feature map; The third feature map is convolved and upsampled to obtain a detection result.

13. The method according to claim 12, wherein: The feature adjustment based on the attention mechanism includes: Performing channel attention adjustment on a target feature map; the target feature map is the shallow feature map, the encoding output or the fusion result; Perform spatial attention adjustment on the channel attention adjustment results.

14. The method according to claim 11, wherein: The method further comprises: Acquire a training sample, wherein the training sample includes a screen image with scratches; Inputting the training samples into an image semantic segmentation network to obtain a training output; Determining a cross entropy loss of the training output and sample labels using a dynamic scaling factor; Based on the cross entropy loss, model parameters of the image semantic segmentation network are adjusted.

15. The method according to any one of claims 1 to 14, wherein: The method further comprises: When the detection result shows that the screen to be detected has scratches, determining the scratch area; The scratched area is subjected to a secondary inspection to determine the final inspection result.

16. A screen scratch detection system, comprising a first detection platform, wherein the first detection platform comprises: The mobile platform is loaded with the screen to be inspected and moves along a specified direction to transport the screen to be inspected; An image acquisition device, in response to a detection start signal, continuously captures a plurality of screen images of the screen to be detected on the mobile platform; A detection sub-platform receives and stores the plurality of screen images transmitted by the image acquisition device; And scratch detection is performed on the plurality of screen images in sequence to obtain the detection result of the screen to be detected; wherein the shooting of the screen image and the scratch detection are performed asynchronously, and the storage of the screen image and the scratch detection are performed in parallel.

17. The system of claim 16, wherein: The system comprises: A first detection platform, used for performing scratch detection on a group of opposite edges of the screen to be detected; The second detection platform is used to perform scratch detection on another set of opposite edges of the screen to be detected.

18. The system of claim 17, wherein: The system further comprises: The third detection platform is used to perform a secondary detection on the scratch area determined by the first detection platform or the second detection platform.

19. The system of claim 18, wherein: The system further comprises: The screen transfer device is used to carry and transfer the screen to be tested between the first testing platform, the second testing platform and the third testing platform.

20. A screen scratch detection device, comprising: The receiving module is configured to receive multiple screen images transmitted by the image acquisition device, wherein the multiple screen images The screen image is obtained by triggering the image acquisition device to continuously photograph the screen to be detected in response to the detection start signal; The screen to be detected moves along a specified direction during continuous shooting; A storage module, configured to store a plurality of the screen images; A detection module is configured to perform scratch detection on the plurality of screen images in sequence to obtain a detection result of the screen to be detected; The shooting of the screen image and the scratch detection are performed asynchronously, and the storage of the screen image and the scratch detection are performed in parallel.

21. An electronic device, comprising: Memory, for storing computer instructions; A processor, configured to read the computer instructions and execute the method according to any one of claims 1 to 15.

22. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 15 is implemented.

Images