An industrial video defect frame automatic extraction and product information perception grid puzzle quality inspection archiving method, system, terminal and storage medium

Abstract

The application relates to the technical field of image processing, and discloses a quality inspection archiving method, system, terminal and storage medium for industrial video defect frame automatic extraction and product information perception grid jigsaw puzzle. The method comprises the following steps: acquiring an industrial video stream, performing frame skipping sampling and model inference processing on the industrial video stream, and obtaining a detection result set of each sampling frame; determining a sampling frame with a non-empty detection result set as a defect frame; performing frame number restoration and title superposition on each defect frame to obtain a target defect frame; arranging the target defect frames into a defect frame queue according to the detection time sequence of the target defect frames; performing grid jigsaw puzzle processing on each target defect frame in the defect frame queue to obtain a target canvas; performing product information superposition and archiving output to obtain an archived quality inspection image file. The application realizes the full automatic generation from an industrial detection video to a standardized quality inspection evidence file, and improves the efficiency and standardization of quality inspection archiving.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a quality inspection archiving method, system, terminal, and computer-readable storage medium for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking. Background Technology

[0002] In industrial product quality inspection scenarios, automated optical inspection equipment and AI vision inspection systems are widely deployed at the end of the production line. During the inspection process, the system continuously acquires product images in the form of video streams and performs real-time inference through a target detection model. When a defect is detected, the image containing the defect needs to be extracted to form a quality inspection evidence file. Current practices mostly involve quality inspectors manually playing back the video, taking screenshots frame by frame, and manually stitching the images together. This method relies on manual operation, is inefficient, and cannot automatically associate the defective frame with its position information in the original video. It also makes it difficult to simultaneously carry defect evidence and product-related information in a single archive file.

[0003] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0004] The main objective of this invention is to provide a quality inspection archiving method, system, terminal, and computer-readable storage medium for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking. This aims to solve the technical problem in the prior art where defect evidence extraction and archiving in the industrial quality inspection process relies on manual video review and manual mosaicking operations, which cannot achieve automatic screening, location, and standardized archiving output of defect frames.

[0005] To achieve the above objectives, this invention provides a quality inspection and archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking. The method includes the following steps: The industrial video stream to be detected is acquired, and the industrial video stream is subjected to frame skipping sampling and model inference processing to obtain the detection result set corresponding to each sampled frame; Sequentially determine whether the detection result set corresponding to each sampling frame is an empty set, and determine the sampling frame whose detection result set is not an empty set as a defect frame to obtain multiple defect frames; Each defective frame is restored by its frame number and its title is overlaid to obtain multiple target defective frames. Arrange the target defect frames into a defect frame queue according to the detection time order of each target defect frame; The target defect frames in the defect frame queue are subjected to grid mosaic processing to obtain the target canvas; The target canvas is processed by overlaying product information and archiving output to obtain archived quality inspection image files.

[0006] Furthermore, the step of acquiring the industrial video stream to be detected, performing frame-skipping sampling and model inference processing on the industrial video stream, and obtaining a set of detection results corresponding to each sampled frame includes: The industrial video stream to be detected is acquired, and the industrial video stream is sampled by skipping frames according to a preset frame skipping step size to obtain multiple sampled frames; Each of the sampled frames is sequentially input into the trained target detection model for inference to obtain a set of detection results corresponding to each sampled frame. The set of detection results contains at least one detection result, which includes a defect category, a confidence score, and bounding box coordinates.

[0007] Further, the step of sequentially determining whether the detection result set corresponding to each of the sampling frames is an empty set, and determining the sampling frames whose detection result set is not empty as defective frames, results in multiple defective frames, including: Take one sampling frame from each of the sampling frames and determine whether the detection result set corresponding to the sampling frame is an empty set; The sampled frames whose detection results are a non-empty set are identified as defective frames. The identification operation is repeated until all the sampled frames have been processed, resulting in multiple defective frames.

[0008] Furthermore, the step of restoring the frame number and overlaying the title on each defective frame to obtain multiple target defective frames includes: For each defective frame, the real frame number of the defective frame in the original video is restored according to the sequence number of the defective frame in the sampling sequence and the frame skipping step size, and a frame title is generated according to the real frame number and the total number of detection results in the defective frame. The frame title is superimposed on the image region of the corresponding defective frame to obtain multiple target defective frames.

[0009] Furthermore, arranging the target defect frames into a defect frame queue according to their detection time sequence includes: Obtain the detection timestamps corresponding to each of the target defect frames; Based on the detection timestamps, the target defect frames are arranged into a defect frame queue in chronological order.

[0010] Further, the step of performing grid mosaicking processing on each target defect frame in the defect frame queue to obtain the target canvas includes: Based on the total number of target defect frames in the defect frame queue, determine the number of columns and rows of the jigsaw puzzle canvas, and allocate the entire canvas memory according to the number of columns, the number of rows, the single frame width, and the single frame height to obtain a blank canvas; Calculate the filling position of each target defect frame in the defect frame queue in the blank canvas, and copy each target defect frame to fill the corresponding single frame grid position in the blank canvas to obtain the target canvas.

[0011] Furthermore, the process of overlaying product information and archiving the target canvas to obtain archived quality inspection image files includes: Obtain product information and overlay the product information onto the bottom area of ​​the target canvas to obtain a jigsaw puzzle canvas carrying product information; The jigsaw puzzle canvas carrying product information is output as a single image file, and the image file is saved according to a preset naming rule to obtain an archived quality inspection image file.

[0012] Furthermore, to achieve the above objectives, the present invention also provides a quality inspection and archiving system for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking. This system is used to implement the quality inspection and archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking as described above. The system includes: The frame skipping inference module is used to acquire the industrial video stream to be detected, perform frame skipping sampling and model inference processing on the industrial video stream, and obtain the detection result set corresponding to each sampled frame. The defect determination module is used to sequentially determine whether the detection result set corresponding to each sampling frame is an empty set, and to determine the sampling frame whose detection result set is not an empty set as a defect frame, thereby obtaining multiple defect frames. The title overlay module is used to perform frame number restoration and title overlay processing on each of the defect frames to obtain multiple target defect frames. The queue arrangement module is used to arrange the target defect frames into a defect frame queue according to the detection time order of each target defect frame; The jigsaw puzzle generation module is used to perform grid jigsaw puzzle processing on each target defect frame in the defect frame queue to obtain the target canvas. The archiving output module is used to overlay product information on the target canvas and perform archiving output processing to obtain archived quality inspection image files.

[0013] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a quality inspection and archiving program for automatic extraction of industrial video defect frames and product information perception grid mosaic, stored in the memory and executable on the processor. When the quality inspection and archiving program for automatic extraction of industrial video defect frames and product information perception grid mosaic is executed by the processor, it implements the steps of the quality inspection and archiving method for automatic extraction of industrial video defect frames and product information perception grid mosaic as described above.

[0014] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a quality inspection and archiving program for automatic extraction of industrial video defect frames and product information perception grid mosaic, and when the quality inspection and archiving program for automatic extraction of industrial video defect frames and product information perception grid mosaic is executed by a processor, it implements the steps of the quality inspection and archiving method for automatic extraction of industrial video defect frames and product information perception grid mosaic as described above.

[0015] The beneficial effects of this invention are as follows: By performing frame-skipping sampling and model inference on industrial video streams, this invention significantly reduces the amount of data to be processed while ensuring the coverage of defect detection, thus reducing the consumption of computing resources. Defect frames are determined based on whether the detection result set is empty, ensuring that only frames containing defect targets are retained, preventing defect-free frames from entering subsequent processing and reducing data redundancy in the mosaic operation. After restoring the true frame number and overlaying a title to each defect frame, each defect screenshot carries its precise location information in the original video, providing a reliable location basis for subsequent auditing and tracing. Arranging the defect frame queue according to the detection time sequence ensures that the mosaic layout is consistent with the temporal logic of defect occurrence, making the mosaic result temporally readable. Grid mosaic processing integrates scattered defect screenshots into a structurally unified target canvas by regularly arranging defect frames in rows and columns, improving the centralized display effect of defect evidence. After overlaying product information on the target canvas and archiving the output, a single image file simultaneously carries defect evidence and product-related information, enabling independent storage, retrieval, and transfer of quality inspection files without the need for external files, enhancing the independence and usability of archived files. Attached Figure Description

[0016] Figure 1 This is a flowchart of a preferred embodiment of the quality inspection archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking according to the present invention; Figure 2 This is a structural diagram of a preferred embodiment of the quality inspection and archiving system for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking according to the present invention; Figure 3 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0017] This application provides a method, system, terminal, and storage medium for quality inspection archiving of industrial video defect frames and product information perception grid mosaic. To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description, with reference to the accompanying drawings and embodiments, further illustrates this application. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0018] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0019] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0020] The preferred embodiment of the present invention describes a quality inspection archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking, such as... Figure 1 As shown, the quality inspection archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking includes the following steps: S10. Obtain the industrial video stream to be detected, perform frame skipping sampling and model inference processing on the industrial video stream, and obtain the detection result set corresponding to each sampled frame.

[0021] The purpose of this step is to reduce the input density of video frames by skipping frames, thereby reducing the computational burden on the subsequent target detection model. At the same time, it obtains defect detection information in each sampled frame through model inference, providing a data basis for the subsequent screening of defect frames.

[0022] Further, the step of acquiring the industrial video stream to be detected, performing frame-skipping sampling and model inference processing on the industrial video stream, and obtaining a set of detection results corresponding to each sampled frame includes: S11. Obtain the industrial video stream to be detected, and perform frame skipping sampling on the industrial video stream according to a preset frame skipping step size to obtain multiple sampling frames.

[0023] S12. Input each of the sampled frames into the trained target detection model for inference to obtain a set of detection results corresponding to each sampled frame. The set of detection results includes at least one detection result, and the detection result includes defect category, confidence score and bounding box coordinates.

[0024] In this embodiment, an automated optical inspection scenario is taken as an example of a surface mount technology (SMT) production line for electronic products. An industrial camera deployed at the reflow soldering exit of the production line continuously captures images of the circuit board surface after the SMT process at a frame rate of 30 frames per second, generating an industrial video stream to be inspected. Based on the production line cycle time (one circuit board passes through every 30 seconds) and the inference speed of the target detection model (approximately 15 milliseconds per frame), a frame skipping step size of six is ​​configured. The system performs frame skipping sampling of the video stream according to this step size, i.e., taking one frame every six frames, resulting in approximately 3,000 images. Subsequently, each sampled frame is sequentially input into the trained target detection model (You Only Look Once, YOLO) for forward inference, outputting a set of detection results corresponding to each sampled frame. Each detection result includes a defect category (e.g., short circuit, cold solder joint, insufficient solder), a confidence score, and bounding box coordinates. If no defect target is found in a sampled frame, the set of detection results corresponding to that sampled frame is empty.

[0025] It should be noted that the frame skipping step size can be configured based on one or more factors, such as the actual production line operating cycle time, detection accuracy requirements, and the inference speed of the target detection model. A smaller frame skipping step size results in higher sampling density and lower risk of missed defects, but also increases computational overhead; a larger frame skipping step size results in lower sampling density and higher computational efficiency, but may increase the risk of missed defects. Those skilled in the art can flexibly adjust this parameter according to specific application scenarios.

[0026] S20. Sequentially determine whether the detection result set corresponding to each sampling frame is an empty set, and determine the sampling frame whose detection result set is not an empty set as a defect frame, thus obtaining multiple defect frames.

[0027] The purpose of this step is to accurately filter out frames that actually contain defective targets by using the criterion of whether the detection result set is empty, and to exclude invalid frames that do not contain any defects, thereby reducing the amount of data in subsequent jigsaw puzzle processing and avoiding redundant archiving of defect-free frames.

[0028] Further, the step of sequentially determining whether the detection result set corresponding to each of the sampling frames is an empty set, and determining the sampling frames whose detection result set is not empty as defective frames, to obtain multiple defective frames, includes: S21. Take out a sampling frame from each of the sampling frames and determine whether the detection result set corresponding to the sampling frame is an empty set.

[0029] S22. The sampled frames whose detection results are set into a non-empty set are determined to be defective frames. The above determination operation is repeated until all the sampled frames have been processed to obtain multiple defective frames.

[0030] In this embodiment, the system extracts each of the three thousand sample frames one by one and performs a judgment operation on each sample frame. When the detection result set corresponding to a sample frame is empty, it means that no defect target was detected in the sample frame, and the system discards the sample frame and does not proceed to the subsequent jigsaw puzzle process. When the detection result set corresponding to a sample frame is not empty, for example, if three defect targets are detected in the frame (two short circuits and one cold solder joint), the system determines the sample frame as a defect frame. The system repeats the above judgment operation until all three thousand sample frames have been processed. In this embodiment, a total of forty-seven defect frames were determined, and the remaining sample frames were discarded.

[0031] It should be noted that the criterion in this step is whether the detection result set is empty, rather than the degree of change in the image content or other image features. This ensures that only the image frames that actually contain defects are retained, avoiding the problem of redundant extraction of defect-free frames in general keyframe extraction methods.

[0032] S30. Perform frame number restoration and title overlay processing on each defect frame to obtain multiple target defect frames.

[0033] The purpose of this step is to restore the actual frame number of the sampled frame in the original video and overlay it on the image area of ​​the defective frame as a frame title, so that each defective screenshot carries its precise location information in the original video, which meets the need to locate the time of defect occurrence during subsequent auditing and tracing.

[0034] Further, the step of restoring the frame number and overlaying the title on each defective frame to obtain multiple target defective frames includes: S31. For each defective frame, based on the sequence number of the defective frame in the sampling sequence and the frame skipping step, restore the real frame number of the defective frame in the original video, and generate a frame title based on the real frame number and the total number of detection results in the defective frame.

[0035] S32. The frame title is superimposed on the image area of ​​the corresponding defect frame to obtain multiple target defect frames.

[0036] In this embodiment, the system restores the frame number and overlays a title onto each of the forty-seven identified defective frames. For the defective frame with the sequence number eleven in the sampling sequence, the system adds one to the sequence number and multiplies it by the frame skipping step size of six, i.e., (eleven plus one) multiplied by six, resulting in seventy-two. This value is the true frame number of the defective frame in the original video, indicating that the frame corresponds to the seventy-second frame of the original video. Three defective targets are detected in this defective frame. The system generates a frame title based on the true frame number and the total number of detection results, in the format "Frame 72 - 3 defects". Subsequently, the system overlays this frame title onto the upper area of ​​the defective frame image. The same processing operation is performed on the other 46 defective frames, resulting in forty-seven target defective frames with frame titles.

[0037] It should be noted that the calculation method of the actual frame number depends on the starting numbering method of the sampled frame sequence. This embodiment uses the example of counting sampled frame sequence numbers starting from zero. When the sampled frame sequence numbers start from zero, the actual frame number equals (sampled frame sequence number plus one) multiplied by the frame skipping step size; when the sampled frame sequence numbers start from one, the actual frame number equals the sampled frame sequence number multiplied by the frame skipping step size. Regardless of the counting method used, the essence is to map the position of the sampled frame in the sampling sequence back to the corresponding position on the original video timeline.

[0038] S40. Arrange the target defect frames into a defect frame queue according to the detection time order of each target defect frame.

[0039] The purpose of this step is to arrange the target defect frames in chronological order of their appearance, forming an ordered queue. This prepares the canvas for the defect frames to be filled in chronological order during the subsequent puzzle-making process, ensuring that the arrangement order of each frame in the puzzle is consistent with the actual chronological order of the defects.

[0040] Further, arranging the target defect frames into a defect frame queue according to the detection time order of each target defect frame includes: S41. Obtain the detection timestamp corresponding to each of the target defect frames.

[0041] S42. Based on the detection timestamps, arrange the target defect frames into a defect frame queue in chronological order.

[0042] In this embodiment, the system associates a corresponding detection timestamp with each frame of the video during the video acquisition process. After each defective frame is identified and the frame number is restored and the title is overlaid, the system obtains the detection timestamp corresponding to each target defective frame. Subsequently, the system arranges the forty-seven target defective frames in ascending order of time according to the chronological order indicated by each timestamp, forming an ordered defective frame queue.

[0043] It should be noted that, in addition to the detection timestamp, the sampled frame sequence number or the actual frame number can also be used as the sorting basis. Since both the sampled frame sequence number and the actual frame number have a monotonically increasing relationship with time, sorting using these two values ​​has the same technical effect as sorting using the timestamp. When the video capture frame rate is constant, a larger actual frame number indicates that the frame appeared later; therefore, sorting by actual frame number can also achieve the purpose of chronological order.

[0044] S50. Perform grid mosaic processing on each target defect frame in the defect frame queue to obtain the target canvas.

[0045] The purpose of this step is to combine multiple scattered target defect frames into a neat jigsaw puzzle canvas according to a preset grid layout rule, so as to realize the centralized display of defect evidence. At the same time, the efficiency of jigsaw puzzle generation is improved by one-time memory allocation and direct addressing filling.

[0046] Further, the step of performing grid mosaicking processing on each target defect frame in the defect frame queue to obtain the target canvas includes: S51. Based on the total number of target defect frames in the defect frame queue, determine the number of columns and rows of the jigsaw puzzle canvas, and allocate the entire canvas memory according to the number of columns, the number of rows, the width of a single frame, and the height of a single frame to obtain a blank canvas.

[0047] S52. Calculate the filling position of each target defect frame in the defect frame queue in the blank canvas, and copy and fill each target defect frame to the corresponding single frame grid position in the blank canvas to obtain the target canvas.

[0048] In this embodiment, the system first obtains a preset column number configuration value, which defaults to four. The total number of target defect frames is forty-seven. The system takes the smaller of the preset column number four and the total number of target defect frames forty-seven as the actual number of columns in the puzzle, i.e., the actual number of columns is four. Then, the system divides the total number of target defect frames forty-seven by the actual number of columns four, obtaining a remainder of 11 and a floor of 3. The quotient is rounded up to obtain twelve, meaning the actual number of rows in the puzzle is twelve. The system obtains a single frame width of 640 pixels and a single frame height of 640 pixels. Multiplying the actual number of columns four by the single frame width of 640 pixels yields a total canvas width of 2560 pixels, and multiplying the actual number of rows twelve by the single frame height of 640 pixels yields a total canvas height of 7680 pixels. The system allocates memory space for the entire blank canvas at once based on this total canvas width and total canvas height.

[0049] Subsequently, the system calculates the fill position in the blank canvas for each target defect frame in the defect frame queue. For the target defect frame with sequence number 24 (i.e., the 25th target defect frame) in the defect frame queue, the system divides the sequence number 24 by the actual column number 4, obtaining an integer quotient of 6 and a remainder of 0. The integer quotient of 6 is used as the row number of the target defect frame, and the remainder of 0 is used as the column number of the target defect frame. The system multiplies the column number 0 by the single frame width of 640 pixels to obtain an x-coordinate of 0, and multiplies the row number 6 by the single frame height of 640 pixels to obtain a y-coordinate of 3840 pixels. The system uses these x-coordinates and y-coordinates as the starting fill position and copies and fills the target defect frame into the corresponding single frame area in the blank canvas. The system performs the above calculation and fill operation on all 47 target defect frames in the defect frame queue. The last row (row 12) only fills the first three single frame positions (corresponding to target defect frames with sequence numbers 44, 45, and 46 in the queue). The fourth single frame position is not filled, and the system keeps the empty space with a black background.

[0050] It should be noted that the default number of columns is four, but this value can be configured according to actual needs. When the total number of defect frames is small, the actual number of columns will adaptively decrease to the total number of defect frames. For example, when the total number of defect frames is three, the actual number of columns is three and the number of rows is one, resulting in a three-column layout for the puzzle. When the total number of defect frames is large, the default number of columns can be increased to reduce the number of rows, making the puzzle wider and shorter for easier viewing on the display screen. Furthermore, the canvas memory is allocated in a one-time manner, which significantly reduces the number of memory allocations and computational overhead compared to the method of expanding the canvas size grid by grid.

[0051] S60. Perform product information overlay and archiving output processing on the target canvas to obtain the archived quality inspection image file.

[0052] The purpose of this step is to overlay product information onto the jigsaw puzzle canvas, so that each output image file carries complete product identification and testing summary information, which can be independently archived and circulated without the need for additional external documents, thus eliminating the risk of mismatch between evidence images and product information.

[0053] Further, the process of overlaying product information and archiving the target canvas to obtain the archived quality inspection image file includes: S61. Obtain product information and overlay the product information onto the bottom area of ​​the target canvas to obtain a jigsaw puzzle canvas carrying product information.

[0054] S62. Output the jigsaw puzzle canvas carrying product information as a single image file, save the image file according to the preset naming rules, and obtain the archived quality inspection image file.

[0055] In this embodiment, the system acquires the product information of the current product, including the product model "PCBA-1234", batch number "BATCH-20260721-01", production date "2026-07-21", and total number of defects "47". The system overlays a product information bar at the bottom of the target canvas. The height of this information bar is 80 pixels, and its width is the same as the total width of the canvas (2,560 pixels). The above product information fields are arranged sequentially in the information bar. After the overlay is completed, the system outputs the jigsaw puzzle canvas carrying the product information as a single PNG image file, and names it according to the preset naming rule "product model_batch number_production date_defect jigsaw puzzle_serial number.png", resulting in an archived quality inspection image file named "PCBA-1234_BATCH-20260721-01_20260721_defect jigsaw puzzle_01.png", which is then saved to the archiving server.

[0056] It should be noted that the fields included in the product information bar can be customized according to the quality management system requirements of different enterprises. For example, fields such as inspection station number, inspector's name, auditor's name, and quality grade can be added. The output file format is not limited to PNG format; it can also be JPEG, BMP, or other image formats, and can be extended to PDF document format for easy printing and binding. Those skilled in the art can flexibly select the output format and product information fields according to actual archiving needs.

[0057] The beneficial effects of this invention are as follows: By performing frame-skipping sampling and model inference on industrial video streams, this invention significantly reduces the amount of data to be processed while ensuring the coverage of defect detection, thus reducing the consumption of computing resources. Defect frames are determined based on whether the detection result set is empty, ensuring that only frames containing defect targets are retained, preventing defect-free frames from entering subsequent processing and reducing data redundancy in the mosaic operation. After restoring the true frame number and overlaying a title to each defect frame, each defect screenshot carries its precise location information in the original video, providing a reliable location basis for subsequent auditing and tracing. Arranging the defect frame queue according to the detection time sequence ensures that the mosaic layout is consistent with the temporal logic of defect occurrence, making the mosaic result temporally readable. Grid mosaic processing integrates scattered defect screenshots into a structurally unified target canvas by regularly arranging defect frames in rows and columns, improving the centralized display effect of defect evidence. After overlaying product information on the target canvas and archiving the output, a single image file simultaneously carries defect evidence and product-related information, enabling independent storage, retrieval, and transfer of quality inspection files without the need for external files, enhancing the independence and usability of archived files. As can be seen, the quality inspection archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking described in this invention realizes the fully automatic generation of standardized quality inspection evidence documents from industrial inspection videos, eliminating the need for manual review and manual mosaicking, thus improving the efficiency and standardization of quality inspection archiving.

[0058] Furthermore, such as Figure 2 As shown, based on the above-mentioned method for automatic extraction of defect frames from industrial videos and quality inspection archiving of product information perception grid mosaic, this invention also provides a quality inspection archiving system for automatic extraction of defect frames from industrial videos and quality inspection archiving of product information perception grid mosaic. The quality inspection archiving system for automatic extraction of defect frames from industrial videos and quality inspection archiving of product information perception grid mosaic includes: The frame skipping inference module 51 is used to acquire the industrial video stream to be detected, perform frame skipping sampling and model inference processing on the industrial video stream, and obtain the detection result set corresponding to each sampled frame. Defect determination module 52 is used to sequentially determine whether the detection result set corresponding to each sampling frame is an empty set, and to determine the sampling frame whose detection result set is not an empty set as a defect frame, thereby obtaining multiple defect frames. The title overlay module 53 is used to perform frame number restoration and title overlay processing on each of the defect frames to obtain multiple target defect frames. The queue arrangement module 54 is used to arrange the target defect frames into a defect frame queue according to the detection time order of each target defect frame; The jigsaw puzzle generation module 55 is used to perform grid jigsaw puzzle processing on each target defect frame in the defect frame queue to obtain the target canvas. The archiving output module 56 is used to overlay product information and perform archiving output processing on the target canvas to obtain archived quality inspection image files.

[0059] The aforementioned modules constitute the fully automated processing system of this invention, from video input to archived file output. Its technical effectiveness lies in the coordinated operation of multiple processing stages. The frame-skipping inference module, through frame-skipping sampling and model inference of the video stream, effectively reduces computational resource consumption while ensuring defect coverage. The defect determination module uses the emptiness of the detection result set as the criterion, retaining only frames containing defect targets, thus simplifying the data volume processed by subsequent modules. The title overlay module restores the true frame number of each defect frame and overlays a title, ensuring that each defect screenshot carries its precise location identifier in the original video. The queue arrangement module sorts the target defect frames according to the detection time sequence, maintaining consistency between the defect arrangement in the jigsaw puzzle canvas and the time logic. The jigsaw puzzle generation module integrates scattered defect screenshots into a structurally regular canvas through gridding processing, enhancing the concentration of defect evidence display. The archived output module overlays product information onto the canvas and outputs a single image file, linking defect evidence and product information within the same file. As can be seen, the quality inspection archiving system for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking described in this invention realizes the fully automatic generation of standardized quality inspection evidence documents from industrial inspection videos, eliminating the need for manual review and manual mosaicking, thus improving the efficiency and standardization of quality inspection archiving.

[0060] Furthermore, such as Figure 3 As shown, based on the above-mentioned method and system for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking for quality inspection archiving, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 3 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0061] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal. Further, the memory 20 may include both internal and external storage units of the terminal. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a quality inspection and archiving program 40 for automatic extraction of industrial video defect frames and product information perception grid mosaicking. This quality inspection and archiving program 40 can be executed by the processor 10, thereby realizing the quality inspection and archiving method for automatic extraction of industrial video defect frames and product information perception grid mosaicking in this application.

[0062] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the quality inspection archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaic.

[0063] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components of the terminal communicate with each other via a system bus.

[0064] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a quality inspection and archiving program for automatic extraction of defective frames from industrial videos and product information perception grid mosaicking, and when the quality inspection and archiving program for automatic extraction of defective frames from industrial videos and product information perception grid mosaicking is executed by a processor, it implements the steps of the quality inspection and archiving method for automatic extraction of defective frames from industrial videos and product information perception grid mosaicking as described above.

[0065] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0066] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0067] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A quality inspection archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking, characterized in that, The quality inspection archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking includes the following steps: The industrial video stream to be detected is acquired, and the industrial video stream is subjected to frame skipping sampling and model inference processing to obtain the detection result set corresponding to each sampled frame; Sequentially determine whether the detection result set corresponding to each sampling frame is an empty set, and determine the sampling frame whose detection result set is not an empty set as a defect frame to obtain multiple defect frames; Each defective frame is restored by its frame number and its title is overlaid to obtain multiple target defective frames. Arrange the target defect frames into a defect frame queue according to the detection time order of each target defect frame; The target defect frames in the defect frame queue are subjected to grid mosaic processing to obtain the target canvas; The target canvas is processed by overlaying product information and archiving output to obtain archived quality inspection image files.

2. The quality inspection archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking according to claim 1, characterized in that, The process involves acquiring the industrial video stream to be detected, performing frame-skipping sampling and model inference on the industrial video stream, and obtaining a set of detection results corresponding to each sampled frame, including: The industrial video stream to be detected is acquired, and the industrial video stream is sampled by skipping frames according to a preset frame skipping step size to obtain multiple sampled frames; Each of the sampled frames is sequentially input into the trained target detection model for inference to obtain a set of detection results corresponding to each sampled frame. The set of detection results contains at least one detection result, which includes a defect category, a confidence score, and bounding box coordinates.

3. The quality inspection archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking according to claim 1, characterized in that, The step of sequentially determining whether the detection result set corresponding to each of the sampling frames is an empty set, and determining the sampling frames whose detection result set is not an empty set as defect frames, results in multiple defect frames, including: Take one sampling frame from each of the sampling frames and determine whether the detection result set corresponding to the sampling frame is an empty set; The sampled frames whose detection results are a non-empty set are identified as defective frames. The identification operation is repeated until all the sampled frames have been processed, resulting in multiple defective frames.

4. The quality inspection and archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking according to claim 1, characterized in that, The process of restoring the frame number and overlaying the title on each defective frame yields multiple target defective frames, including: For each defective frame, the real frame number of the defective frame in the original video is restored according to the sequence number of the defective frame in the sampling sequence and the frame skipping step size, and a frame title is generated according to the real frame number and the total number of detection results in the defective frame. The frame title is superimposed on the image region of the corresponding defective frame to obtain multiple target defective frames.

5. The quality inspection archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking according to claim 1, characterized in that, The step of arranging the target defect frames into a defect frame queue according to the detection time sequence of each target defect frame includes: Obtain the detection timestamps corresponding to each of the target defect frames; Based on the detection timestamps, the target defect frames are arranged into a defect frame queue in chronological order.

6. The quality inspection archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking according to claim 1, characterized in that, The step of performing grid mosaicking processing on each target defect frame in the defect frame queue to obtain the target canvas includes: Based on the total number of target defect frames in the defect frame queue, determine the number of columns and rows of the jigsaw puzzle canvas, and allocate the entire canvas memory according to the number of columns, the number of rows, the single frame width, and the single frame height to obtain a blank canvas; Calculate the filling position of each target defect frame in the defect frame queue in the blank canvas, and copy each target defect frame to fill the corresponding single frame grid position in the blank canvas to obtain the target canvas.

7. The quality inspection archiving method for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking according to claim 1, characterized in that, The process of overlaying product information and archiving output on the target canvas to obtain archived quality inspection image files includes: Obtain product information and overlay the product information onto the bottom area of ​​the target canvas to obtain a jigsaw puzzle canvas carrying product information; The jigsaw puzzle canvas carrying product information is output as a single image file, and the image file is saved according to a preset naming rule to obtain an archived quality inspection image file.

8. A quality inspection and archiving system for automatic extraction of defect frames from industrial videos and product information perception grid mosaicking, characterized in that, The quality inspection and archiving system for automatic extraction of defective frames from industrial videos and product information perception grid mosaicking is used to implement the quality inspection and archiving method for automatic extraction of defective frames from industrial videos and product information perception grid mosaicking as described in any one of claims 1-7. The quality inspection and archiving system for automatic extraction of defective frames from industrial videos and product information perception grid mosaicking includes: The frame skipping inference module is used to acquire the industrial video stream to be detected, perform frame skipping sampling and model inference processing on the industrial video stream, and obtain the detection result set corresponding to each sampled frame. The defect determination module is used to sequentially determine whether the detection result set corresponding to each sampling frame is an empty set, and to determine the sampling frame whose detection result set is not an empty set as a defect frame, thereby obtaining multiple defect frames. The title overlay module is used to perform frame number restoration and title overlay processing on each of the defect frames to obtain multiple target defect frames. The queue arrangement module is used to arrange the target defect frames into a defect frame queue according to the detection time order of each target defect frame; The jigsaw puzzle generation module is used to perform grid jigsaw puzzle processing on each target defect frame in the defect frame queue to obtain the target canvas. The archiving output module is used to overlay product information on the target canvas and perform archiving output processing to obtain archived quality inspection image files.

9. A terminal, characterized in that, The terminal includes: a memory, a processor, and a quality inspection and archiving program for automatic extraction of industrial video defect frames and product information perception grid mosaic, stored in the memory and executable on the processor. When the quality inspection and archiving program for automatic extraction of industrial video defect frames and product information perception grid mosaic is executed by the processor, it implements the steps of the quality inspection and archiving method for automatic extraction of industrial video defect frames and product information perception grid mosaic as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a quality inspection and archiving program for automatic extraction of defective frames from industrial videos and product information perception grid mosaicking. When the quality inspection and archiving program for automatic extraction of defective frames from industrial videos and product information perception grid mosaicking is executed by a processor, it implements the steps of the quality inspection and archiving method for automatic extraction of defective frames from industrial videos and product information perception grid mosaicking as described in any one of claims 1-7.

Images