Data set construction method, device and equipment

Abstract

The embodiment of the invention provides a data set construction method, device and equipment. The method comprises the following steps: acquiring an image sequence of target objects of the same model, and a key frame and a prediction frame in the image sequence; on the basis of an intra-frame prediction technology, carrying out coding processing on key frames in the image sequence, and carrying out coding processing on prediction frames in the image sequence on the basis of an entropy coding technology to obtain a video file; carrying out decoding processing on the video file to obtain a decoded key frame and a decoded prediction frame; performing defect detection processing on the decoded key frame, and determining a sample category of the decoded key frame; based on the sample category of the decoded key frame, determining the sample category of the decoded prediction frame; and constructing a data set of the target object based on the sample category of the decoded key frame and the sample category of the decoded prediction frame. The method is used for achieving the effect of improving the data set construction efficiency.

Description

Technical Field

[0001] This application relates to the field of industrial testing technology, and in particular to a method, apparatus and equipment for constructing a dataset. Background Technology

[0002] In the quality inspection of used mobile phones, X-ray inspection models are widely used to identify internal structural defects such as poor motherboard soldering, battery swelling, and foreign object residue.

[0003] The training and iteration of detection models rely on high-quality, reusable X-ray labeled datasets. Currently, most methods involve manually analyzing X-ray images frame by frame and manually classifying positive and negative samples, resulting in low dataset construction efficiency. This directly affects the training speed and generalization ability of the detection model, ultimately hindering the large-scale application and performance improvement of X-ray detection technology. Summary of the Invention

[0004] The dataset construction method, apparatus, and device provided in this application are intended to improve the efficiency of dataset construction.

[0005] In a first aspect, embodiments of this application provide a method for constructing a dataset, comprising:

[0006] Acquire an image sequence of a target object of the same model, and keyframes and prediction frames in the image sequence; wherein the image sequence includes multiple X-ray images arranged in time;

[0007] Based on intra-frame prediction technology, keyframes in the image sequence are encoded, and based on entropy coding technology, predicted frames in the image sequence are encoded to obtain a video file.

[0008] The video file is decoded to obtain decoded keyframes and decoded predicted frames;

[0009] Defect detection processing is performed on the decoded keyframes to determine the sample category of the decoded keyframes;

[0010] Based on the sample categories of the decoded keyframes, determine the sample categories of the decoded predicted frames;

[0011] The dataset of the target object is constructed based on the sample categories of the decoded keyframes and the sample categories of the decoded predicted frames.

[0012] In one possible implementation, obtaining keyframes and predicted frames from the image sequence includes:

[0013] Determine the complexity of each image in the image sequence;

[0014] Based on the complexity of each image, multiple keyframes are determined from the image sequence;

[0015] Use the image between every two keyframes as the prediction frame.

[0016] In one possible implementation, determining multiple keyframes from the image sequence based on the complexity of each image includes:

[0017] For the i-th frame image in the image sequence, perform the following steps, where the initial value of i is 1:

[0018] The interval counter value is incremented by one;

[0019] If it is determined that the value of the interval counter is greater than or equal to the preset maximum keyframe interval, then the i-th frame image is set as a keyframe, and the value of the interval counter is set to 0;

[0020] If it is determined that the value of the interval counter is greater than or equal to the preset minimum keyframe interval and less than the preset maximum keyframe interval, then it is determined whether the complexity of the i-th frame image is greater than a third preset threshold; if yes, then the i-th frame image is set as a keyframe and the value of the interval counter is set to 0; if no, then the value of the interval counter remains unchanged.

[0021] If it is determined that the value of the interval counter is less than the preset minimum keyframe interval, then the value of the interval counter remains unchanged.

[0022] In one possible implementation, determining multiple keyframes from the image sequence based on the complexity of each image includes:

[0023] Based on the image sequence and the preset keyframe interval, multiple sub-sequences are determined;

[0024] For each subsequence, the average complexity of the subsequence is determined based on the complexity of the images included in the subsequence;

[0025] The keyframes of the subsequence are determined based on the average complexity of the subsequence.

[0026] In one possible implementation, determining the keyframes of the subsequence based on the average complexity of the subsequence includes:

[0027] If the average complexity of the subsequence is determined to be greater than a first preset threshold, the images included in the subsequence are sorted in descending order of complexity, and the first N images are used as keyframes of the subsequence; N is an integer greater than 2.

[0028] If the average complexity of the subsequence is determined to be less than or equal to a first preset threshold, then the adjacent subsequent subsequences of the subsequence are merged as a new subsequence, and the average complexity of the new subsequence is calculated.

[0029] In one possible implementation, determining the complexity of each image in the image sequence includes:

[0030] For each image in the image sequence, the image is converted into a grayscale image, and the grayscale distribution entropy value of the grayscale image is determined;

[0031] Based on the edge detection algorithm, the edge pixels of the grayscale image are extracted, and the edge density of the grayscale image is determined based on the edge pixels.

[0032] The complexity of the image is determined based on the grayscale distribution entropy value and the edge density.

[0033] In one possible implementation, determining the sample category of the decoded predicted frame based on the sample category of the decoded keyframe includes:

[0034] For each intermediate frame, determine the similarity between the intermediate frame and the preceding decoded keyframe; wherein, the intermediate frame is the decoded predicted frame between two decoded keyframes;

[0035] If the similarity is determined to be greater than the second preset threshold, the sample category of the decoded predicted frame is determined to be consistent with the sample category of the previously decoded keyframe.

[0036] Secondly, embodiments of this application provide a dataset construction apparatus, comprising:

[0037] An acquisition module is used to acquire an image sequence of a target object of the same model, as well as keyframes and prediction frames in the image sequence; wherein the image sequence includes multiple X-ray images arranged in time.

[0038] The encoding module is used to encode keyframes in the image sequence based on intra-frame prediction technology, and to encode predicted frames in the image sequence based on entropy coding technology to obtain a video file.

[0039] The decoding module is used to decode the video file to obtain decoded keyframes and decoded predicted frames.

[0040] The detection module is used to perform defect detection processing on the decoded keyframes and determine the sample category of the decoded keyframes.

[0041] The determination module is used to determine the sample category of the decoded predicted frame based on the sample category of the decoded keyframe;

[0042] A construction module is used to construct a dataset of the target object based on the sample categories of the decoded keyframes and the sample categories of the decoded predicted frames.

[0043] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0044] The memory stores computer-executed instructions;

[0045] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0046] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0047] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0048] The dataset construction method, apparatus, and device provided in this application involve acquiring a time-series X-ray image sequence of a target object of the same model and determining keyframes and prediction frames in the image sequence. Intra-frame prediction and entropy coding techniques are used to generate video files of the image sequence. After decoding and restoring the images to standardized images, defect detection is performed on the decoded keyframes to determine their sample categories. Then, based on the sample categories of the decoded keyframes, the sample categories of the decoded prediction frames are inferred, ultimately constructing a dataset of the target object. This method, which infers the sample categories of prediction frames based on keyframe settings and keyframe sample categories, reduces redundant workload, eliminates the need for manual annotation of all images, improves dataset construction efficiency, reduces subjective errors in manual judgment, increases annotation accuracy, and thus shortens the development cycle of the detection model. Attached Figure Description

[0049] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0050] Figure 1 A flowchart illustrating the method for constructing the dataset provided in this application. Figure 1 ;

[0051] Figure 2 A flowchart illustrating the method for constructing the dataset provided in this application. Figure 2 ;

[0052] Figure 3 A flowchart illustrating the method for constructing the dataset provided in this application. Figure 3 ;

[0053] Figure 4 A schematic diagram of the structure of the apparatus for constructing the dataset provided in this application;

[0054] Figure 5 A schematic diagram of the structure of the electronic device provided in this application.

[0055] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0056] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0057] The construction of X-ray datasets in the industrial inspection field mainly involves manually examining X-ray images frame by frame, manually classifying and labeling positive and negative samples. This method is inefficient and costly. The inventors of this application observed the similarity between X-ray image sequences and video streams, treating the X-ray image sequences as video streams. They utilized the independent coding characteristics of keyframes as classification anchors, inferring the category of intermediate frames through inter-frame similarity analysis, thereby reducing redundant analysis workload and improving the efficiency of dataset construction.

[0058] The execution subject of this application embodiment can be an electronic device with processing capabilities, such as a computer, server, etc., and this application embodiment is not limited thereto.

[0059] It should be noted that the user information and user data involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant laws, regulations and standards, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0060] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0061] Figure 1 A flowchart illustrating the method for constructing the dataset provided in this application. Figure 1 ,like Figure 1 As shown, the method includes:

[0062] S101. Obtain an image sequence of the same type of target object, as well as keyframes and prediction frames in the image sequence.

[0063] For example, the target object refers to the item to be tested, which may be a product such as a mobile phone or tablet, or an industrial part, electronic component, or mechanical assembly. This application embodiment does not limit the type of item to be tested. Target objects of the same model refer to target objects with completely identical attributes such as brand, series, and specific model.

[0064] An image sequence is a collection of multiple consecutively captured images arranged in chronological order, where each image is an X-ray image. X-ray images are obtained through X-ray imaging technology, which can penetrate the surface of a target object and reveal its internal structure, and can be used to detect internal defects.

[0065] Keyframes are representative images with anchor point attributes in an image sequence. They are encoded independently and can be decoded without relying on other images.

[0066] Predicted frames refer to images other than keyframes that require the encoding information of adjacent keyframes to complete decoding. They are redundant or similar feature images that account for a higher proportion in the image sequence.

[0067] In one example, after continuously photographing the same type of target object using an industrial X-ray inspection device, the images are sorted by shooting timestamps to form an image sequence. Then, based on a preset keyframe selection rule, keyframes are determined from the image sequence, and the remaining images are used as prediction frames. The preset keyframe selection rule can be, for example, a fixed interval selection, such as selecting a keyframe every 10 frames; or it can be based on an adaptively adjusted interval, for example, automatically reducing the interval to select a keyframe when a sudden change in structural features (such as a suspected defect shadow) is detected in adjacent images, with the remaining images used as prediction frames. Optionally, a unique frame ID can be assigned to each image.

[0068] S102. Based on intra-frame prediction technology, key frames in the image sequence are encoded, and based on entropy coding technology, predicted frames in the image sequence are encoded to obtain a video file.

[0069] For example, intra-frame prediction technology is a technique that uses the internal features of the current frame, such as pixel distribution, texture structure, and edge features, for encoding. During the encoding process, these internal features are transformed into structured intra-frame prediction coding information. This intra-frame prediction coding information is stored independently and does not depend on other frame data, ensuring that key frames have independent decoding capabilities. This intra-frame prediction coding information can be, for example, prediction mode data based on pixel blocks or an edge feature mapping table, etc., and is not limited to this embodiment.

[0070] Entropy coding is a lossless compression technique based on the statistical characteristics of data. It reduces data redundancy in prediction frames by assigning short codes to frequently occurring feature differences and long codes to low-frequency information. Entropy coding can employ methods such as Huffman coding and arithmetic coding, which are not limited to the specific implementation details in this application.

[0071] Encoding processing refers to the process of converting discrete image sequences into a continuous video stream according to a preset video encoding format. It is necessary to simultaneously preserve inter-frame temporal relationships and feature associations, and store intra-frame predictive coding information for keyframes and inter-frame difference information for predicted frames. It should be noted that this application does not limit the type of video encoding format; for example, it can be the H.265 encoding format.

[0072] A video file is a structured file containing keyframes and prediction frames. It can be understood that the metadata of a video file embeds information such as the frame ID, encoding parameters, intra-frame predictive coding information, and inter-frame difference information for each frame, facilitating subsequent decoding processing.

[0073] In one example, the electronic device uses intra-frame prediction technology to extract the structural features of the keyframe in the image sequence and encode them independently, converting them into intra-frame prediction coding information to ensure that the image information can be completely restored after individual decoding. For the predicted frame, entropy coding technology is used to compress and record the inter-frame difference information (such as pixel offset and grayscale value change) between the predicted frame and the preceding and following keyframes, reducing the storage volume. Furthermore, during the encoding process, the assigned unique frame ID is embedded in the video metadata to generate a video file.

[0074] S103. Decode the video file to obtain the decoded keyframes and decoded prediction frames in the video file.

[0075] For example, decoding is the reverse process of encoding. It refers to the process of using a decoder that matches the encoding format to parse the encoded data in the video file and restore the original image features of keyframes and prediction frames. Understandably, during decoding, it is necessary to extract information such as frame IDs, encoding parameters, intra-frame predictive coding information, and inter-frame difference information from the video metadata to ensure that the restored image corresponds one-to-one with the images in the image sequence.

[0076] During the decoding process, the decoding of keyframes does not depend on other frames and the complete image can be directly restored through intra-frame predictive coding information; the decoding of predictive frames requires calling the coding data of associated keyframes in order to restore their feature association with the keyframes.

[0077] Decoded keyframes are normalized images reconstructed through inverse operations on intra-frame predictive coding information, retaining the internal structural features of the original keyframes. Decoded prediction frames are normalized images reconstructed through inverse operations on the features of decoded keyframe images combined with inter-frame difference information, retaining their temporal and feature relationships with the keyframes.

[0078] In one example, an electronic device can read the video metadata of a video file, extract all frame IDs and their corresponding encoding parameters, and then first decode the keyframes. The intra-frame predictive coding information is then input into the inverse prediction module of the decoder to independently reconstruct the keyframe image. Based on the decoding results of the keyframes, the inter-frame difference information is called to decode the prediction frame in order to fully reconstruct the image features of the prediction frame, thus obtaining the decoded keyframes and the decoded prediction frames.

[0079] S104. Perform defect detection processing on the decoded keyframes to determine the sample category of the decoded keyframes.

[0080] For example, defect detection processing refers to automatically identifying the type of defect in the decoded keyframe using a preset artificial intelligence model. It should be noted that this application embodiment does not limit the type of defect; defect types may include, for example, poor motherboard soldering, battery swelling, foreign object residue, missing components, water corrosion, etc. It should also be noted that this application embodiment does not limit the type of artificial intelligence model.

[0081] Samples can be categorized into positive samples and negative samples. Positive samples are keyframes without any defects, while negative samples are keyframes containing at least one defect, with the defect type clearly labeled.

[0082] In one example, an electronic device can use a ResNet-based defect classification model for defect detection. The decoded keyframes are input into the defect classification model, which outputs either a positive sample or a negative sample plus the sample category of the defect type.

[0083] S105. Based on the sample categories of the decoded keyframes, determine the sample categories of the decoded predicted frames.

[0084] For example, as mentioned above, there is an inter-frame correlation between the decoded predicted frame and the decoded keyframe, and the features of adjacent frames in the X-ray image sequence are highly continuous. Therefore, the sample category of the decoded predicted frame can be determined based on the sample category of the decoded keyframe, eliminating the need to repeatedly perform defect detection processing or manual annotation on the decoded predicted frame, thus reducing the number of detections and improving the efficiency of dataset construction. In other words, by quantifying the feature similarity between the decoded predicted frame and its nearest decoded keyframe, a higher value indicates that the inter-frame features are closer, thereby determining that the sample category of the decoded predicted frame is consistent with the sample category of the decoded keyframe, eliminating the need to perform defect detection processing on the decoded keyframe again.

[0085] In one example, for each decoded predicted frame, the image feature vector of that frame is extracted, and the image feature vector of the nearest decoded keyframe is also extracted. The cosine similarity between the two feature vectors is calculated as the feature similarity. If the feature similarity is greater than or equal to a preset threshold, and the corresponding decoded keyframe is a positive sample, then the decoded predicted frame is determined to be a positive sample; if the feature similarity is greater than or equal to the preset threshold, and the corresponding decoded keyframe is a negative sample, then the decoded predicted frame is determined to be a negative sample of the corresponding defect type; if the feature similarity is less than the preset threshold, it is marked as a pending frame, stored separately, and its sample category is manually labeled; alternatively, the pending frame is input into the aforementioned defect classification model for detection processing, and the sample category of the pending frame is output.

[0086] S106. Construct a dataset of the target object based on the sample categories of the decoded keyframes and the sample categories of the decoded predicted frames.

[0087] For example, a dataset refers to a structured collection of standardized images and sample categories containing decoded keyframes and decoded predicted frames.

[0088] In one example, an electronic device can combine the decoded keyframes and the decoded predicted frames of samples classified as positive samples to obtain a positive sample dataset, and combine the decoded keyframes and the decoded predicted frames of samples classified as negative samples to obtain a negative sample dataset.

[0089] The dataset construction method provided in this application involves acquiring a time-series X-ray image sequence of the same type of target object and determining the keyframes and prediction frames in the image sequence. Intra-frame prediction and entropy coding techniques are used to generate video files of the image sequence. After decoding and restoring the images to standardized images, defect detection is performed on the decoded keyframes to determine their sample categories. Then, based on the sample categories of the decoded keyframes, the sample categories of the decoded prediction frames are inferred, ultimately constructing a dataset of the target object. This method, which infers the sample categories of prediction frames based on keyframe settings and keyframe sample categories, reduces redundant workload, eliminates the need for manual annotation of all images, improves dataset construction efficiency, reduces subjective errors in manual judgment, increases annotation accuracy, and thus shortens the development cycle of the detection model.

[0090] Figure 2 A flowchart illustrating the method for constructing the dataset provided in this application. Figure 2 ,like Figure 2 As shown, in this embodiment... Figure 1 Based on the examples, the method for constructing the dataset is described in detail, including:

[0091] S201. Obtain an image sequence of the same type of target object.

[0092] It should be noted that this step is similar to the aforementioned step S101, and will not be repeated here.

[0093] S202. Determine the complexity of each image in the image sequence.

[0094] For example, the complexity of an image refers to the richness and diversity of its internal structural features. The higher the complexity, the more detailed information the image contains, and the more attention it needs to receive.

[0095] In some possible implementations, the electronic device may have a pre-set artificial intelligence model, into which each image is input and the complexity of the image is output. It should be noted that this application does not limit the type of artificial intelligence model; for example, it may be a neural network model.

[0096] In some possible implementations, for each image in the image sequence, the image is converted into a grayscale image, and the grayscale distribution entropy value of the grayscale image is determined; based on an edge detection algorithm, the edge pixels of the grayscale image are extracted, and the edge density of the grayscale image is determined based on the edge pixels; based on the grayscale distribution entropy value and the edge density, the complexity of the image is determined.

[0097] A grayscale image is a single-channel image that retains brightness information but removes color information. Grayscale entropy is an indicator that measures the dispersion of grayscale values ​​in a grayscale image. It reflects the non-uniformity of the grayscale distribution in the image. The higher the entropy value, the more dispersed the pixel distribution of different grayscale levels in the image, and the more detailed the features may be. The lower the entropy value, the more concentrated the grayscale distribution and the simpler the image features.

[0098] Edge detection algorithms are techniques for extracting regions of abrupt changes in grayscale values ​​(i.e., edges) in an image. Edges are key markers of structural changes in an image (such as motherboard outlines, component boundaries, and the boundary between defects and normal areas). This application does not limit the type of edge detection algorithm used; for example, it could be the Canny algorithm, the Sobel algorithm, etc. Edge pixels refer to pixels with abrupt changes in grayscale values ​​selected by the edge detection algorithm. Edge density refers to the proportion of edge pixels to the total number of pixels in the image, reflecting the complexity of the image's internal structure. Higher edge density indicates more and more complex structural outlines in the image, potentially containing more defects or complex component distributions.

[0099] Specifically, the electronic device reads a single X-ray image from the image sequence, obtains its original color channel data, and, based on each pixel, performs a weighted average of the multi-channel color data to obtain a grayscale value. The grayscale value is then normalized to ensure it falls within the range of 0-255, resulting in a single-channel grayscale image. Next, it iterates through all pixels in the grayscale image, counting the number of pixels corresponding to each grayscale level (256 levels in total, 0-255), calculating the probability of each grayscale level, and then substituting the probability of each grayscale level into the entropy calculation formula to obtain the grayscale distribution entropy value of the grayscale image. Simultaneously, edge pixels in the grayscale image are extracted using an edge detection algorithm, and the total number of edge pixels and the total number of pixels are determined. The ratio of the total number of edge pixels to the total number of pixels is used as the edge density. Furthermore, the electronic device can preset weight coefficients for the grayscale distribution entropy value and the edge density, allowing for a weighted sum of the grayscale distribution entropy value and the edge density to obtain the image complexity.

[0100] S203. Based on the complexity of each image, determine multiple keyframes from the image sequence.

[0101] For example, as mentioned above, a key frame is a representative image in an image sequence that reflects the core features of the image. Therefore, images with high complexity can be selected as key frames to ensure that potential defective frames are not missed, while also taking into account the representativeness of features and avoiding redundancy caused by an excessive number of key frames.

[0102] In some possible implementations, the images in the image sequence are sorted in descending order of complexity, and the top M images are set as keyframes. Here, M is an integer greater than 2, and the specific value can be set based on the required number of keyframes.

[0103] Among some possible implementations, Figure 3 A flowchart illustrating the method for constructing the dataset provided in this application. Figure 3 ,like Figure 3 As shown, step S203 may include the following sub-steps:

[0104] For the i-th frame of the image sequence, perform the following steps, where the initial value of i is 1:

[0105] S2031, increment the value of the interval counter by one.

[0106] For example, the i-th frame image refers to the i-th X-ray image in the image sequence, sorted by acquisition timestamp. The interval counter refers to the number of consecutive frames without a keyframe since the last keyframe was set. Its initial value is 0, and it is used to control the setting frequency of keyframes to avoid being too dense or too sparse.

[0107] S2032. Determine whether the value of the interval counter is greater than or equal to the preset maximum keyframe interval.

[0108] For example, the preset minimum keyframe interval refers to the minimum number of consecutive frames without keyframes allowed, used to limit the minimum interval between keyframe settings and avoid redundancy caused by excessively dense keyframes. The preset maximum keyframe interval refers to the maximum number of consecutive frames without keyframes allowed, used as a fallback; even if the image complexity is low, keyframes must be set if the maximum keyframe interval is reached to avoid missing potential defects.

[0109] If the value of the interval counter is determined to be greater than or equal to the preset maximum keyframe interval, then step S2033 is executed; if the value of the interval counter is determined to be greater than or equal to the preset minimum keyframe interval, and less than the preset maximum keyframe interval, then step S2034 is executed; if the value of the interval counter is determined to be less than the preset minimum keyframe interval, then step S2036 is executed.

[0110] S2033. Set the i-th frame as a keyframe and set the value of the interval counter to 0.

[0111] After this step is completed, increment the value of i by one, which means processing the new i-th frame image.

[0112] S2034. Determine whether the complexity of the i-th frame image is greater than the third preset threshold.

[0113] For example, the third preset threshold is a critical value used to determine whether an image is a high-complexity potential defect frame. Images with a complexity higher than the third preset threshold indicate that they contain more details or abnormal features and should be prioritized as key frames.

[0114] If yes, proceed to step S2035; otherwise, proceed to step S2036.

[0115] S2035. Set the i-th frame as a keyframe and set the value of the interval counter to 0.

[0116] After this step is completed, increment the value of i by one, which means processing the new i-th frame image.

[0117] S2036. Keep the value of the interval counter unchanged.

[0118] After this step is completed, increment the value of i by one, which means processing the new i-th frame image.

[0119] This approach employs a two-dimensional strategy: controlling the frequency based on an interval counter and determining the focus of the screening based on image complexity. This ensures uniform coverage of keyframes, accurately captures high-value potential defect frames, avoids keyframe redundancy, and balances screening efficiency with feature coverage.

[0120] In some possible implementations, multiple subsequences are determined based on the image sequence and a preset keyframe interval; for each subsequence, the average complexity of the subsequence is determined based on the complexity of the images included in the subsequence; and the keyframes of the subsequence are determined based on the average complexity of the subsequence.

[0121] For example, the preset keyframe interval refers to a fixed frame threshold used to divide subsequences. A subsequence refers to a continuous image segment split from an image sequence according to the preset keyframe interval. Each subsequence contains a number of frames less than or equal to the preset keyframe interval. When the last subsequence is less than the preset interval, they are merged into a single subsequence based on the actual remaining frames. The average complexity of a subsequence is the arithmetic mean of the complexity of all individual frames within the subsequence. It reflects the overall feature richness and potential defect probability of the subsequence. The higher the average complexity, the greater the likelihood that the subsequence contains defective frames. A keyframe of a subsequence refers to an anchor image that represents the core features of the subsequence.

[0122] Specifically, the electronic device can split the image sequence into multiple sub-sequences based on a preset keyframe interval. Then, for each sub-sequence, the average complexity of the sub-sequence is calculated based on the complexity of the images it includes. If the average complexity of the sub-sequence is greater than a first preset threshold, the images in the sub-sequence are sorted in descending order of complexity, and the first N images are used as the keyframes of the sub-sequence; N is an integer greater than 2. If the average complexity of the sub-sequence is less than or equal to the first preset threshold, adjacent subsequent sub-sequences of the sub-sequence are merged to form a new sub-sequence, and the average complexity of the new sub-sequence is calculated. In other words, when the average complexity of sub-sequence 1 is less than or equal to the first preset threshold, sub-sequence 1 and sub-sequence 2 can be merged to obtain a new sub-sequence. The average complexity of the new sub-sequence is then calculated, and the keyframes of the new sub-sequence are determined based on the relationship between the average complexity of the new sub-sequence and the first preset threshold.

[0123] This method quantifies the overall value based on the average complexity of the subsequences, dynamically adjusts the number of keyframes, and achieves precise setting of selecting more keyframes for high-value subsequences and fewer keyframes for low-value subsequences, thus balancing feature coverage and data redundancy.

[0124] S204. Use the image between every two keyframes as the prediction frame.

[0125] For example, the prediction frame is divided with the key frame as the boundary, forming a continuous interval of key frame-prediction frame-key frame.

[0126] S205. Based on intra-frame prediction technology, key frames in the image sequence are encoded, and based on entropy coding technology, predicted frames in the image sequence are encoded to obtain a video file.

[0127] It should be noted that this step is similar to the aforementioned step S102, and will not be repeated here.

[0128] S206. Decode the video file to obtain the decoded keyframes and the decoded predicted frames.

[0129] It should be noted that this step is similar to the aforementioned step S103, and will not be repeated here.

[0130] S207. Perform defect detection processing on the decoded keyframes to determine the sample category of the decoded keyframes.

[0131] It should be noted that this step is similar to the aforementioned step S10 4, and will not be repeated here.

[0132] S208. For each intermediate frame, determine the similarity between the intermediate frame and the keyframes decoded in the preceding sequence.

[0133] For example, an intermediate frame refers to all decoded predicted frames between two decoded keyframes. A preceding decoded keyframe refers to the decoded keyframe with the earlier timestamp among the two decoded keyframes.

[0134] Similarity is an indicator of the degree of feature matching between the decoded prediction frame and the previously decoded keyframe. It should be noted that the implementation of this application does not limit the calculation method of similarity. For example, the cosine similarity (0-1) of the feature vectors of the decoded prediction frame and the previously decoded keyframe can be directly used as the similarity. Alternatively, the structural similarity index (SSIM) of the two frames can be used as the similarity, or the average of the structural similarity index and the cosine similarity of the feature vectors can be used as the similarity. The specific setting can be based on actual needs.

[0135] In one example, for each intermediate frame, based on the grayscale image of that intermediate frame and the grayscale image of the previously decoded keyframe, the SSIM value between them is calculated. Simultaneously, the image feature vectors of both are extracted, and the cosine similarity of the feature vectors is calculated. Finally, the SSIM value and the cosine similarity are weighted and averaged, or the arithmetic mean is directly calculated, to obtain the final similarity. It should be noted that this embodiment does not limit the weighting coefficient between the SSIM value and the cosine similarity; it can be set based on actual needs.

[0136] S209. If the similarity is determined to be greater than the second preset threshold, the sample category of the decoded prediction frame is determined to be consistent with the sample category of the previously decoded key frame.

[0137] For example, the second preset threshold is a similarity threshold used to determine whether the sample categories of the intermediate frame and the keyframes decoded in the preceding sequence are consistent. It can be divided into a positive sample threshold and a negative sample threshold. This is because positive samples have higher stability in terms of defect-free features, while negative samples may have slight morphological differences in defect features. For example, the second preset threshold for positive samples is 85%, and the second preset threshold for negative samples is 70%. It can be set according to different target objects or different target objects. If the similarity does not reach the corresponding threshold, the intermediate frame is marked as a pending frame, stored separately, and its sample category is determined by manual annotation or re-entry into the defect classification model to avoid misjudgment.

[0138] In one example, the second preset threshold for positive samples is 85%, and the second preset threshold for negative samples is 70%. If the similarity between the intermediate frame and the keyframe after decoding the preceding positive sample is 0.88 (≥85%), then the sample category of the intermediate frame is determined to be a positive sample. If the similarity between the intermediate frame and the keyframe after decoding the preceding negative sample (defect type: battery swelling) is 0.73 (≥70%), then the sample category of the intermediate frame is determined to be a negative sample of the battery swelling type. If the similarity between the intermediate frame and the keyframe after decoding the preceding negative sample is 0.68 (< the second preset threshold), then it is marked as a pending frame, stored separately, and manually labeled. If it is labeled as a negative sample with missing components, it is added to the negative sample set, and the pending frame can be upgraded to a new keyframe.

[0139] S210. Construct a positive sample dataset based on the positive samples in the sample categories of the decoded keyframes and the positive samples in the sample categories of the decoded predicted frames; and construct a negative sample dataset based on the negative samples in the sample categories of the decoded keyframes and the negative samples in the sample categories of the decoded predicted frames.

[0140] The dataset construction method provided in this application involves acquiring a time-series X-ray image sequence of the same type of target object, calculating the complexity of each image to set keyframes and divide prediction frames, generating video files using intra-frame prediction and entropy coding techniques, and performing defect detection on the keyframes after decoding to determine the sample category of the decoded keyframes. Then, the similarity between intermediate frames and previously decoded keyframes is calculated, and the sample category of the decoded prediction frames is inferred based on the similarity. Finally, positive and negative sample datasets are constructed respectively. This method prioritizes keyframe setting based on image complexity, ensuring that high-value potential defect frames are not missed. Simultaneously, it utilizes inter-frame similarity to quickly infer the sample category of prediction frames, significantly reducing the redundant workload of frame-by-frame detection. Combined with a fallback mechanism for pending frames, it ensures sample accuracy, improves the efficiency of constructing positive and negative sample datasets, and guarantees the accuracy of dataset sample category classification. This, in turn, improves model training efficiency and detection accuracy, while simplifying the dataset construction process and reducing enterprise labor costs and time investment.

[0141] Figure 4 A schematic diagram of the structure of the dataset construction device provided in this application is shown below. Figure 4 As shown, the dataset construction apparatus 300 provided in this embodiment includes:

[0142] The acquisition module 301 is used to acquire an image sequence of a target object of the same model, as well as keyframes and prediction frames in the image sequence; wherein the image sequence includes multiple X-ray images arranged in time.

[0143] The encoding module 302 is used to encode keyframes in an image sequence based on intra-frame prediction technology, and to encode predicted frames in an image sequence based on entropy coding technology to obtain a video file.

[0144] The decoding module 303 is used to decode the video file to obtain the decoded keyframes and the decoded predicted frames.

[0145] The detection module 304 is used to perform defect detection processing on the decoded keyframes and determine the sample category of the decoded keyframes.

[0146] The determination module 305 is used to determine the sample category of the decoded prediction frame based on the sample category of the decoded keyframe;

[0147] Module 306 is used to construct a dataset of the target object based on the sample categories of the decoded keyframes and the sample categories of the decoded predicted frames.

[0148] In one possible implementation, the acquisition module 301 is used for:

[0149] Determine the complexity of each image in the image sequence;

[0150] Based on the complexity of each image, multiple keyframes are determined from the image sequence;

[0151] Use the image between every two keyframes as the prediction frame.

[0152] In one possible implementation, the acquisition module 301 is used for:

[0153] For the i-th frame of the image sequence, perform the following steps, where the initial value of i is 1:

[0154] The interval counter value is incremented by one;

[0155] If the value of the interval counter is determined to be greater than or equal to the preset maximum keyframe interval, then the i-th frame image is set as a keyframe, and the value of the interval counter is set to 0;

[0156] If the value of the interval counter is determined to be greater than or equal to the preset minimum keyframe interval and less than the preset maximum keyframe interval, then determine whether the complexity of the i-th frame image is greater than the third preset threshold; if yes, then set the i-th frame image as a keyframe and set the value of the interval counter to 0; if no, then keep the value of the interval counter unchanged.

[0157] If the value of the interval counter is determined to be less than the preset minimum keyframe interval, then the value of the interval counter remains unchanged.

[0158] In one possible implementation, multiple keyframes are determined from the image sequence based on the complexity of each image, including:

[0159] Based on the image sequence and the preset keyframe interval, multiple sub-sequences are determined;

[0160] For each subsequence, determine the average complexity of the subsequence based on the complexity of the images included in the subsequence;

[0161] The keyframes of a subsequence are determined based on its average complexity.

[0162] In one possible implementation, the acquisition module 301 is used for:

[0163] If the average complexity of a subsequence is determined to be greater than a first preset threshold, the images included in the subsequence are sorted in descending order of complexity, and the first N images are used as keyframes of the subsequence; N is an integer greater than 2.

[0164] If the average complexity of a subsequence is determined to be less than or equal to a first preset threshold, then the adjacent subsequent subsequences of that subsequence are merged into a new subsequence, and the average complexity of the new subsequence is calculated.

[0165] In one possible implementation, the acquisition module 301 is used for:

[0166] For each image in the image sequence, convert the image to a grayscale image and determine the grayscale distribution entropy value of the grayscale image;

[0167] Based on the edge detection algorithm, the edge pixels of the grayscale image are extracted, and the edge density of the grayscale image is determined based on the edge pixels.

[0168] The complexity of the image is determined based on the grayscale distribution entropy and edge density.

[0169] In one possible implementation, the determining module 305 is configured to:

[0170] For each intermediate frame, determine the similarity between the intermediate frame and the preceding decoded keyframe; where the intermediate frame is the decoded predicted frame between two decoded keyframes.

[0171] If the similarity is determined to be greater than the second preset threshold, the sample category of the decoded predicted frame is determined to be consistent with the sample category of the previously decoded key frame.

[0172] The dataset construction apparatus provided in this embodiment can execute the methods provided in the above method embodiments. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0173] Figure 5A schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device 400 provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the electronic device 400 further includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus.

[0174] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-described method.

[0175] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0176] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0177] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0178] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0179] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0180] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0181] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0182] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0183] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0184] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0185] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0186] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0187] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0188] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for constructing a dataset, characterized in that, include: Acquire an image sequence of a target object of the same model, and keyframes and prediction frames in the image sequence; wherein the image sequence includes multiple X-ray images arranged in time; Based on intra-frame prediction technology, keyframes in the image sequence are encoded, and based on entropy coding technology, predicted frames in the image sequence are encoded to obtain a video file. The video file is decoded to obtain decoded keyframes and decoded predicted frames; Defect detection processing is performed on the decoded keyframes to determine the sample category of the decoded keyframes; Based on the sample categories of the decoded keyframes, determine the sample categories of the decoded predicted frames; The dataset of the target object is constructed based on the sample categories of the decoded keyframes and the sample categories of the decoded predicted frames.

2. The method according to claim 1, characterized in that, Obtaining keyframes and predicted frames from the image sequence includes: Determine the complexity of each image in the image sequence; Based on the complexity of each image, multiple keyframes are determined from the image sequence; Use the image between every two keyframes as the prediction frame.

3. The method according to claim 2, characterized in that, The determination of multiple keyframes from the image sequence based on the complexity of each image includes: For the i-th frame image in the image sequence, perform the following steps, where the initial value of i is 1: The interval counter value is incremented by one; If it is determined that the value of the interval counter is greater than or equal to the preset maximum keyframe interval, then the i-th frame image is set as a keyframe, and the value of the interval counter is set to 0; If it is determined that the value of the interval counter is greater than or equal to the preset minimum keyframe interval and less than the preset maximum keyframe interval, then it is determined whether the complexity of the i-th frame image is greater than a third preset threshold; if yes, then the i-th frame image is set as a keyframe and the value of the interval counter is set to 0; if no, then the value of the interval counter remains unchanged. If it is determined that the value of the interval counter is less than the preset minimum keyframe interval, then the value of the interval counter remains unchanged.

4. The method according to claim 2, characterized in that, The determination of multiple keyframes from the image sequence based on the complexity of each image includes: Based on the image sequence and the preset keyframe interval, multiple sub-sequences are determined; For each subsequence, the average complexity of the subsequence is determined based on the complexity of the images included in the subsequence; The keyframes of the subsequence are determined based on the average complexity of the subsequence.

5. The method according to claim 4, characterized in that, Determining the keyframes of the subsequence based on its average complexity includes: If the average complexity of the subsequence is determined to be greater than a first preset threshold, the images included in the subsequence are sorted in descending order of complexity, and the first N images are used as keyframes of the subsequence; N is an integer greater than 2. If the average complexity of the subsequence is determined to be less than or equal to a first preset threshold, then the adjacent subsequent subsequences of the subsequence are merged as a new subsequence, and the average complexity of the new subsequence is calculated.

6. The method according to claim 2, characterized in that, Determining the complexity of each image in the image sequence includes: For each image in the image sequence, the image is converted into a grayscale image, and the grayscale distribution entropy value of the grayscale image is determined; Based on the edge detection algorithm, the edge pixels of the grayscale image are extracted, and the edge density of the grayscale image is determined based on the edge pixels. The complexity of the image is determined based on the grayscale distribution entropy value and the edge density.

7. The method according to any one of claims 1-6, characterized in that, Determining the sample category of the decoded predicted frame based on the sample categories of the decoded keyframes includes: For each intermediate frame, determine the similarity between the intermediate frame and the preceding decoded keyframe; wherein, the intermediate frame is the decoded predicted frame between two decoded keyframes; If the similarity is determined to be greater than the second preset threshold, the sample category of the decoded predicted frame is determined to be consistent with the sample category of the previously decoded keyframe.

8. A dataset construction apparatus, characterized in that, include: An acquisition module is used to acquire an image sequence of a target object of the same model, as well as keyframes and prediction frames in the image sequence; wherein the image sequence includes multiple X-ray images arranged in time. The encoding module is used to encode keyframes in the image sequence based on intra-frame prediction technology, and to encode predicted frames in the image sequence based on entropy coding technology to obtain a video file. The decoding module is used to decode the video file to obtain decoded keyframes and decoded predicted frames. The detection module is used to perform defect detection processing on the decoded keyframes and determine the sample category of the decoded keyframes. The determination module is used to determine the sample category of the decoded predicted frame based on the sample category of the decoded keyframe; A construction module is used to construct a dataset of the target object based on the sample categories of the decoded keyframes and the sample categories of the decoded predicted frames.

9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.

Images