An extension method and device of periodic video data

Abstract

The embodiment of the present application provides a kind of extension method and device of periodic video data, method includes: first, the multiple frames of images in periodic video data in a set time interval are obtained, the multiple frames of images form an initial image frame instance based on the time sequence information of each image;Then each frame of image of the initial image frame instance is respectively used as the starting frame of a sliding acquisition window, and multiple groups of image frame instances are obtained by sliding acquisition, the length of the sliding acquisition window is N periods, N is positive integer, and the length of the sliding acquisition window is less than the length of the set time interval.The present application can quickly expand multiple quantities and high quality periodic video data for the training of machine learning model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for expanding periodic video data. Background Art

[0002] AI algorithms have permeated every aspect of our lives, from shopping recommendations and ad push notifications to search engines and multimedia entertainment. As a crucial information carrier in multimedia, video is a prime example. However, the application of AI algorithms to video is still immature, and understanding video content remains a critical challenge that needs to be addressed.

[0003] The ultimate goal of utilizing video data is to enable algorithms to understand videos. Understanding videos is a very abstract concept. At a time when neuroscience is still not fully understood, if we interpret this concept based on human perception, we will ultimately be stuck. Let's be more specific. What exactly are we doing in the task of understanding videos? First, we need to understand the characteristics of videos compared to text, images, and audio. Videos are dynamic, time-ordered image instances, yet image frames are closely connected and contextual. Videos also contain audio information. Therefore, to understand videos, we must model both temporal instances and spatial relationships.

[0004] Compared to images, videos have an additional dimension of temporal information. How to make good use of the temporal information in videos is the key to studying this type of method.

[0005] In the field of computer vision, high-quality data, efficient algorithms, and powerful computing power are considered its three pillars. Data is the most fundamental of these three. Without high-quality data, even the best algorithms and the most powerful computing power are useless. Researchers, such as Google and DeepMind, have invested significant resources in building large-scale datasets, especially video datasets. Summary of the Invention

[0006] To address the problems in the prior art, the present application provides a method and apparatus for expanding periodic video data. The method first acquires multiple frames of image data within a set time interval from the periodic video data. These multiple frames form an initial image frame instance based on the timing information of each image. Next, each frame of the initial image frame instance is used as the starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances. The sliding acquisition window has a length of N periods, where N is a positive integer and is less than the length of the set time interval. The present invention can rapidly expand a large amount of high-quality periodic video data for training machine learning models.

[0007] One aspect of the present invention provides a method for extending periodic video data, comprising:

[0008] Acquire multiple frames of images within a set time interval from periodic video data; the multiple frames of images form an initial image frame instance based on the timing information of each image;

[0009] Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval;

[0010] Wherein, if the length of the set time interval is one cycle, before sliding acquisition to obtain multiple groups of image frame instances, the step of generating the training set is:

[0011] When the starting frame of the sliding acquisition window is the second frame of the initial image frame instance, each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance.

[0012] In a preferred embodiment, the acquiring of a plurality of frames of images within a set time interval in the periodic video data includes: extracting the plurality of frames of images at equal time intervals from all the frames of images within the set time interval.

[0013] In a preferred embodiment, it also includes:

[0014] The annotation information of the image frame instance is consistent with the annotation information of the initial image frame instance.

[0015] Another aspect of the present invention provides a device for extending periodic video data, comprising:

[0016] An acquisition module is configured to acquire a plurality of frames of images within a set time interval from periodic video data; the plurality of frames of images are formed into an initial image frame instance based on the timing information of each image;

[0017] a sliding acquisition module, using each frame of the initial image frame instance as a starting frame of a sliding acquisition window, and performing sliding acquisition to obtain multiple groups of image frame instances, wherein the length of the sliding acquisition window is N cycles, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval;

[0018] Wherein, if the length of the set time interval is one cycle, before sliding acquisition to obtain multiple groups of image frame instances, the expansion device further includes:

[0019] The supplementing module starts when the starting frame of the sliding acquisition window is the second frame of the initial image frame instance, and supplements the tail of the initial image frame instance with each frame of image before the current starting frame to form an updated initial image frame instance.

[0020] In a preferred embodiment, the acquiring of a plurality of frames of images within a set time interval in the periodic video data includes: extracting the plurality of frames of images at equal time intervals from all the frames of images within the set time interval.

[0021] In a preferred embodiment, it also includes:

[0022] The annotation information of the image frame instance is consistent with the annotation information of the initial image frame instance.

[0023] Another aspect of the present invention provides a method for determining the collapse of fresh concrete, comprising:

[0024] Acquiring video data of the fresh concrete;

[0025] intercepting multiple frames of images from the video data, and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is the collapse degree of the fresh concrete; wherein the machine learning model is trained using a training set;

[0026] The steps of generating the training set include:

[0027] Acquire multiple frames of images within a set time interval from the periodic video data of the fresh concrete; the multiple frames of images form an initial image frame instance based on the time sequence information of each image;

[0028] Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval;

[0029] Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance.

[0030] In another aspect of the present invention, a system for determining the collapse of fresh concrete is provided, comprising:

[0031] An acquisition module, for acquiring video data of the fresh concrete;

[0032] an interception module for intercepting multiple frames of images from the video data and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is the collapse degree of the fresh concrete; wherein the machine learning model is trained using a training set;

[0033] The steps of generating the training set include:

[0034] Acquire multiple frames of images within a set time interval from periodic video data of fresh concrete; the multiple frames of images form an initial image frame instance based on time sequence information of each image;

[0035] Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval;

[0036] Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance.

[0037] Another aspect of the present invention provides a method for determining whether a cake is mixed properly, comprising:

[0038] Obtain video data of the cake stirring;

[0039] Intercepting multiple frames of images from the video data and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is pass or fail; wherein the machine learning model is trained using a training set;

[0040] The steps of generating the training set include:

[0041] Acquire multiple frames of images within a set time interval from the periodic video data of cake stirring; the multiple frames of images form an initial image frame instance based on the time sequence information of each image;

[0042] Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval;

[0043] Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance.

[0044] Another aspect of the present invention provides a system for determining whether cake mixing is qualified, comprising:

[0045] An acquisition module, for acquiring video data of the cake mixing;

[0046] an interception module for intercepting multiple frames of images from the video data and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is a pass or fail; wherein the machine learning model is trained using a training set;

[0047] The steps of generating the training set include:

[0048] Acquire multiple frames of images within a set time interval from periodic video data of cake mixing; the multiple frames of images form an initial image frame instance based on time sequence information of each image;

[0049] Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval;

[0050] Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance.

[0051] Another aspect of the present invention provides a method for detecting abnormal rotation of a ship propeller, comprising:

[0052] Acquiring video data of the ship's propeller rotation;

[0053] Intercepting multiple frames of images from the video data and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is normal or abnormal; wherein the machine learning model is trained using a training set;

[0054] The steps of generating the training set include:

[0055] Acquire multiple frames of images within a set time interval from the periodic video data of the ship propeller rotation; the multiple frames of images form an initial image frame instance based on the time sequence information of each image;

[0056] Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval;

[0057] Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance.

[0058] Another aspect of the present invention provides a system for detecting abnormal rotation of a ship propeller, comprising:

[0059] An acquisition module, for acquiring video data of the ship's propeller rotation;

[0060] an interception module, intercepting multiple frames of images from the video data and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is normal or abnormal; wherein the machine learning model is trained using a training set;

[0061] The steps of generating the training set include:

[0062] Acquire multiple frames of images within a set time interval from periodic video data of the rotation of a ship propeller; the multiple frames of images form an initial image frame instance based on the time sequence information of each image;

[0063] Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval;

[0064] Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance.

[0065] In still another aspect of the present application, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the extension method of periodic video data when executing the program.

[0066] In still another aspect of the present application, a computer readable storage medium is provided, which stores a computer program, and the computer program is executable on a processor to implement the extension method of periodic video data.

[0067] According to the above technical solution, the present application provides an extension method of periodic video, which includes: first, acquiring a plurality of image frames in a set time interval in periodic video data, and forming an initial image frame instance based on the time sequence information of each image; then, taking each image of the initial image frame instance as the starting frame of a sliding acquisition window, and slidingly acquiring a plurality of groups of image frame instances, the length of the sliding acquisition window is N periods, N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval. The present application can quickly expand a plurality of quantities and high-quality periodic video data for training of a machine learning model. BRIEF DESCRIPTION OF DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0069] Figure 1 is a flowchart of the extension method of periodic video.

[0070] Figure 2 is a sliding diagram of the sliding acquisition window.

[0071] Figure 3 is a supplementary diagram of the initial image frame instance.

[0072] Figure 4 is a structural diagram of the extension device of periodic video.

[0073] Figure 5 is a flowchart of the determination method of fresh concrete slump.

[0074] Figure 6 is a structural diagram of the determination system of fresh concrete slump.

[0075] Figure 7 is a flowchart of the determination method of cake stirring qualification.

[0076] Figure 8 This is a schematic diagram of the system structure for determining whether cake mixing is qualified.

[0077] Figure 9 It is a flow chart of a method for detecting abnormal rotation of a ship propeller.

[0078] Figure 10 It is a structural diagram of the ship propeller rotation abnormality detection system.

[0079] Figure 11 It is a structural diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0080] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0081] It should be noted that the method and device for expanding periodic videos disclosed in this application can be used in the field of computer technology, and can also be used in any field other than the field of computer technology. The application field of the method and device for expanding periodic videos disclosed in this application is not limited.

[0082] AI algorithms have permeated every aspect of our lives, from shopping recommendations and ad push notifications to search engines and multimedia entertainment. As a crucial information carrier in multimedia, video is a prime example. However, the application of AI algorithms to video is still immature, and understanding video content remains a critical challenge that needs to be addressed.

[0083] The ultimate goal of utilizing video data is to enable algorithms to understand videos. Understanding video is a very abstract concept. At a time when neuroscience is still not fully understood, attempting to understand it based on human perception will ultimately lead us into a quagmire. First, we need to understand the unique characteristics of video compared to text, images, and audio. Video is a dynamic collection of image instances ordered by time, yet image frames are closely connected and contextual. Video also contains audio information. Therefore, video understanding requires modeling both temporal instances and spatial relationships.

[0084] Compared to images, videos have an additional dimension of temporal information. How to make good use of the temporal information in videos is the key to studying this type of method.

[0085] In the field of computer vision, high-quality data, efficient algorithms, and powerful computing power are considered its three pillars. Data is the most fundamental of these three. Without high-quality data, even the best algorithms and the most powerful computing power are useless. Researchers, such as Google and DeepMind, have invested significant resources in building large-scale datasets, especially video datasets.

[0086] To address the problems in the prior art, the present application provides a method and apparatus for expanding periodic video. The method first acquires multiple frames of image data within a set time interval from periodic video data. These multiple frames form an initial image frame instance based on the timing information of each image. Next, each frame of the initial image frame instance is used as the starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances. The sliding acquisition window has a length of N periods, where N is a positive integer and is less than the length of the set time interval. The present invention can rapidly expand large quantities of high-quality periodic video data for training machine learning models.

[0087] The effectiveness of machine learning models is based on the utilization of large amounts of training data. The quantity and quality of samples in the training set determine the effectiveness of a machine learning model. In some real-world problems, the amount of raw data available, or even a limited amount of data of a particular type, is extremely limited. Using such data to train a model is likely to yield unsatisfactory results. Therefore, expanding the training data set is crucial for building machine learning models.

[0088] The following describes in detail the periodic video expansion method and device provided by the present invention with reference to the accompanying drawings.

[0089] The present invention provides a method for extending periodic video, such as Figure 1 As shown, the specific steps include:

[0090] S1: Acquire multiple frames of images within a set time interval in periodic video data; the multiple frames of images form an initial image frame instance based on the timing information of each image.

[0091] Specifically, a periodic video means that the actions in the video are periodic, such as the rotation of the tank of a concrete mixer truck, the rotation of the propeller of a ship, etc. The time to complete a complete action is usually called a cycle. In essence, a video is an image frame instance composed of multiple images arranged in chronological order. The acquisition of multiple frames of images in a set time interval in the periodic video data refers to taking any image frame in the video as the starting point and acquiring multiple frames of images in a set time interval after the corresponding moment of the frame image. There are no special requirements for the selection of the starting point. For the determination of the set time, the set time needs to be greater than or equal to the cycle time of the periodic video. For example, if the cycle of the video is 2 seconds, the set time needs to be greater than or equal to 2 seconds before the subsequent steps can be performed.

[0092] On the other hand, the time intervals between adjacent image frames in the video are different. For example, the time interval between adjacent image frames in video one is 100 milliseconds, and the time interval between adjacent image frames in video two can be 50 milliseconds. Therefore, if the set time is 2 seconds, the number of image frames obtained from video one is 20 frames, and the number of image frames obtained from video two is 40 frames. It can be understood that the larger the number of image frames, the higher the computing power required for the computer. When the computing power of the computer is limited, the original image frames can be extracted according to certain rules. For example, the 40 frames of images obtained in video two can be extracted by extracting 20 frames at intervals of one frame. Therefore, the multi-frame images within the set time interval after the corresponding moment of the frame image can be all the image frames in the video within the set time interval, or can be the multi-frame images obtained after all the image frames in the video within the set time interval are extracted.

[0093] S2: Each frame of the initial image frame instance is used as the starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances. The length of the sliding acquisition window is N cycles, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval.

[0094] Specifically, each frame of the initial image frame instance is used as the starting frame of a sliding acquisition window, and multiple groups of image frame instances are obtained by sliding acquisition. For example, Figure 2As shown, there are 23 frames of images in the initial original image frame instance, and the length of the sliding acquisition window is 7. The first frame of the initial image frame is taken as the starting frame, and the image frames collected in the sliding acquisition window are 1, 2, 3, 4, 5, 6, and 7, thereby obtaining a first group of image frame instances; the sliding window is moved backward so that the first frame of the sliding window moves to the second frame of the initial original image frame instance, and the image frames collected in the sliding acquisition window are 2, 3, 4, 5, 6, 7, and 8, thereby obtaining a second group of image frame instances; and so on, 17 groups of image frame instances can be obtained. The length of the sliding acquisition window is N cycles, where N is a positive integer. It can be understood that the image frame instances collected by the sliding window need to fully represent the actions performed in the periodic video, so that the relationship between the actions and the results can be effectively established when training the model. Therefore, the length of the sliding acquisition window is N cycles, which ensures that there must be a complete action in the window. On the other hand, the length of the window needs to be less than the set time interval length. It can be understood that, assuming the set time interval length is 2 seconds, the time span of the initial image frame instance obtained according to step S1 is 2 seconds. If the length of the sliding acquisition window is set to 3 seconds, the sliding acquisition window cannot slide on the initial image frame instance. Since the image frame instance acquired by the sliding acquisition window is obtained from the initial image instance, the annotation information of the expanded video obtained by the sliding window is consistent with that of the original video. For example, if the annotation information of an original video is qualified, the annotation information of the video expanded by the sliding window is also qualified. If the annotation information of an original video is unqualified, the annotation information of the video expanded by the sliding window is also unqualified.

[0095] Wherein, if the length of the set time interval is one cycle, before sliding acquisition to obtain multiple groups of image frame instances, the step of generating the training set is:

[0096] S0: Starting from the second frame of the initial image frame instance when the starting frame of the sliding acquisition window is the first frame, each frame image before the current starting frame is added to the tail of the initial image frame instance to form an updated initial image frame instance.

[0097] Specifically, if the selected set time interval is a cycle, it can be seen from the combination of steps S1 and S2 that the length of the sliding acquisition window must also be a cycle. When the starting frame of the sliding acquisition window slides to the second frame of the initial image frame instance, there will be a gap in the window, so it is necessary to reasonably supplement the initial image frame instance so that there is no gap in the sliding acquisition window. For example, if Figure 3As shown, the number of image frames in one cycle of a periodic video is 7, that is, the number of initial image frame instances is 7, and the length of the sliding acquisition window is one cycle, which means that the sliding acquisition window contains 7 image frames. When the starting frame of the sliding acquisition window slides to the second frame of the initial image frame instance, the last frame in the window will be missing. Since the video is periodic, before the sliding acquisition, each frame of image before the current starting frame is added to the end of the initial image frame instance to form an updated initial image frame instance. For example, before the starting frame of the sliding acquisition window slides to the second frame, the first frame of image is added to the end of the initial image frame instance to form an initial image instance with a length of 8 for the sliding acquisition window to capture multiple frames of images. Before the starting frame of the sliding acquisition window slides to the third frame, the first frame and the second frame of image are added to the end of the initial image frame instance, and the order remains unchanged, forming an initial image instance with a length of 9 for the sliding acquisition window to capture multiple frames of images. Before the sliding acquisition window's starting frame slides to the fourth frame, the first, second, and third frames are added to the end of the initial image frame instance, forming an initial image instance with a length of 9 for the sliding acquisition window to capture multiple frames. This process is repeated in this way, ensuring that there are no gaps in the sliding acquisition window.

[0098] As can be seen from the above description, the present application provides a method for expanding periodic video. First, multiple frames of images within a set time interval are acquired from periodic video data. The multiple frames of images form an initial image frame instance based on the timing information of each image. Then, each frame of the initial image frame instance is used as the starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances. The length of the sliding acquisition window is N periods, where N is a positive integer and the length of the sliding acquisition window is less than the length of the set time interval. The present invention can quickly expand a large amount of high-quality periodic video data for training machine learning models.

[0099] From the software level, the present application provides an embodiment of a periodic video expansion device for executing all or part of the content of the periodic video expansion method, see Figure 4 The periodic video expansion device specifically includes the following contents:

[0100] An acquisition module 1 acquires a plurality of frames of images within a set time interval from periodic video data; the plurality of frames of images form an initial image frame instance based on the time sequence information of each image;

[0101] a sliding acquisition module 2, which uses each frame of the initial image frame instance as a starting frame of a sliding acquisition window, and obtains a plurality of groups of image frame instances by sliding acquisition, wherein the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval;

[0102] Wherein, if the length of the set time interval is one cycle, before sliding acquisition to obtain multiple groups of image frame instances, the expansion device further includes:

[0103] The supplementing module 0 starts with the second frame of the initial image frame instance being the starting frame of the sliding acquisition window, and supplements the tail of the initial image frame instance with each frame of image before the current starting frame to form an updated initial image frame instance.

[0104] In a preferred embodiment, the acquiring of a plurality of frames of images within a set time interval in the periodic video data includes: extracting the plurality of frames of images at equal time intervals from all the frames of images within the set time interval.

[0105] In a preferred embodiment, it also includes:

[0106] The annotation information of the image frame instance is consistent with the annotation information of the initial image frame instance.

[0107] As can be seen from the above description, the present application provides a device for expanding periodic video, including: an acquisition module, which acquires multiple frames of images in a set time interval in periodic video data, and the multiple frames of images form an initial image frame instance based on the timing information of each image; a sliding acquisition module, which uses each frame of the initial image frame instance as the starting frame of a sliding acquisition window, and obtains multiple groups of image frame instances by sliding acquisition, wherein the length of the sliding acquisition window is N cycles, N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval; wherein, if the length of the set time interval is one cycle, before the sliding acquisition obtains multiple groups of image frame instances, the expansion device also includes: a supplement module 0, which starts with the second frame of the initial image frame instance when the starting frame of the sliding acquisition window is the second frame of the initial image frame instance, and supplements the tail of the initial image frame instance with each frame of the image before the current starting frame to form an updated initial image frame instance. The present invention can quickly expand a large amount of high-quality periodic video data for training machine learning models.

[0108] From the perspective of specific application, since the mixing process of fresh concrete is periodic, the present application provides an embodiment of a method for determining the collapse of fresh concrete, such as Figure 5 As shown, the specific steps include:

[0109] S11: Acquire video data of the fresh concrete;

[0110] Specifically, a device capable of capturing video data, such as a camera, is installed at the opening of the concrete mixer's mixing drum to record the concrete as it mixes. The focus of the captured video frames is the concrete itself, so the device needs to be close enough to the concrete to minimize any unwanted interference from the image frames.

[0111] S12: intercepting multiple frames of images from the video data, and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is the collapse degree of the fresh concrete;

[0112] Specifically, the collapse of fresh concrete refers to the plasticization and pumpability of concrete. Specifically, it is a comprehensive evaluation indicator of water retention, viscosity, and plasticity that can ensure normal construction. The machine learning model can be a BP neural network model, a RPF neural network model, or a recurrent neural network model (RNN) that processes time instance information.

[0113] The machine learning model is trained using a training set; the videos in the training set can be directly captured by a video surveillance device, or can be expanded from the captured videos according to the following steps, wherein the expansion step includes:

[0114] S121: Acquire multiple frames of images within a set time interval from the periodic video data of the fresh concrete; the multiple frames of images form an initial image frame instance based on the time sequence information of each image;

[0115] S122: Each frame image of the initial image frame instance is used as the starting frame of a sliding acquisition window, and multiple groups of image frame instances are obtained by sliding acquisition. The multiple groups of image frame instances form the training set. The length of the sliding acquisition window is N cycles, N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval. If the length of the set time interval is one cycle, the starting frame of the sliding acquisition window is the second frame of the initial image frame instance, and each frame image before the current starting frame is added to the tail of the initial image frame instance to form an updated initial image frame instance.

[0116] As can be seen from the above description, the present invention provides a method for determining the collapse of fresh concrete. First, video data of the fresh concrete is obtained; then, multiple frames of images from the video data are intercepted and input into a machine learning model. The output of the machine learning model is the collapse of the fresh concrete. The machine learning model is trained using a training set. The training set expansion step is as follows: first, multiple frames of images in a set time interval are obtained from the periodic video data of the fresh concrete; the multiple frames of images form an initial image frame instance based on the timing information of each image; then, each frame of the initial image frame instance is used as the starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances. The multiple groups of image frame instances form the training set. The length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval. If the length of the set time interval is one period, the starting frame of the sliding acquisition window is the second frame of the initial image frame instance, and each frame of the image before the current starting frame is added to the end of the initial image frame instance to form an updated initial image frame instance. The present invention can quickly expand a large amount of high-quality periodic video data for training a machine learning model to obtain a model for determining the collapse degree of fresh concrete.

[0117] From the software level, the present application provides an embodiment of a system for determining the collapse of fresh concrete for executing all or part of the content of the method for determining the collapse of fresh concrete, see Figure 6 The system for determining the collapse degree of fresh concrete specifically includes the following contents:

[0118] An acquisition module, for acquiring video data of the fresh concrete;

[0119] an interception module for intercepting multiple frames of images from the video data and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is the collapse degree of the fresh concrete; wherein the machine learning model is trained using a training set;

[0120] The steps of generating the training set include:

[0121] Acquire multiple frames of images within a set time interval from periodic video data of fresh concrete; the multiple frames of images form an initial image frame instance based on time sequence information of each image;

[0122] Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval;

[0123] Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance.

[0124] As can be seen from the above description, the system for determining the degree of collapse of fresh concrete provided by the present invention includes an acquisition module for acquiring video data of the fresh concrete; an interception module for intercepting multiple frames of images in the video data and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is the degree of collapse of the fresh concrete; wherein the machine learning model is trained using a training set; the generation step of the training set includes: acquiring multiple frames of images in a set time interval from the periodic video data of fresh concrete; the multiple frames of images forming an initial image frame based on the time sequence information of each image Example: Each frame of the initial image frame instance is used as the starting frame of a sliding acquisition window, and multiple groups of image frame instances are obtained by sliding acquisition. The multiple groups of image frame instances form the training set. The length of the sliding acquisition window is N cycles, N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval. Wherein, if the length of the set time interval is one cycle, the starting frame of the sliding acquisition window is the second frame of the initial image frame instance, and each frame before the current starting frame is added to the end of the initial image frame instance to form an updated initial image frame instance. The present invention can quickly expand a large amount of high-quality periodic video data for training machine learning models to obtain a model for determining the collapse degree of fresh concrete.

[0125] From the perspective of specific applications, since the cake mixing process is periodic, the present application provides an embodiment of a method for determining whether cake mixing is qualified, such as Figure 7 As shown, the specific steps include:

[0126] S21: Obtain video data of the cake stirring;

[0127] Specifically, a device capable of capturing video data, such as a camera, is installed at the opening of the cake mixer's mixing drum to record video data of the machine stirring the cake ingredients. The captured video data is captured with the focus of the image frames on the cake ingredients, so the device needs to be close to the ingredients to minimize any unwanted interference from the image frames.

[0128] S22: intercepting multiple frames of images from the video data, and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is qualified or unqualified;

[0129] Specifically, the output of the model is either qualified or unqualified. It is understandable that making a cake requires beating the egg whites, that is, stirring them continuously, and then mixing them with other raw materials. If the stirring time is insufficient or the stirring speed or force is incorrect, qualified ingredients for making the cake cannot be obtained. Therefore, if it is qualified, the cake can be made, and if it is unqualified, the cake cannot be made. The machine learning model can be a BP neural network model, an RPF neural network model, or a recurrent neural network model (RNN) that processes time instance information.

[0130] The machine learning model is trained using a training set; the videos in the training set can be directly captured by a video surveillance device, or can be expanded from the captured videos according to the following steps, wherein the expansion step includes:

[0131] S221: Acquire multiple frames of images within a set time interval from the periodic video data of cake stirring; the multiple frames of images form an initial image frame instance based on the time sequence information of each image;

[0132] S222: Each frame image of the initial image frame instance is used as the starting frame of a sliding acquisition window, and multiple groups of image frame instances are obtained by sliding acquisition. The multiple groups of image frame instances form the training set. The length of the sliding acquisition window is N cycles, N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval; wherein, if the length of the set time interval is one cycle, the starting frame of the sliding acquisition window is the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance.

[0133] As can be seen from the above description, the present invention provides a method for determining whether cake mixing is qualified. First, video data of the cake mixing is obtained; then, multiple frames of images in the video data are intercepted and input into a machine learning model. The output of the machine learning model is qualified or unqualified. The machine learning model is trained using a training set. The training set expansion step is as follows: first, multiple frames of images in a set time interval are obtained from the periodic video data of the cake mixing; the multiple frames of images form an initial image frame instance based on the timing information of each image; then, each frame of the initial image frame instance is used as the starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances. The multiple groups of image frame instances form the training set. The length of the sliding acquisition window is N cycles, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval. Wherein, if the length of the set time interval is one cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame of the image before the current starting frame is added to the end of the initial image frame instance to form an updated initial image frame instance. The present invention can quickly expand a large amount of high-quality periodic video data for training machine learning models to obtain a determination model for qualified cake mixing.

[0134] From the software level, the present application provides an embodiment of a cake mixing qualified determination system for executing all or part of the content of the cake mixing qualified determination method, see Figure 8 The cake mixing qualified determination system specifically includes the following contents:

[0135] An acquisition module, for acquiring video data of the cake mixing;

[0136] an interception module, which intercepts multiple frames of images from the video data and inputs the multiple frames of images into a machine learning model, wherein the output of the machine learning model is either qualified or unqualified; wherein the machine learning model is trained using a training set;

[0137] The steps of generating the training set include:

[0138] Acquire multiple frames of images within a set time interval from periodic video data of cake mixing; the multiple frames of images form an initial image frame instance based on time sequence information of each image;

[0139] Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval;

[0140] Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance.

[0141] As can be seen from the above description, the present invention provides a system for determining whether cake mixing is qualified. The system includes an acquisition module for acquiring video data of the cake mixing; an interception module for intercepting multiple frames of images in the video data and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is qualified or unqualified; wherein the machine learning model is trained using a training set; the step of generating the training set includes: acquiring multiple frames of images within a set time interval from the periodic video data of the cake mixing; the multiple frames of images forming an initial image frame instance based on the timing information of each image; using each frame of the initial image frame instance as the starting frame of a sliding acquisition window, sliding acquisition to obtain multiple groups of image frame instances, and the multiple groups of image frame instances forming the training set, the length of the sliding acquisition window is N cycles, N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval; wherein, if the length of the set time interval is one cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame of the image before the current starting frame is added to the end of the initial image frame instance to form an updated initial image frame instance. The present invention can quickly expand a large amount of high-quality periodic video data for training a machine learning model, thereby obtaining a definite model for qualified cake mixing.

[0142] From the perspective of specific applications, since the rotation process of a ship propeller is periodic, the present application provides an embodiment of a method for detecting abnormal rotation of a ship propeller, such as Figure 9 As shown, the specific steps include:

[0143] S31: Acquire video data of the ship's propeller rotation;

[0144] Specifically, a device capable of capturing video data, such as a camera, is installed directly behind the propeller to record the propeller's rotation. The image frames captured in the captured video data are focused on the propeller, so the device needs to be close to the propeller to minimize any unwanted interference from the image frame.

[0145] S32: intercepting multiple frames of images from the video data, and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is normal or abnormal;

[0146] Specifically, the model output is normal or abnormal. It is understood that propeller rotation performance, such as speed and propeller shape, directly affects propeller operation. During propeller rotation, abnormalities such as uneven speed and propeller deformation may occur. These abnormalities need to be detected promptly to avoid unnecessary accidents. The machine learning model can be a BP neural network model, a RPF neural network model, or a recurrent neural network model (RNN) that processes time instance information.

[0147] The machine learning model is trained using a training set; the videos in the training set can be directly captured by a video surveillance device, or can be expanded from the captured videos according to the following steps, wherein the expansion step includes:

[0148] S321: Acquire multiple frames of images within a set time interval from the periodic video data of the ship propeller rotation; the multiple frames of images form an initial image frame instance based on the time sequence information of each image;

[0149] S322: Each frame image of the initial image frame instance is used as the starting frame of a sliding acquisition window, and multiple groups of image frame instances are obtained by sliding acquisition. The multiple groups of image frame instances form the training set. The length of the sliding acquisition window is N cycles, N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval; wherein, if the length of the set time interval is one cycle, the starting frame of the sliding acquisition window is the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance.

[0150] From the above description, the application provides a ship propeller rotation anomaly detection method, which first acquires video data of the ship propeller rotation; then, multiple frames of images in the video data are intercepted, and the multiple frames of images are input into a machine learning model, the output of the machine learning model is normal or abnormal, the machine learning model is obtained by training a training set, and the expansion step of the training set is: first, multiple frames of images in a set time interval in periodic video data of the ship propeller rotation are acquired; the multiple frames of images form an initial image frame instance based on the time sequence information of each image; then, each frame of image of the initial image frame instance is taken as a starting frame of a sliding collection window, and multiple groups of image frame instances are obtained by sliding collection, the multiple groups of image frame instances form the training set, the length of the sliding collection window is N periods, N is a positive integer, and the length of the sliding collection window is less than the length of the set time interval; wherein, if the length of the set time interval is one period, the starting frame of the sliding collection window is the second frame of the initial image frame instance, and each frame of image before the current starting frame is supplemented at the tail of the initial image frame instance to form an updated initial image frame instance. The application can quickly expand a large number of periodic video data with high quality for training of a machine learning model to obtain a ship propeller rotation anomaly detection model.

[0151] From the software aspect, the application provides an embodiment of a ship propeller rotation anomaly detection system for executing all or part of the contents of the ship propeller rotation anomaly detection method, referring to Figure 10 , the ship propeller rotation anomaly detection system specifically includes the following contents:

[0152] An acquisition module acquires video data of the ship propeller rotation;

[0153] An interception module intercepts multiple frames of images in the video data, and inputs the multiple frames of images into a machine learning model, the output of the machine learning model is qualified or unqualified; wherein, the machine learning model is obtained by training a training set;

[0154] The generation step of the training set includes:

[0155] Multiple frames of images in a set time interval in periodic video data of the ship propeller rotation are acquired; the multiple frames of images form an initial image frame instance based on the time sequence information of each image;

[0156] Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval;

[0157] Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance.

[0158] As can be seen from the above description, the ship propeller rotation abnormality detection system provided by the present invention includes an acquisition module for acquiring video data of the ship propeller rotation; an interception module for intercepting multiple frames of images in the video data and inputting the multiple frames of images into a machine learning model, and the output of the machine learning model is normal or abnormal; wherein, the machine learning model is trained using a training set; the generation step of the training set includes: acquiring multiple frames of images in a set time interval in the periodic video data of the ship propeller rotation; the multiple frames of images form an initial image frame based on the time sequence information of each image Example: Each frame of the initial image frame instance is used as the starting frame of a sliding acquisition window, and multiple groups of image frame instances are obtained by sliding acquisition. The multiple groups of image frame instances form the training set. The length of the sliding acquisition window is N cycles, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval. Wherein, if the length of the set time interval is one cycle, the starting frame of the sliding acquisition window is the second frame of the initial image frame instance, and each frame before the current starting frame is added to the end of the initial image frame instance to form an updated initial image frame instance. The present invention can quickly expand a large amount of high-quality periodic video data for training machine learning models, thereby obtaining a model for detecting abnormal rotation of ship propellers.

[0159] From a hardware perspective, the present application provides an embodiment of an electronic device for implementing all or part of the method for extending periodic video. The electronic device specifically includes the following:

[0160] Figure 11 Schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Figure 11 As shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It is worth noting that the Figure 11are exemplary; other types of structures also can be used to supplement or replace the structure to implement telecommunication functions or other functions.

[0161] In an embodiment, the extension function of the periodic video can be integrated into the central processor. The central processor can be configured to control as follows:

[0162] S1: Obtain multiple frames of images in a set time interval in the periodic video data; the multiple frames of images form an initial image frame instance based on the time sequence information of each frame of image;

[0163] S2: Respectively take each frame of image of the initial image frame instance as the starting frame of a sliding acquisition window, and slidingly acquire multiple groups of image frame instances, the length of the sliding acquisition window is N periods, N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval;

[0164] Wherein, if the length of the set time interval is one period, before the multiple groups of image frame instances are acquired by sliding, the generation step of the training set:

[0165] S0: The starting frame of the sliding acquisition window is the second frame of the initial image frame instance, and each frame of image before the current starting frame is supplemented at the tail of the initial image frame instance to form an updated initial image frame instance.

[0166] From the above description, it can be seen that the electronic device provided by the embodiments of the present application can quickly expand a large number of high-quality periodic video data for training of a machine learning model.

[0167] In another embodiment, the extension device of the periodic video can be configured separately from the central processor 9100, for example, the extension device of the periodic video can be configured as a chip connected with the central processor 9100, and the extension function of the periodic video is realized through the control of the central processor.

[0168] As shown in Figure 11 , the electronic device 9600 can further include a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It should be noted that the electronic device 9600 does not necessarily include all the components shown in Figure 11 ; in addition, the electronic device 9600 can also include components not shown in Figure 11 , which can refer to the prior art.

[0169] As shown in Figure 11As shown, the central processing unit 9100 is sometimes also referred to as a controller or operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0170] Memory 9140 can be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It can store the aforementioned failure-related information and also store programs that execute the relevant information. The CPU 9100 can execute the programs stored in memory 9140 to implement information storage or processing.

[0171] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 may be, for example, a keypad or touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display objects such as images and text. The display may be, for example, an LCD display, but is not limited thereto.

[0172] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), or a SIM card. Alternatively, it may be a memory that retains information even when power is off, can be selectively erased, and is provided with more data. Examples of such memory are sometimes referred to as EPROMs. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs or processes for executing the operation of the electronic device 9600 by the central processing unit 9100.

[0173] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various driver programs for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0174] The communication module 9110 is a transmitter / receiver 9110 that transmits and receives signals via an antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as in a conventional mobile communication terminal.

[0175] Based on different communication technologies, multiple communication modules 9110 can be provided in the same electronic device, such as a cellular network module, a Bluetooth module, and / or a wireless local area network module. The communication module (transmitter / receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby implementing common telecommunication functions. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. Furthermore, the audio processor 9130 is also coupled to the central processing unit 9100, enabling local recording via the microphone 9132 and playback of stored audio via the speaker 9131.

[0176] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all steps of the method for extending periodic videos in the above embodiments. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the computer program implements all steps of the method for extending periodic videos in the above embodiments, where the execution subject is a server or a client. For example, when the processor executes the computer program, the following steps are implemented:

[0177] S1: Acquire multiple frames of images within a set time interval from periodic video data; the multiple frames of images form an initial image frame instance based on the timing information of each image;

[0178] S2: Using each frame of the initial image frame instance as a starting frame of a sliding acquisition window, sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval;

[0179] Wherein, if the length of the set time interval is one cycle, before sliding acquisition to obtain multiple groups of image frame instances, the step of generating the training set is:

[0180] S0: Starting from the second frame of the initial image frame instance when the starting frame of the sliding acquisition window is the first frame, each frame image before the current starting frame is added to the tail of the initial image frame instance to form an updated initial image frame instance.

[0181] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application can quickly expand a large amount of high-quality periodic video data for training machine learning models.

[0182] Those skilled in the art will appreciate that embodiments of the present application can be readily used as software, hardware, or a combination of software and hardware. In a software embodiment, the methods can be tangibly embodied in a machine-readable storage medium having stored thereon instructions that can be used to program a computer to perform any of the operations described herein. The software implementation can be for example, in the form of a computer program product which can include software agents or objects embedded in a computer readable storage medium. The computer readable storage medium can be a floppy disk, flexible disk, hard disk, USB (universal serial bus), RAM (random-access memory), flash memory, magnetic tape, or any other form of a computer readable storage medium.

[0183] The present application is described in relation to flow charts and / or block diagrams of methods, apparatus (devices) and computer program products according to embodiments of the application. It is understood that each flow and / or block in the flow charts and / or block diagrams, and combinations of flows and / or blocks in the flow charts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow charts and / or block diagrams block or blocks. Figure 1 one or more flows and / or blocks Figure 1 an apparatus to perform the functions specified in the flow chart

[0184] These computer program instructions can also be stored in a computer readable storage medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable storage medium produce an article of manufacture including instructions which implement the flow chart Figure 1 one or more flows and / or blocks Figure 1 an apparatus to perform the functions specified in the flow chart

[0185] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow chart Figure 1 one or more flows and / or blocks Figure 1 an apparatus to perform the functions specified in the flow chart

[0186] The principles and implementation of the present application are described in the detailed description of specific embodiments. The above description of the embodiments is only for the purpose of understanding the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific embodiments and the scope of application will be changed; in view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for expanding periodic video data, characterized in that: include: Acquire multiple frames of images within a set time interval in periodic video data, including all image frames in the video within the set time interval, or multiple frames of images obtained by extracting all image frames in the video within the set time; the set time interval is greater than or equal to the period time of the periodic video; The multiple frames of images form an initial image frame instance based on the time sequence information of each image; Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval; Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance: before the starting frame of the sliding acquisition window slides to the third frame, the first frame and the second frame image are supplemented at the end of the initial image frame instance, and the order remains unchanged.

2. The method for expanding video data according to claim 1, wherein: The acquiring of a plurality of frames of images within a set time interval from the periodic video data includes: extracting the plurality of frames of images at equal time intervals from all the frame images within the set time interval.

3. The method for expanding video data according to claim 1, wherein: Also includes: The annotation information of the image frame instance is consistent with the annotation information of the initial image frame instance.

4. A device for expanding periodic video data, characterized in that: include: An acquisition module is configured to acquire multiple frames of images within a set time interval from periodic video data, including all image frames in the video within the set time interval, or multiple frames of images obtained by extracting all image frames in the video within the set time interval; the set time interval is greater than or equal to the period time of the periodic video; The multiple frames of images form an initial image frame instance based on the time sequence information of each image; a sliding acquisition module, using each frame of the initial image frame instance as a starting frame of a sliding acquisition window, and performing sliding acquisition to obtain multiple groups of image frame instances, wherein the length of the sliding acquisition window is N cycles, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval; Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance: before the starting frame of the sliding acquisition window slides to the third frame, the first frame and the second frame image are supplemented at the end of the initial image frame instance, and the order remains unchanged.

5. The video data expansion device according to claim 4, characterized in that: The acquiring of a plurality of frames of images within a set time interval from the periodic video data includes: extracting the plurality of frames of images at equal time intervals from all the frame images within the set time interval.

6. The video data expansion device according to claim 4, characterized in that: Also includes: The annotation information of the image frame instance is consistent with the annotation information of the initial image frame instance.

7. A method for determining the collapse of fresh concrete, comprising: Acquiring video data of the fresh concrete; intercepting multiple frames of images from the video data, and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is the collapse degree of the fresh concrete; wherein the machine learning model is trained using a training set; The steps of generating the training set include: Acquire multiple frames of images within a set time interval from the periodic video data of the fresh concrete, including all image frames in the video within the set time interval, or multiple frames of images obtained by extracting all image frames in the video within the set time interval; the set time interval is greater than or equal to the period time of the periodic video; the multiple frames of images form an initial image frame instance based on the timing information of each image; Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval; Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance: before the starting frame of the sliding acquisition window slides to the third frame, the first frame and the second frame image are supplemented at the end of the initial image frame instance, and the order remains unchanged.

8. A system for determining the collapse of fresh concrete, comprising: An acquisition module, for acquiring video data of the fresh concrete; an interception module for intercepting multiple frames of images from the video data and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is the collapse degree of the fresh concrete; wherein the machine learning model is trained using a training set; The steps of generating the training set include: Acquire multiple frames of images within a set time interval from periodic video data of fresh concrete, including all image frames in the video within the set time interval, or multiple frames of images obtained by extracting all image frames in the video within the set time interval; the set time interval is greater than or equal to the period time of the periodic video; and the multiple frames of images form an initial image frame instance based on the timing information of each image; Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval; Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance: before the starting frame of the sliding acquisition window slides to the third frame, the first frame and the second frame image are supplemented at the end of the initial image frame instance, and the order remains unchanged.

9. A method for determining whether cake mixing is qualified, comprising: Obtain video data of the cake stirring; Intercepting multiple frames of images from the video data and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is pass or fail; wherein the machine learning model is trained using a training set; The steps of generating the training set include: Acquire multiple frames of images within a set time interval from the periodic video data of the cake mixing, including all image frames in the video within the set time interval, or multiple frames of images obtained by extracting all image frames in the video within the set time interval; the set time interval is greater than or equal to the period time of the periodic video; the multiple frames of images form an initial image frame instance based on the timing information of each image; Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval; Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance: before the starting frame of the sliding acquisition window slides to the third frame, the first frame and the second frame image are supplemented at the end of the initial image frame instance, and the order remains unchanged.

10. A system for determining whether cake mixing is qualified, comprising: An acquisition module, for acquiring video data of the cake mixing; an interception module for intercepting multiple frames of images from the video data and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is a pass or fail; wherein the machine learning model is trained using a training set; The steps of generating the training set include: Acquire multiple frames of images within a set time interval from periodic video data of cake mixing, including all image frames in the video within the set time interval, or multiple frames of images obtained by extracting all image frames in the video within the set time interval; the set time interval is greater than or equal to the period of the periodic video; the multiple frames of images form an initial image frame instance based on the timing information of each image; Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval; Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance: before the starting frame of the sliding acquisition window slides to the third frame, the first frame and the second frame image are supplemented at the end of the initial image frame instance, and the order remains unchanged.

11. A method for detecting abnormal rotation of a ship propeller, comprising: Acquiring video data of the ship's propeller rotation; Intercepting multiple frames of images from the video data and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is normal or abnormal; wherein the machine learning model is trained using a training set; The steps of generating the training set include: Acquire multiple frames of images within a set time interval from the periodic video data of the ship's propeller rotation, including all image frames in the video within the set time interval, or multiple frames of images obtained by extracting all image frames in the video within the set time interval; the set time interval is greater than or equal to the period of the periodic video; and the multiple frames of images form an initial image frame instance based on the timing information of each image; Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval; Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance: before the starting frame of the sliding acquisition window slides to the third frame, the first frame and the second frame image are supplemented at the end of the initial image frame instance, and the order remains unchanged.

12. A ship propeller rotation abnormality detection system, comprising: An acquisition module, for acquiring video data of the ship's propeller rotation; an interception module, intercepting multiple frames of images from the video data and inputting the multiple frames of images into a machine learning model, wherein the output of the machine learning model is normal or abnormal; wherein the machine learning model is trained using a training set; The steps of generating the training set include: Acquire multiple frames of images within a set time interval from periodic video data of a rotating ship propeller, including all image frames in the video within the set time interval, or multiple frames of images obtained by extracting all image frames in the video within the set time interval; the set time interval is greater than or equal to the period of the periodic video; and the multiple frames of images form an initial image frame instance based on the timing information of each image; Each frame of the initial image frame instance is used as a starting frame of a sliding acquisition window, and sliding acquisition is performed to obtain multiple groups of image frame instances, wherein the multiple groups of image frame instances form the training set, and the length of the sliding acquisition window is N periods, where N is a positive integer, and the length of the sliding acquisition window is less than the length of the set time interval; Among them, if the length of the set time interval is a cycle, the starting frame of the sliding acquisition window starts from the second frame of the initial image frame instance, and each frame image before the current starting frame is supplemented at the end of the initial image frame instance to form an updated initial image frame instance: before the starting frame of the sliding acquisition window slides to the third frame, the first frame and the second frame image are supplemented at the end of the initial image frame instance, and the order remains unchanged.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 3 and 7, 9, and 11 is implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 3 and 7, 9, and 11 is implemented.

Images