A multi-camera image frame extraction method and system

Abstract

The application discloses a multi-path camera image frame extraction method and system, relates to the technical field of data processing, and extracts frames through a two-stage frame extraction strategy combining low-frequency monitoring and high-frequency tracking. When no event occurs, monitoring is performed at a very low first preset data sampling frequency, and subsequent processing is triggered only when a scene change is detected, so that data storage under a static scene is greatly reduced. When a key target is recognized, high-frequency sampling is immediately switched to, so that the integrity of key information is ensured. Compared with fixed-frequency frame extraction, the on-demand resource allocation manner can more effectively save storage and bandwidth.

Description

A method and system for frame extraction from multi-channel camera images Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a method and system for extracting frames from multi-channel camera images. Background Technology

[0002] In many practical applications, such as event detection, behavior analysis, and target tracking, it is not necessary to process every single frame of an image because there is significant temporal and spatial redundancy between consecutive video frames. Processing all video frames would place enormous pressure on computation, storage, and bandwidth, resulting in wasted resources and potential delays in analysis and processing. To address this issue, image frame extraction technology has emerged. Its core idea is to select key frames or representative frames from a continuous video stream according to certain rules or strategies to reduce the amount of data and storage required for subsequent processing. Existing technologies generally employ a fixed-interval frame extraction method, extracting one frame every fixed N frames. For example, in a video of 25 frames per second, extracting one frame every 5 frames results in an image sequence of 5 frames per second. The advantage of this method is its simplicity and low computational cost, but its disadvantages are also significant. If a critical event (such as a traffic accident or abnormal behavior) occurs within the frame extraction interval, the key frame of that event may be missed, leading to information loss. Conversely, in long-term static scenes, a large number of redundant frames with similar content will be extracted, resulting in waste. Summary of the Invention

[0003] This application aims to provide a method and system for frame extraction from multi-channel camera images, which addresses the problems of existing technologies using fixed-interval frame extraction methods, where excessively long frame extraction intervals may lead to the omission of key information or excessively short frame extraction times may result in the waste of storage and computing resources, making it difficult to achieve a balance between the two.

[0004] The first aspect of this application provides a method for frame extraction from multi-channel camera images, including:

[0005] Image data from multiple cameras is collected at a first preset data sampling frequency, and a frame-skipping decision action is obtained based on the image data from each camera; the frame-skipping decision action includes either storing the skipped frames or not storing them.

[0006] If the frame-sampling decision action is frame-sampling storage, then a pre-set target recognition model is used to identify the image data and determine the target captured by each camera.

[0007] The first preset data sampling frequency is adjusted to the second preset data sampling frequency corresponding to the target, and the target image frame is obtained by performing linked frame extraction processing on multiple cameras through the second preset data sampling frequency.

[0008] The attributes of the target image frame include its corresponding camera and the frame extraction time; the second preset data sampling frequency is higher than the first preset data sampling frequency.

[0009] All target image frames are associated and stored to form an event set, and the event set is arranged in chronological order to complete the image frame extraction from multiple cameras.

[0010] In one possible implementation, after obtaining the frame-skipping decision action based on the image data corresponding to each camera, the method further includes:

[0011] If the frame-skipping decision action is to not skim and store frames, then the process of determining the next frame-skipping decision action will begin.

[0012] In one possible implementation, image data corresponding to multiple cameras is acquired at a first preset data sampling frequency, and a frame-skipping decision action is obtained based on the image data corresponding to each camera, including:

[0013] For any camera, image data captured by the camera is collected at a first preset data sampling frequency, and it is determined whether the scene has changed between the current frame and the previous frame. If so, the frame extraction decision action is determined to be frame extraction and storage; otherwise, the frame extraction and storage action is determined to be no frame extraction and storage.

[0014] In one possible implementation, the pre-setting method for the target recognition model is as follows:

[0015] Construct a YOLO model and initialize the corresponding model parameters. Encode the initialized model parameters into a vector to obtain the parameter vector.

[0016] Repeatedly obtain multiple different parameter vectors;

[0017] Obtain the loss function value corresponding to each parameter vector, and determine the parameter vector with the smallest loss function value as the optimal vector;

[0018] Based on the optimal vector, a boundary-constrained search strategy is used to perform a local region search on the parameter vector to obtain the parameter vector after the local region search.

[0019] For any parameter vector after searching a local region, a gradient information search strategy is used to perform a neighborhood search on the parameter vector after searching the local region, so as to obtain the parameter vector after the neighborhood search.

[0020] For any parameter vector after neighborhood search, an adaptive memory learning strategy is used to perform a balance search on the parameter vector after neighborhood search, so as to obtain the parameter vector after balance search.

[0021] For any parameter vector after a balance search, a random mutation search strategy is used to perform a global search on the parameter vector after the balance search, so as to obtain the parameter vector after the global search.

[0022] Obtain the current training stage; wherein, the current training stage is either completed training or incomplete training;

[0023] If the current training phase is not yet complete, then based on the parameter vector after global search, the optimal vector is re-obtained, and the hyperparameters of the re-obtained optimal vector are used as the final hyperparameters of the YOLO model to obtain the target recognition model.

[0024] If the current training stage is incomplete, then the step of determining the parameter vector with the minimum loss function value is returned as the optimal vector.

[0025] In one possible implementation, based on the optimal vector, a boundary-constrained search strategy is used to perform a local region search on the parameter vector to obtain a parameter vector after the local region search, including:

[0026] Based on the optimal vector, the upper and lower bound vectors for the search of the parameter vector are determined as follows:

[0027]

[0028]

[0029] in, Represents the optimal vector. This represents the lower bound vector corresponding to the parameter vector. This represents the upper bound vector corresponding to the parameter vector. This represents the lower boundary vector of the search for the parameter vector. This represents the upper bound vector of the search for the parameter vector. This represents the search boundary control parameters, and =1-k / K, where k represents the current training iterations and K represents the maximum training iterations. Indicates will and The model parameters for each dimension are compared, and the larger model parameters for each dimension are combined into a vector to obtain... ; Indicates will and The model parameters for each dimension are compared, and the smaller model parameters for each dimension are combined into a vector to obtain... ;

[0030] Based on the upper and lower bound vectors of the search, a local region search is performed on the parameter vector to obtain the parameter vector after the local region search:

[0031]

[0032] in, Let m represent the m-th parameter vector in the k-th training process, where m = 1, 2, ..., M, and M represents the total number of parameter vectors. This represents the parameter vector after the search of the m-th local region. Represents the first random number between (0,1). This represents the second random number between (0,1).

[0033] In one possible implementation, for any parameter vector obtained after searching a local region, a gradient information search strategy is used to perform a neighborhood search on the parameter vector obtained after the local region search, resulting in a parameter vector obtained after the neighborhood search, including:

[0034] Arrange the parameter vectors after the local region search in descending order of the loss function values ​​to obtain the sorted parameter vectors;

[0035] A gradient information search strategy is used to perform a neighborhood search on the permuted parameter vector, resulting in the following parameter vector after the neighborhood search:

[0036]

[0037]

[0038]

[0039] in, Let represent the parameter vector after the nth permutation in the kth training process. This represents the parameter vector after the nth neighborhood search. This represents the gradient information search control factor. This represents a parameter vector where all elements are 1. Represents a symbolic function. express gradient information, express The corresponding solution quality factor, express The corresponding solution quality factor is: Solution quality factor = 1 / (loss function value + 0.0001). This represents a random parameter vector where each element is randomly set to either 0.1 or -0.1. Represents a random number between (-1, 1). This represents the basic search volume for gradient information. This represents the parameter influencing the gradient information search control factor. Represents the sine function. Represents pi (π). Represents a logarithmic function.

[0040] In one possible implementation, for any parameter vector obtained after a neighborhood search, an adaptive memory learning strategy is used to perform a balance search on the parameter vector obtained after the neighborhood search, resulting in the following parameter vector after the balance search:

[0041]

[0042]

[0043] in, Let represent the parameter vector after the j-th neighborhood search during the k-th training iteration. Let represent the parameter vector after the j-th balance search. A random number represented as 1 or -1. This represents the historical position memory coefficient between (0, 0.2). Indicates inertia weight, This represents the parameter vector after the j-th neighborhood search during the (k-1)-th training process. Represents the worst-case vector. This represents the maximum value of the inertia weight. This represents the minimum value of the inertia weight.

[0044] In one possible implementation, for any parameter vector obtained after a balance search, a random mutation search strategy is used to perform a global search on the parameter vector obtained after the balance search, resulting in the following parameter vector after the global search:

[0045]

[0046]

[0047] in, Let represent the parameter vector after the i-th balance search during the k-th training process. This represents the parameter vector after the i-th global search. Represents the cosine function. Represents a random variable factor. Represents the control coefficient for random variation. The 'h' indicates the mutation level, and 'h' indicates the current level. Represented as Other parameter vectors that are randomly matched, express and The Euclidean distance between them.

[0048] In one possible implementation, the first preset data sampling frequency is adjusted to a second preset data sampling frequency corresponding to the target, and the target image frame is obtained by performing linked frame extraction processing on multiple cameras using the second preset data sampling frequency, including:

[0049] Adjust the first preset data sampling frequency to the second preset data sampling frequency corresponding to the target;

[0050] The camera that stores the captured image data is designated as the first camera to extract frames. It is determined whether the target intersects with the overlapping area of ​​the captured images of the first camera to extract frames and other cameras. If so, the other cameras are designated as the second cameras to extract frames; otherwise, no processing is performed on the other cameras.

[0051] Based on the second preset data sampling frequency, the first and second cameras to be sampled are subjected to linked frame extraction processing to obtain the target image frame.

[0052] A second aspect of this application provides a multi-channel camera image frame extraction system, including: a frame extraction decision module, a target recognition module, a linked frame extraction module, and an event set storage module;

[0053] The frame-skipping decision module is used to collect image data corresponding to multiple cameras at a first preset data sampling frequency, and to obtain a frame-skipping decision action based on the image data corresponding to each camera; the frame-skipping decision action includes storing frames or not storing frames.

[0054] The target recognition module is used to identify the image data using a pre-set target recognition model when the frame-slicing decision action is frame-slicing storage, thereby determining the target captured by each camera.

[0055] The linked frame extraction module is used to adjust the first preset data sampling frequency to the second preset data sampling frequency corresponding to the target, and to perform linked frame extraction processing on multiple cameras through the second preset data sampling frequency to obtain the target image frame.

[0056] The attributes of the target image frame include its corresponding camera and the frame extraction time; the second preset data sampling frequency is higher than the first preset data sampling frequency.

[0057] The event set storage module is used to associate and store all target image frames to form an event set, and to arrange the event set in chronological order to complete the image frame extraction from multiple cameras.

[0058] Beneficial effects:

[0059] This application provides a multi-channel camera image frame extraction method and system. The method uses a two-level frame extraction strategy that combines low-frequency monitoring and high-frequency tracking. When no event occurs, monitoring is performed at an extremely low first preset data sampling frequency. Subsequent processing is triggered only when a scene change is detected, which greatly reduces the data storage in static scenes. When a key target is identified, the method immediately switches to high-frequency sampling to ensure the integrity of key information. This on-demand resource allocation method can save storage and bandwidth more effectively than fixed-frequency frame extraction. Attached Figure Description

[0060] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 is a flowchart of a multi-channel camera image frame extraction method according to an embodiment of this application;

[0062] Figure 2 is a structural diagram of a multi-channel camera image frame extraction system according to an embodiment of this application;

[0063] Explanation of reference numerals in the attached diagram: 201-Frame extraction decision module, 202-Target recognition module, 203-Linked frame extraction module, 204-Event set storage module. Detailed Implementation

[0064] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0065] Traditional image frame extraction methods typically employ fixed-frequency frame extraction, such as extracting one frame every N frames for storage. While simple to implement, this method has significant drawbacks: it cannot distinguish between static scenes and dynamic events. In static scenes with no changes over a long period, fixed-frequency frame extraction still stores a large number of images with repetitive content, wasting storage resources. In dynamic scenes with important events (such as intrusions or traffic violations), fixed-frequency frame extraction may lose crucial details due to insufficient sampling rate, affecting subsequent event analysis and tracing. Therefore, this application provides a multi-channel camera image frame extraction method that uses a two-stage frame extraction strategy combining low-frequency monitoring and high-frequency tracking. When no events occur, monitoring is performed at an extremely low first preset data sampling frequency, triggering subsequent processing only when scene changes are detected, greatly reducing data storage in static scenes. Once a key target is identified, high-frequency sampling is immediately switched to ensure the integrity of critical information. This on-demand resource allocation method saves storage and bandwidth more effectively compared to fixed-frequency frame extraction.

[0066] As shown in Figure 1, this application embodiment provides a multi-camera image frame extraction method, including:

[0067] S101. Image data corresponding to multiple cameras is collected at a first preset data sampling frequency, and a frame-skipping decision action is obtained based on the image data corresponding to each camera. The frame-skipping decision action includes either storing the skipped frames or not storing them.

[0068] The first preset data sampling frequency can be set to a low frequency, i.e., a longer sampling interval, thereby achieving low-frequency monitoring and reducing the amount of data processing. For example, it can be set to sample once every 5 seconds, so that the image data corresponding to each camera can be identified once every 5 seconds to obtain frame-sampling decision-making action. However, for practical applications, a 5-second interval is too long and can be set according to actual needs. For example, it can be set to sample once every 1 second or 2 seconds, which can achieve the effect of low-frequency monitoring while avoiding the omission of key information.

[0069] S102. If the frame-slicing decision action is frame-slicing storage, then a pre-set target recognition model is used to recognize the image data and determine the target captured by each camera.

[0070] The pre-set target recognition model can be a deep learning model, such as the YOLO model, which can recognize various targets and provide data support for subsequent frame extraction processing.

[0071] S103. Adjust the first preset data sampling frequency to the second preset data sampling frequency corresponding to the target, and perform linked frame extraction processing on multiple cameras through the second preset data sampling frequency to obtain the target image frame.

[0072] The attributes of the target image frame include its corresponding camera and the frame extraction time. The second preset data sampling frequency is higher than the first preset data sampling frequency, thereby forming a two-level frame extraction strategy that combines low-frequency monitoring and high-frequency tracking. This ensures that critical moments are not missed, while also reducing computing power consumption and storage space consumption in the absence of events.

[0073] By analyzing the overlapping areas of images from different cameras, it is possible to perform synchronized frame extraction for moving targets across cameras. When one camera detects a target and increases its sampling rate, it will trigger other cameras with overlapping fields of view to also increase their sampling rates. This means that stored image frames are no longer isolated data points, but can be linked together to form a complete and continuous set of events, greatly improving the efficiency and accuracy of event backtracking and global analysis.

[0074] S104. Associate and store all target image frames to form an event set, and arrange the event set in chronological order to complete the image frame extraction from multiple cameras.

[0075] The extracted target image frames can be associated with and stored along with their source camera, timestamp, and other attribute information, and arranged into an event set in chronological order. This structured data organization not only facilitates rapid retrieval and querying of events at specific times and locations, but also provides a high-quality, high-value data foundation for subsequent intelligent applications such as data mining and behavioral analysis.

[0076] In one possible implementation, after obtaining the frame-skipping decision action based on the image data corresponding to each camera, the method further includes:

[0077] If the frame-skipping decision action is to not skim and store frames, then the process of determining the next frame-skipping decision action will begin.

[0078] In one possible implementation, image data corresponding to multiple cameras is acquired at a first preset data sampling frequency, and a frame-skipping decision action is obtained based on the image data corresponding to each camera, including:

[0079] For any camera, image data captured by the camera is collected at a first preset data sampling frequency, and it is determined whether the scene has changed between the current frame and the previous frame. If so, the frame extraction decision action is determined to be frame extraction and storage; otherwise, the frame extraction and storage action is determined to be no frame extraction and storage.

[0080] Whether a scene has changed can be determined by calculating the differences between adjacent frames (such as pixel differences, histogram differences, or brightness differences). There are many existing technologies that can be used to determine scene changes, and this application will not elaborate on them; existing technologies can be used. For example, the sum of the absolute differences of the grayscale values ​​or color values ​​of corresponding pixels in two frames can be calculated. If the sum exceeds a preset threshold, it is determined that the scene has changed.

[0081] In one possible implementation, the pre-setting method for the target recognition model is as follows:

[0082] Construct a YOLO model and initialize the corresponding model parameters. Encode the initialized model parameters into a vector to obtain the parameter vector.

[0083] For example, the model parameters can be randomly initialized between the upper and lower limits and encoded as a vector to obtain the parameter vector.

[0084] Repeatedly obtain multiple different parameter vectors.

[0085] Obtain the loss function value corresponding to each parameter vector, and determine the parameter vector with the smallest loss function value as the optimal vector.

[0086] Obtaining the loss function value for the YOLO model is relatively straightforward, and will not be described in detail in the embodiments of this application.

[0087] Based on the optimal vector, a boundary-constrained search strategy is used to perform a local region search on the parameter vector to obtain the parameter vector after the local region search.

[0088] For any parameter vector obtained after searching a local region, a gradient information search strategy is used to perform a neighborhood search on the parameter vector obtained after the local region search, thus obtaining the parameter vector after the neighborhood search.

[0089] For any parameter vector obtained after a neighborhood search, an adaptive memory learning strategy is used to perform a balance search on the parameter vector obtained after the neighborhood search, thus obtaining the parameter vector after the balance search.

[0090] For any parameter vector obtained after a balance search, a random mutation search strategy is used to perform a global search on the parameter vector obtained after the balance search, resulting in a parameter vector obtained after the global search.

[0091] Obtain the current training stage. The current training stage can be defined as either completed training or incomplete training.

[0092] If the current training phase is not yet complete, the optimal vector is re-obtained based on the parameter vector obtained after global search, and the hyperparameters of the re-obtained optimal vector are used as the final hyperparameters of the YOLO model to obtain the target recognition model.

[0093] If the current training stage is incomplete, then the step of determining the parameter vector with the minimum loss function value is returned as the optimal vector.

[0094] This application proposes a YOLO model training method that integrates four strategies: boundary constraint search, gradient information search, adaptive memory learning, and random mutation search. This hybrid optimization algorithm can balance the ability of global exploration and local exploitation, effectively avoiding the problem of traditional optimization algorithms getting trapped in local optima. This results in a target recognition model with higher recognition accuracy and stronger generalization ability, ensuring accurate subsequent frame extraction decisions.

[0095] Optionally, after each change in the parameter vector, out-of-bounds handling can be performed to ensure training effectiveness.

[0096] In one possible implementation, based on the optimal vector, a boundary-constrained search strategy is used to perform a local region search on the parameter vector to obtain a parameter vector after the local region search, including:

[0097] Based on the optimal vector, the upper and lower bound vectors for the search of the parameter vector are determined as follows:

[0098]

[0099]

[0100] in, Represents the optimal vector. This represents the lower bound vector corresponding to the parameter vector. This represents the upper bound vector corresponding to the parameter vector. This represents the lower boundary vector of the search for the parameter vector. This represents the upper bound vector of the search for the parameter vector. This represents the search boundary control parameters, and =1-k / K, where k represents the current training iterations and K represents the maximum training iterations. Indicates will and The model parameters for each dimension are compared, and the larger model parameters for each dimension are combined into a vector to obtain... . Indicates will and The model parameters for each dimension are compared, and the smaller model parameters for each dimension are combined into a vector to obtain... .

[0101] Based on the upper and lower bound vectors of the search, a local region search is performed on the parameter vector to obtain the parameter vector after the local region search:

[0102]

[0103] in, Let m represent the m-th parameter vector in the k-th training process, where m = 1, 2, ..., M, and M represents the total number of parameter vectors. This represents the parameter vector after the search of the m-th local region. Represents the first random number between (0,1). This represents the second random number between (0,1).

[0104] This application employs a boundary-restricted search strategy to perform local region search on the parameter vector, which can effectively improve the probability of the algorithm finding a better position. At the same time, compared with the prior art that directly learns the information of the optimal vector, this boundary-restricted search strategy can also delay getting trapped in local optima.

[0105] In one possible implementation, for any parameter vector obtained after searching a local region, a gradient information search strategy is used to perform a neighborhood search on the parameter vector obtained after the local region search, resulting in a parameter vector obtained after the neighborhood search, including:

[0106] Arrange the parameter vectors obtained after local region search in descending order of loss function values ​​to obtain the sorted parameter vectors.

[0107] A gradient information search strategy is used to perform a neighborhood search on the permuted parameter vector, resulting in the following parameter vector after the neighborhood search:

[0108]

[0109]

[0110]

[0111] in, Let represent the parameter vector after the nth permutation in the kth training process. This represents the parameter vector after the nth neighborhood search. This represents the gradient information search control factor. This represents a parameter vector where all elements are 1. Represents a symbolic function. express gradient information, express The corresponding solution quality factor, express The corresponding solution quality factor is: Solution quality factor = 1 / (loss function value + 0.0001). This represents a random parameter vector where each element is randomly set to either 0.1 or -0.1. Represents a random number between (-1, 1). This represents the basic search volume for gradient information. This represents the parameter influencing the gradient information search control factor. Represents the sine function. Represents pi (π). Represents a logarithmic function.

[0112] The embodiments of this application employ a gradient information search strategy to perform neighborhood search on the parameter vector after local region search. This not only effectively realizes the search of the neighborhood of the parameter vector in the solution space, but also effectively avoids search collisions and improves the local search effect.

[0113] In one possible implementation, for any parameter vector obtained after a neighborhood search, an adaptive memory learning strategy is used to perform a balance search on the parameter vector obtained after the neighborhood search, resulting in the following parameter vector after the balance search:

[0114]

[0115]

[0116] in, Let represent the parameter vector after the j-th neighborhood search during the k-th training iteration. Let represent the parameter vector after the j-th balance search. A random number represented as 1 or -1. This represents the historical position memory coefficient between (0, 0.2). Indicates inertia weight, This represents the parameter vector after the j-th neighborhood search during the (k-1)-th training process. Represents the worst-case vector. This represents the maximum value of the inertia weight. This represents the minimum value of the inertia weight.

[0117] The embodiments of this application employ an adaptive memory learning strategy to perform a balanced search on the parameter vector after the neighborhood search. This can enable the parameter vector to have a greater global search capability and a wider search range in the early and middle stages of the algorithm, while gradually reducing the influence of inferior information in the later stages of the algorithm and improving the convergence accuracy of the algorithm.

[0118] In one possible implementation, for any parameter vector obtained after a balance search, a random mutation search strategy is used to perform a global search on the parameter vector obtained after the balance search, resulting in the following parameter vector after the global search:

[0119]

[0120]

[0121] in, Let represent the parameter vector after the i-th balance search during the k-th training process. This represents the parameter vector after the i-th global search. Represents the cosine function. Represents a random variable factor. Represents the control coefficient for random variation. The 'h' indicates the mutation level, and 'h' indicates the current level. Represented as Other parameter vectors that are randomly matched, express and The Euclidean distance between them.

[0122] This application employs a random mutation search strategy to perform a global search on the parameter vector after the balance search, which can provide training diversity and give the algorithm a strong ability to escape local optima. As the algorithm progresses, a large global search capability may not be necessary. Therefore, when the number of training iterations exceeds 2K / 3, a greedy strategy can be used to control the global search, accepting only global searches that reduce the loss function value of the parameter vector, thereby improving the algorithm's search accuracy and speed.

[0123] By combining four strategies—boundary constraint search, gradient information search, adaptive memory learning, and random mutation search—it can balance the ability of global exploration and local development, effectively avoiding the problem of traditional optimization algorithms getting trapped in local optima. This results in a target recognition model with higher recognition accuracy and stronger generalization ability, ensuring accurate frame extraction decisions in the future.

[0124] In one possible implementation, the first preset data sampling frequency is adjusted to a second preset data sampling frequency corresponding to the target, and the target image frame is obtained by performing linked frame extraction processing on multiple cameras using the second preset data sampling frequency, including:

[0125] Adjust the first preset data sampling frequency to the second preset data sampling frequency corresponding to the target.

[0126] The camera that captures the image data is stored in the target is designated as the first camera to extract frames. It is then determined whether the target intersects with the overlapping area of ​​the images captured by the first camera to extract frames and other cameras. If so, the other cameras are designated as the second cameras to extract frames; otherwise, no processing is performed on the other cameras.

[0127] Based on the second preset data sampling frequency, the first and second cameras to be sampled are subjected to linked frame extraction processing to obtain the target image frame.

[0128] The second preset data sampling frequency needs to be less than the first preset data sampling frequency. For example, if the first preset data sampling frequency is set to collect data once every 1 second, then the second preset data sampling frequency can be set to collect data once every 0.5 seconds.

[0129] Optionally, a corresponding second preset data sampling frequency can be set for each type of target. For example, for a certain park, if it is necessary to monitor vehicles and personnel, a second preset data sampling frequency can be set for vehicles and a second preset data sampling frequency can be set for personnel. When a vehicle or personnel is identified, frame extraction can be performed according to the second preset data sampling frequency.

[0130] Frame extraction can also be performed only when a target is detected. The type of target to be extracted can be defined by the staff, thus achieving a highly customized frame extraction task. If the target disappears during the frame extraction process, the extraction ends, and the extracted target image frames are stored as an event set.

[0131] It's worth noting that if two or more targets appear in the same camera, the tag information for all targets can be annotated in the attributes of the target image frames, and all target image frames obtained from the linked frame extraction can be merged into an event set. Therefore, the event set can also be named, namely the tag information of the appearing targets and the initial time of their appearance.

[0132] As shown in Figure 2, based on the same inventive concept, this application provides a multi-channel camera image frame extraction system, including: a frame extraction decision module 201, a target recognition module 202, a linkage frame extraction module 203, and an event set storage module 204.

[0133] The frame-skipping decision module 201 is used to collect image data corresponding to multiple cameras at a first preset data sampling frequency, and to obtain a frame-skipping decision action based on the image data corresponding to each camera. The frame-skipping decision action includes either storing frames or not storing frames.

[0134] The target recognition module 202 is used to identify the image data using a pre-set target recognition model when the frame-slicing decision action is frame-slicing storage, thereby determining the target captured by each camera.

[0135] The linked frame extraction module 203 is used to adjust the first preset data sampling frequency to the second preset data sampling frequency corresponding to the target, and to perform linked frame extraction processing on multiple cameras through the second preset data sampling frequency to obtain the target image frame.

[0136] The attributes of the target image frame include its corresponding camera and the frame extraction time; the second preset data sampling frequency is higher than the first preset data sampling frequency.

[0137] The event set storage module 204 is used to associate and store all target image frames to form an event set, and to arrange the event set in chronological order to complete the image frame extraction from multiple cameras.

[0138] The multi-channel camera image frame extraction system provided in this application embodiment can execute the Shanghai Su method technical solution, and its principle and beneficial effects are similar, so they will not be described again here.

[0139] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0140] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus, electronic devices, and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal equipment to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal equipment, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0141] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0142] These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable terminal equipment, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0143] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0144] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0145] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for frame extraction from multi-channel camera images, characterized in that, include: Image data from multiple cameras is collected at a first preset data sampling frequency, and a frame-skipping decision action is obtained based on the image data from each camera; the frame-skipping decision action includes either storing the skipped frames or not storing them. When the frame extraction decision action is frame extraction storage, a pre-set target recognition model is used to identify the image data and determine the target captured by each camera; the first preset data sampling frequency is adjusted to the second preset data sampling frequency corresponding to the target, and the multi-camera linked frame extraction process is performed through the second preset data sampling frequency to obtain the target image frame; wherein, the attributes of the target image frame include its corresponding camera and the frame extraction time; the second preset data sampling frequency is higher than the first preset data sampling frequency; all target image frames are stored together to form an event set, and the event set is arranged in chronological order to complete the image frame extraction of multiple cameras; the pre-setting method of the target recognition model is as follows: construct a YOLO model and initialize the model parameters corresponding to the YOLO model, encode the initialized model parameters into vectors to obtain parameter vectors; repeatedly obtain multiple different parameter vectors; obtain the loss function value corresponding to each parameter vector, and determine the parameter vector with the smallest loss function value as the optimal vector; based on the optimal vector, a boundary constraint search strategy is used to perform local region search on the parameter vector to obtain the parameter vector after the local region search; for any parameter vector after the local region search, a boundary constraint search strategy is used to perform local region search on the parameter vector. The parameter vector after local region search is searched using a gradient information search strategy to obtain a parameter vector after local region search. For any parameter vector after local region search, an adaptive memory learning strategy is used to perform a balance search to obtain a parameter vector after balance search. For any parameter vector after balance search, a random mutation search strategy is used to perform a global search to obtain a parameter vector after global search. The current training stage is obtained, where the current training stage is either completed or incomplete. If the current training stage is completed, the optimal vector is re-obtained based on the parameter vector after global search, and the hyperparameters of the re-obtained optimal vector are used as the final hyperparameters of the YOLO model to obtain the target recognition model. If the current training stage is incomplete, the step of determining the parameter vector with the minimum loss function value as the optimal vector is returned. Based on the optimal vector, a boundary constraint search strategy is used to perform local region search on the parameter vector to obtain a parameter vector after local region search, including: determining the upper boundary vector and lower boundary vector of the parameter vector search based on the optimal vector. in, Represents the optimal vector. This represents the lower bound vector corresponding to the parameter vector. This represents the upper bound vector corresponding to the parameter vector. This represents the lower boundary vector of the search for the parameter vector. This represents the upper bound vector of the search for the parameter vector. This represents the search boundary control parameters, and =1-k / K, where k represents the current training iterations and K represents the maximum training iterations. Indicates will and The model parameters for each dimension are compared, and the larger model parameters for each dimension are combined into a vector to obtain... ； Indicates will and The model parameters for each dimension are compared, and the smaller model parameters for each dimension are combined into a vector to obtain... Based on the upper and lower bound vectors of the search, a local region search is performed on the parameter vector to obtain the parameter vector after the local region search: in, Let m represent the m-th parameter vector in the k-th training process, where m = 1, 2, ..., M, and M represents the total number of parameter vectors. This represents the parameter vector after the search of the m-th local region. Represents the first random number between (0,1). Let represent the second random number between (0,1); for any parameter vector after searching a local region, a gradient information search strategy is used to perform a neighborhood search on the parameter vector after searching the local region, resulting in a parameter vector after neighborhood search. This includes: arranging the parameter vectors after searching the local region in descending order of loss function values ​​to obtain an arranged parameter vector; and then using the gradient information search strategy to perform a neighborhood search on the arranged parameter vector to obtain the parameter vector after neighborhood search. in, Let represent the parameter vector after the nth permutation in the kth training process. This represents the parameter vector after the nth neighborhood search. This represents the gradient information search control factor. This represents a parameter vector where all elements are 1. Represents a symbolic function. express gradient information, express The corresponding solution quality factor, express The corresponding solution quality factor is: Solution quality factor = 1 / (loss function value + 0.0001). This represents a random parameter vector where each element is randomly set to either 0.1 or -0.

1. Represents a random number between (-1, 1). This represents the basic search volume for gradient information. This represents the parameter influencing the gradient information search control factor. Represents the sine function. Represents pi (π). Let represent the logarithmic function; for any parameter vector after neighborhood search, an adaptive memory learning strategy is used to perform a balance search on the parameter vector after neighborhood search, resulting in the parameter vector after balance search: in, Let represent the parameter vector after the j-th neighborhood search during the k-th training iteration. Let represent the parameter vector after the j-th balance search. A random number represented as 1 or -1. This represents the historical position memory coefficient between (0, 0.2). Indicates inertia weight, This represents the parameter vector after the j-th neighborhood search during the (k-1)-th training process. Represents the worst-case vector. This represents the maximum value of the inertia weight. This represents the minimum value of the inertia weight. For any parameter vector obtained after the balance search, a random mutation search strategy is used to perform a global search on the parameter vector obtained after the balance search, resulting in the following parameter vector: in, Let represent the parameter vector after the i-th balance search during the k-th training process. This represents the parameter vector after the i-th global search. Represents the cosine function. Represents a random variable factor. Represents the control coefficient for random variation. The 'h' indicates the mutation level, and 'h' indicates the current level. Represented as Other parameter vectors that are randomly matched, express and The Euclidean distance between them.

2. The multi-channel camera image frame extraction method according to claim 1, characterized in that, After obtaining the frame-sampling decision action based on the image data corresponding to each camera, the process further includes: if the frame-sampling decision action is not to store frames, then proceeding to the judgment process for the next frame-sampling decision action.

3. The multi-channel camera image frame extraction method according to claim 1, characterized in that, The image data corresponding to multiple cameras is collected through a first preset data sampling frequency, and a frame-skipping decision action is obtained based on the image data corresponding to each camera. This includes: for any camera, collecting the image data collected by the camera through the first preset data sampling frequency, and determining whether the scene between the current frame and the previous frame has changed. If so, the frame-skipping decision action is determined to be frame-skipping storage; otherwise, the frame-skipping storage action is determined to be no-frame-skipping storage.

4. The multi-channel camera image frame extraction method according to claim 1, characterized in that, The process involves adjusting the first preset data sampling frequency to the second preset data sampling frequency corresponding to the target, and performing linked frame extraction processing on multiple cameras using the second preset data sampling frequency to obtain the target image frame. This includes: adjusting the first preset data sampling frequency to the second preset data sampling frequency corresponding to the target; designating the camera whose captured image data is stored at the target as the first camera to be extracted from, and determining whether the target intersects with the overlapping area of ​​the captured images from the first camera to be extracted from and other cameras. If so, the other cameras are designated as the second cameras to be extracted from; otherwise, no processing is performed on the other cameras. Based on the second preset data sampling frequency, linked frame extraction processing is performed on the first camera to be extracted from and the second camera to be extracted from to obtain the target image frame.

5. A multi-channel camera image frame extraction system, wherein the multi-channel camera image frame extraction system is used to execute the multi-channel camera image frame extraction method according to any one of claims 1-4, characterized in that, include: The module includes a frame extraction decision module, a target recognition module, a linked frame extraction module, and an event set storage module. The frame-skipping decision module is used to collect image data corresponding to multiple cameras at a first preset data sampling frequency, and to obtain a frame-skipping decision action based on the image data corresponding to each camera; the frame-skipping decision action includes storing frames or not storing frames. The target recognition module is used to identify the target captured by each camera by using a pre-set target recognition model when the frame extraction decision action is frame extraction storage. The linked frame extraction module is used to adjust the first preset data sampling frequency to the second preset data sampling frequency corresponding to the target, and to perform linked frame extraction processing on multiple cameras through the second preset data sampling frequency to obtain the target image frame. The attributes of the target image frame include its corresponding camera and the frame extraction time. The second preset data sampling frequency is higher than the first preset data sampling frequency. The event set storage module is used to store all target image frames together to form an event set, and to arrange the event set in chronological order to complete the image frame extraction from multiple cameras.

Images