A method and system for quickly retrieving video data from the back kitchen of a catering industry

Abstract

The present invention provides a method and system for quickly retrieving back kitchen video data in a catering industry, and relates to the technical field of data processing. The method comprises: receiving a back kitchen video retrieval instruction; responding to the back kitchen video retrieval instruction, determining a video trigger frame according to a moving target appearing in a back kitchen monitoring area; capturing constant features of the moving target in the video trigger frame, wherein the constant features include key points based on image gradient extreme value points and the main direction of the key points; extracting a video segment from a back kitchen video using the constant features as a start identifier and an end identifier; calculating the Lyapunov index of the video segment; extracting the critical point of the Lyapunov index, and using the video frame corresponding to the critical point as a video key frame; using the video key frame as an index mark of the video segment, and outputting the video segment; displaying the video key frame, and completing the retrieval of the back kitchen video. The retrieval speed of the back kitchen video data is optimized to ensure the safety of the back kitchen.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and system for quickly retrieving kitchen video data in a catering industry. Background Art

[0002] As people's living standards continue to improve, food safety issues have gradually come into the spotlight. As the source of video production, the catering industry kitchen is the most important node to ensure food safety. Therefore, monitoring the catering industry kitchen is a necessary link to ensure food safety. In order to ensure the compliance and safety of the food production process, the catering industry kitchen is equipped with video surveillance for continuous video collection. For problematic food, responsibility can be traced. Retrieving surveillance video data can help restaurant managers monitor the food processing and preparation process in real time to ensure that food safety standards are followed. If a food safety incident occurs, it can be quickly traced back to the specific operation process and the person responsible. In this way, food safety can be better ensured.

[0003] In addition, in order to ensure the safety of the catering industry's back kitchen, continuous and uninterrupted monitoring is required to ensure unmanned operation, prevent non-back kitchen personnel from entering and leaving the back kitchen, and maintain 24-hour security of the back kitchen. When problems arise, the video information of the back kitchen intrusion can be quickly and accurately located so that the problem can be solved in real time. Video surveillance can act as a deterrent and reduce improper behavior of employees or outsiders.

[0004] However, since the kitchen surveillance video is often disconnected based on a fixed time interval or is a long-distance surveillance video, no matter which method is used to obtain the surveillance video, it cannot be previewed when the video is retrieved, and a lot of time is consumed to review it frame by frame to avoid missing key information. It can be seen that the existing kitchen video data retrieval process has a large workload, low efficiency and is very easy to miss key information. Summary of the invention

[0005] In order to solve the problems existing in the prior art that the kitchen surveillance video is often disconnected based on fixed time intervals or is a long-distance surveillance video, no matter which surveillance video is obtained, it cannot be previewed when the video is retrieved, and a lot of time is consumed to flip through frame by frame to avoid missing key information. It can be seen that the existing kitchen video data retrieval process has a large workload, low efficiency and is very easy to miss the key point information. The present invention provides a method and system for quickly retrieving kitchen video data in the catering industry.

[0006] The technical solution provided by the embodiment of the present invention is as follows:

[0007] First aspect

[0008] An embodiment of the present invention provides a method for quickly retrieving video data of a catering kitchen, comprising:

[0009] S1: Receive the kitchen video retrieval instruction;

[0010] S2: responding to the kitchen video retrieval instruction, determining the video trigger frame according to the moving target appearing in the kitchen monitoring area;

[0011] S3: Capturing constant features of the moving target in the video trigger frame, wherein the constant features include key points based on image gradient extreme points and key point main directions;

[0012] S4: extracting a video segment from the kitchen video using the constant feature as a start identifier and an end identifier, wherein the video segment includes multiple video frames having the same moving target;

[0013] S5: Calculate the Lyapunov exponent of the video segment;

[0014] S6: extracting the critical point of the Lyapunov exponent, and taking the video frame corresponding to the critical point as the video key frame;

[0015] S7: Use the video key frame as an index mark of the video segment and output the video segment;

[0016] S8: Display the video key frames and complete the retrieval of the kitchen video.

[0017] Furthermore, the back kitchen video retrieval instructions include voice instructions, touch screen instructions, API instructions and GUI button instructions.

[0018] Furthermore, S2 specifically includes:

[0019] S201: Determine the initial kitchen background frame:

[0020]

[0021] in, Represents the initial kitchen background frame, Indicates i The grayscale value of the video frame behind the frame, N Indicates the number of kitchen video frames involved in the construction of kitchen background frames;

[0022] S202: Calculate the pixel difference between the current kitchen video frame and the initial kitchen background frame:

[0023]

[0024] in, Indicates time t The pixel difference is Indicates taking the absolute value;

[0025] S203: Preprocessing the pixel differences including binarization and morphological dilation in sequence, and outputting the moving target represented by morphology:

[0026] Binarization:

[0027] Morphological expansion:

[0028] in, Representing coordinates The binary value at Representing coordinates At the moment t The pixel difference is Representing coordinates The pixel difference threshold at represents the morphological dilation operator, express t The moving target represented by the morphology collected at every moment;

[0029] S204: Update the initial kitchen video frame and return to step S202:

[0030]

[0031] in, represents the learning rate, Indicates the updated initial kitchen background frame;

[0032] S205: Outputting the video frame with the moving target as a video trigger frame.

[0033] Furthermore, S3 specifically includes:

[0034] S301: Constructing scale spaces of video trigger frames at multiple scales:

[0035]

[0036] in, express The video trigger frame at this scale is Scale space under scale, represents the Gaussian kernel, Indicates the video trigger frame. Indicates the pixel coordinates of the video trigger frame. Represents the convolution operation;

[0037] S302: Obtain the maximum and minimum points of each pixel point in each scale space within a preset neighborhood, and use the maximum and minimum points as key points;

[0038] S303: Perform Taylor series expansion on the key points to relocate the key points, wherein the key points obtained by relocation are the coincidence points of the gradient optimal point and the curvature optimal point:

[0039]

[0040] in, Indicates key points p The differential scale space value at describes the key point p A function of the change in pixel value of surrounding pixels, Indicates key points The difference scale space value at , Indicates key points To the real key point The offset of the key point The gradient value at Indicates key points The Hessian matrix at the key point The curvature of the scale space at that point. The subscript T indicates transposition.

[0041] S304: Determine the main direction of the key point according to the distribution of the relocated key point in the direction of the local image gradient:

[0042]

[0043] in, Represents the coordinates of the key points obtained by relocation The main direction of the key points, Indicates the gradient direction of each pixel in the preset neighborhood of the key point. Represents the inverse tangent function, sin represents the calculation of the sine value, and cos represents the calculation of the cosine value. Represents the summation symbol.

[0044] Furthermore, S4 specifically includes:

[0045] S401: Retrieving constant features in each video frame in the kitchen video;

[0046] S402: When there are constant features in the video frame, the video frame is retained; otherwise, the process returns to S401 until the video frames in the kitchen video are retrieved.

[0047] Furthermore, S5 specifically includes:

[0048] S501: extracting video frame intensity features of each video frame in the video segment, wherein the video frame intensity features form a state vector in chronological order;

[0049] S502: Perform time delay embedding on the state vector to reconstruct the state space of the video segment:

[0050]

[0051] in, express t The state vector of the video frame at time instant, Indicates time delay, m represents the embedding dimension, represents the state space;

[0052] S503: Calculate the Lyapunov exponent of the state space, that is, the Lyapunov exponent of the video segment:

[0053]

[0054] in, represents the Lyapunov exponent, Indicates infinity, and They represent the initial state space and t The time state space, log represents the natural logarithmic function, and lim represents the limit.

[0055] Furthermore, S6 specifically includes:

[0056] S601: Perform sliding average on the Lyapunov exponent to smooth the Lyapunov exponent:

[0057]

[0058] in, express t The Lyapunov exponent at time is obtained by smoothing the Lyapunov exponent. express t Time increase i After a time unit, , express Lyapunov exponent of the moment;

[0059] S602: extracting extreme values ​​from the Lyapunov exponents obtained after smoothing by using a findpeaks function, wherein the moment corresponding to the extreme value is the critical point, wherein the extreme value includes a peak value and a valley value;

[0060] S603: Output the video frame at the time corresponding to the critical point as a video key frame.

[0061] Furthermore, there are multiple video key frames, and the method further includes:

[0062] Receive a click instruction of a video key frame and display the video segment corresponding to the video key frame.

[0063] Second aspect

[0064] An embodiment of the present invention provides a system for quickly retrieving video data of a catering kitchen, comprising:

[0065] processor;

[0066] A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method for quickly retrieving the video data of the back kitchen of the catering industry as described in the first aspect is implemented.

[0067] The third aspect

[0068] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for quickly retrieving back-kitchen video data of the catering industry as described in the first aspect is implemented.

[0069] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0070] In the present invention, a video trigger frame is determined according to a moving target appearing in a back kitchen monitoring area, and the video trigger frame at the moment when the moving target enters the back kitchen is quickly located. The entire video segment containing the same moving target is extracted by using a constant feature that does not change across frames, and the video segment is regarded as a dynamic system. The Lyapunov exponent of this dynamic system is calculated to select video key frames, identify critical point moments with significant content changes, that is, moments corresponding to important events or dynamic changes in the video, and finally display these video key frames to provide a fast and representative content preview for video retrieval personnel, ensure that all key information is effectively displayed, avoid the problem of easily missing key point information caused by manual video retrieval, and then help video retrieval personnel quickly locate and intercept the required video content. Through these steps, fast, efficient and accurate retrieval of back kitchen video data can be achieved, effectively reducing the workload of retrieval of back kitchen video data, which is suitable for back kitchen scenes that need to quickly review key events, ensure the safety of the back kitchen environment, and quickly pay attention to and hold accountable for irregular behaviors in the back kitchen based on this video extraction method. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0072] Figure 1 A schematic diagram of a process for quickly retrieving video data of a catering kitchen provided by an embodiment of the present invention;

[0073] Figure 2 A schematic diagram of the structure of a system for quickly retrieving video data from a catering kitchen provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0074] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0075] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0076] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0077] In the embodiments of the present invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are consistent.

[0078] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0079] Reference Manual Attached Figure 1 , which shows a flow chart of a method for quickly retrieving kitchen video data in the catering industry provided by an embodiment of the present invention.

[0080] The embodiment of the present invention provides a method for quickly retrieving video data from a catering kitchen. The method can be implemented by a device for quickly retrieving video data from a catering kitchen. The device for quickly retrieving video data from a catering kitchen can be a terminal or a server. The processing flow of the method for quickly retrieving video data from a catering kitchen can include the following steps:

[0081] S1: Receive the kitchen video retrieval instruction.

[0082] Among them, the kitchen video retrieval instructions include direct user requests from different sources or automatically triggered events.

[0083] In a possible implementation, the kitchen video retrieval instruction includes a voice instruction, a touch screen instruction, an API instruction, and a GUI button instruction.

[0084] It should be noted that the diversity of back-kitchen video retrieval commands includes voice commands, touch screen commands, API commands and GUI button commands, allowing users to choose the most suitable interaction method according to different scenarios and preferences. These options provide flexibility and convenience, ensuring that users of different technical levels can effectively access and manage back-kitchen video data.

[0085] S2: In response to the kitchen video retrieval instruction, a video trigger frame is determined according to a moving target appearing in the kitchen monitoring area.

[0086] The video trigger frame is the video frame when a person in the kitchen video enters the monitoring area for the first time. Directly positioning to the scene when a person enters the kitchen video can avoid repeatedly pulling too long kitchen videos, greatly shortening the time to retrieve kitchen videos. Identify the frame in the video where a person or moving target enters the monitoring area for the first time, so as to quickly locate the specific moment when the person's activity begins, thereby effectively reducing unnecessary video playback and search time, improving the efficiency and accuracy of video retrieval, speeding up video retrieval response, and ensuring that only video content related to the request is focused on.

[0087] In a possible implementation, S2 specifically includes:

[0088] S201: Determine the initial kitchen background frame:

[0089]

[0090] in, Represents the initial kitchen background frame, Indicates i The grayscale value of the video frame behind the frame, N Indicates the number of kitchen video frames involved in constructing the kitchen background frame.

[0091] S202: Calculate the pixel difference between the current kitchen video frame and the initial kitchen background frame:

[0092]

[0093] in, Indicates time t The pixel difference is Indicates taking the absolute value.

[0094] S203: Preprocessing the pixel differences including binarization and morphological dilation in sequence, and outputting the moving target represented by morphology:

[0095] Binarization:

[0096] Morphological expansion:

[0097] in, Representing coordinates The binary value at Representing coordinates At the moment t The pixel difference is Representing coordinates The pixel difference threshold at represents the morphological dilation operator, express t The moving target represented by the morphology collected at every moment.

[0098] It should be noted that those skilled in the art can set the pixel difference threshold value according to actual needs, and the present invention does not limit this.

[0099] S204: Update the initial kitchen video frame and return to step S202:

[0100]

[0101] in, represents the learning rate, Represents the updated initial kitchen background frame.

[0102] Optionally, the learning rate can be set to 0.01 or 0.05 to ensure that the background can be updated smoothly and can adapt to environmental changes quickly. This method can effectively smooth the effects caused by noise or light changes. When the environment changes quickly, the learning rate value can be appropriately increased to quickly adapt to the environment. In this way, the background model can be continuously updated and will not overreact to temporary changes, avoiding the collection of too many redundant identical moving targets.

[0103] S205: Outputting the video frame with the moving target as a video trigger frame.

[0104] It is understandable that by continuously updating the background image, different video trigger frames can be obtained, and video frames at multiple moments can be accurately captured to avoid missing key information of the moving target.

[0105] Specifically, first, the initial kitchen background frame is determined by averaging the grayscale values ​​of multiple frames. Then, the system calculates the pixel difference between the new frame and the background frame, and processes these differences through binarization and morphological dilation to accurately identify moving targets. The background frame will be updated according to the real-time video data to adapt to environmental changes and ensure that only relevant moving targets are captured. Finally, the moving target will be output as a video trigger frame, thereby effectively monitoring and responding to activities in the kitchen area, ensuring that the system captures key events in real time and accurately. This process is automated and efficient, greatly improving the practicality and response speed of the video surveillance system.

[0106] S3: Capturing constant features of moving targets in video trigger frames.

[0107] Among them, the constant features include key points based on image gradient extreme points and the main directions of key points.

[0108] Among them, constant features are features that keep the moving target consistent across frames, so as to determine whether the moving target is the same target. The constant features of the moving target are captured in the determined video trigger frame. These features are used to ensure that the same target can be tracked consistently across frames in the video. Constant features include key points based on the extreme points of the image gradient and their main directions. These key points are points with high significance in the image, which usually appear stably in multiple frames, and the main direction of the key points provides important information about the directionality of the target. Through the continuous tracking of these features, the system can effectively identify and track the same moving target in the entire video segment, ensuring the accurate extraction of the complete dynamic sequence, and providing support for subsequent analysis and decision-making.

[0109] In a possible implementation, S3 specifically includes:

[0110] S301: Constructing scale spaces of video trigger frames at multiple scales:

[0111]

[0112] in, express The video trigger frame at this scale is Scale space under scale, represents the Gaussian kernel, Indicates the video trigger frame. Indicates the pixel coordinates of the video trigger frame. Represents a convolution operation.

[0113] S302: Obtain the maximum and minimum points of each pixel point in each scale space within a preset neighborhood, and use the maximum and minimum points as key points.

[0114] It should be noted that those skilled in the art can set the size of the preset neighborhood according to actual needs, and the present invention is not limited thereto.

[0115] Optionally, the preset neighborhood may be set to a 26-neighborhood.

[0116] S303: Perform Taylor series expansion on the key points to relocate the key points, wherein the key points obtained by relocation are the coincidence points of the gradient optimal point and the curvature optimal point:

[0117]

[0118] in, Indicates key points p The differential scale space value at describes the key point p A function of the change in pixel value of surrounding pixels, Indicates key points The difference scale space value at , Indicates key points To the real key point The offset of the key point The gradient value at Indicates key points The Hessian matrix at the key point The curvature of the scale space at this point. The superscript T indicates transposition.

[0119] It should be noted that the Taylor series expansion of key points is to accurately determine the optimal position of key points through mathematical optimization. This process involves calculating the differential scale space values ​​around the key points, and using these values ​​to estimate the offset of the key points to their optimal positions (the coincidence point of the gradient optimal point and the curvature optimal point). By calculating the gradient and Hessian matrix of the key points (indicating the rate and curvature of change of pixel values), the system can accurately adjust the position of the key points to ensure that their representation in the image is both stable and accurate, significantly improving the positioning accuracy of feature points.

[0120] S304: Determine the main direction of the key point according to the distribution of the relocated key point in the direction of the local image gradient:

[0121]

[0122] in, Represents the coordinates of the key points obtained by relocation The main direction of the key points, Indicates the gradient direction of each pixel in the preset neighborhood of the key point. Represents the inverse tangent function, sin represents the calculation of the sine value, and cos represents the calculation of the cosine value. Represents the summation symbol.

[0123] It should be noted that and It is the sine and cosine values ​​of the gradient direction, which is used to convert the direction angle into the component representation in the orthogonal coordinate system to facilitate summation and average calculation. and The summation means that the sine and cosine values ​​of all gradient directions in the area around the key point are accumulated separately. This is done to average the gradient direction in the entire area and obtain a representative direction value. , which is used to convert the accumulated sine and cosine values ​​back to an angle, which is the main direction of the key point.

[0124] Specifically, capturing the constant features of moving targets through multi-scale analysis in video trigger frames can ensure cross-frame consistency and accurate tracking. First, a scale space is constructed, and Gaussian blurs of different scales are applied to the video trigger frames to enhance the reliability of key point detection. Next, the maximum and minimum points are identified in these scale spaces, which are used as key points. These key points represent unique structural information in the image. The key points are then finely positioned through Taylor series expansion to optimize the key point position and determine its precise coordinates. Finally, the main direction of each key point is determined by calculating the gradient direction around the relocated key point, which helps to describe the directional characteristics of the key point and provide important information for subsequent feature matching and target tracking. The completion of this step not only enhances the accuracy of target recognition and tracking, but also provides key visual information for video analysis.

[0125] S4: Extract video segments from the kitchen video using constant features as the start identifier and the end identifier.

[0126] The video segment includes multiple video frames with the same moving target.

[0127] It is understandable that in the process of retrieving the back kitchen video, the starting point is actually the video trigger frame, because the video trigger frame is the video frame where the constant feature appears for the first time. The video segment including the same moving target is extracted from the back kitchen video through the constant feature that does not change across frames. This video segment records when the moving target enters and leaves the back kitchen area. The video segment obtained in this way can accurately reflect the activities of the same moving target in the back kitchen area. If another moving target enters during this period, the video segment of another moving target is re-captured. In this way, the precise activity behaviors of different moving targets in the back kitchen area can be accurately obtained, so as to reduce the workload of management personnel and video retrieval personnel, and avoid the failure to extract the video segment of the moving target in the back kitchen area due to human negligence, which greatly improves the ability to quickly and accurately extract back kitchen video data.

[0128] In a possible implementation, S4 specifically includes:

[0129] S401: Retrieve constant features in each video frame in the kitchen video.

[0130] S402: When there are constant features in the video frame, the video frame is retained; otherwise, the process returns to S401 until the video frames in the kitchen video are retrieved.

[0131] It should be noted that the system first retrieves predetermined constant features in each frame of the entire kitchen video. These features help identify and track moving targets in the video. If a frame contains these constant features, the frame is retained. Otherwise, the system will continue to retrieve other frames. This process continues until all frames of the video have been checked. This method ensures that only frames containing key information are retained in the video data, improving the efficiency and accuracy of video analysis.

[0132] S5: Calculate the Lyapunov exponent of the video segment.

[0133] Among them, the Lyapunov exponent is a measure used to quantify how small perturbations of initial conditions in a dynamic system affect the state of the system over time. When understanding video content as a dynamic system, the Lyapunov exponent can identify stable and chaotic regions in video sequences. Specifically, this index measures the speed of separation between adjacent states, i.e., consecutive frames. A positive Lyapunov exponent indicates that the system exhibits chaotic behavior, that is, small changes in initial conditions lead to significant differences in state predictions. In contrast, a zero or negative Lyapunov exponent indicates that the system is more stable and predictable. Analysis of the Lyapunov exponent can help us identify moments when the content changes significantly, which often correspond to important events or dynamic transitions in the video.

[0134] Frames with significant changes in the Lyapunov exponent are preferentially selected as key frames. These frames mark important state transitions in the dynamic system and contain important information that viewers need to know. The video sequence can be regarded as a dynamic system, and each video frame in the video segment is a state representation of the dynamic system. By applying dynamic system theory such as Lyapunov exponent analysis, chaotic and ordered areas in the video can be identified, and then frames showing key changes in the dynamic system can be preferentially selected based on the Lyapunov exponent.

[0135] In a possible implementation, S5 specifically includes:

[0136] S501: extracting video frame intensity features of each video frame in a video segment, wherein the video frame intensity features form a state vector in chronological order.

[0137] It should be noted that using video frame intensity as a feature does not require complex preprocessing or feature extraction techniques, saving computing resources. The video frame intensity of each frame directly reflects the visual content. By retaining the intensity information, it is possible to capture subtle changes in the video, which is critical for certain applications such as subtle motion detection. Moreover, using video frame intensity does not require prior knowledge of the video content, making this method very flexible when processing unknown or unlabeled video data. In addition, the advantage of using video frame intensity as a feature is that it simplifies data processing, reduces computational complexity, and effectively captures brightness changes, making it suitable for rapid analysis and monitoring of lighting changes or scene transitions. This method improves processing speed and saves resources.

[0138] S502: Perform time delay embedding on the state vector to reconstruct the state space of the video segment:

[0139]

[0140] in, express t The state vector of the video frame at time instant, Indicates time delay, m represents the embedding dimension, Represents the state space.

[0141] S503: Calculate the Lyapunov exponent of the state space, that is, the Lyapunov exponent of the video segment:

[0142]

[0143] in, represents the Lyapunov exponent, Indicates infinity, and They represent the initial state space and t The time state space, log represents the natural logarithmic function, and lim represents the limit.

[0144] Among them, the time t Represents from the initial time 0 to the time t The evolution time, in theory, t tends to infinity ( t →∞) to ensure the long-term behavior of the evaluation system. This factor represents how we normalize the time when evaluating exponential growth in order to get an average rate of divergence per unit time. is the Jacobian matrix, which represents the initial state To status The derivative of , that is, how the system state responds to the initial small change. The logarithm of the change in the system state from the initial time point to time t is calculated, which shows how the system diverges or converges exponentially over time for a small perturbation to the initial state. The calculation of the Lyapunov exponent quantifies the dynamic stability or degree of chaos of the system by evaluating the average rate of this logarithmic growth.

[0145] Specifically, the system first extracts the intensity features of each video frame and composes a state vector in time series. This approach simplifies the feature extraction process by directly using pixel intensity, making the analysis more direct and resource-efficient. Subsequently, the state space of the video data is reconstructed in S502 through time delay embedding technology, which enhances the system's ability to characterize dynamic changes in the video. Finally, the Lyapunov exponent of the state space is calculated in S503, which is a mathematical tool for measuring the stability of dynamic systems and predicting changes in system behavior. Positive Lyapunov exponents indicate that the system exhibits chaotic characteristics, while negative values ​​indicate stability. Through these analyses, key dynamic events in the video can be accurately identified and marked, providing support for subsequent processing and decision-making.

[0146] S6: extract the critical point of the Lyapunov exponent, and use the video frame corresponding to the critical point as the video key frame.

[0147] Among them, critical points, also known as turning points, are time points when the Lyapunov exponent shows significant changes. These points mark an important change in system behavior, which may shift from stability to instability, or from one dynamic mode to another. Critical points are of special significance in dynamic systems because they represent fundamental changes in system parameters or system states. At these points, the system may exhibit new behavior patterns, or the original stability will be broken. For video content analysis, these critical points correspond to: sudden changes in scenes, sudden occurrences of events, etc. In video analysis, identifying these turning points can help automated systems more effectively index video content and extract key frames, which are particularly critical in narrating video stories or highlighting important events in videos.

[0148] In a possible implementation, S6 specifically includes:

[0149] S601: Perform sliding average on the Lyapunov exponent to smooth the Lyapunov exponent:

[0150]

[0151] in, express t The Lyapunov exponent at time is obtained by smoothing the Lyapunov exponent. express t Time increase iAfter a time unit, , express Lyapunov exponent of the moment;

[0152] It should be noted that the Lyapunov exponent will be affected by noise, and data smoothing can ensure the accuracy of the analysis. The sliding average method is used to reduce the impact of occasional fluctuations on the Lyapunov exponent analysis, making the dynamic system characteristics revealed by the index clearer and more reliable. This method is particularly suitable for situations where the data noise is large or there are rapid changes between data points, and can effectively improve the stability and credibility of the analysis.

[0153] S602: extracting extreme values ​​from the Lyapunov exponents obtained after smoothing by using a findpeaks function, wherein the moment corresponding to the extreme value is the critical point, wherein the extreme value includes a peak value and a valley value.

[0154] Among them, findpeaks is a mathematical function commonly used in signal processing and data analysis, which is used to automatically detect local maxima (peaks) in a given data sequence. Most data analysis and scientific computing software packages such as MATLAB, Python (especially through SciPy or Pandas libraries), R language, etc. provide findpeaks or similar functions.

[0155] S603: Output the video frame at the time corresponding to the critical point as a video key frame.

[0156] Specifically, first, the Lyapunov exponents are smoothed by the sliding average method, which helps to reduce the impact of noise and occasional fluctuations, thereby improving the stability and accuracy of the analysis. Next, the findpeaks function is used to identify extreme points from the smoothed data. These extreme points mark significant changes in system behavior, namely critical points. Finally, the system marks the video frames corresponding to these critical points as key frames and outputs them. These key frames capture the most critical events or transitions in the video, providing key information for subsequent video analysis and decision-making. The entire process ensures that the focus of video analysis is on the most influential dynamic events, greatly improving the preview effect of key content when retrieving back-kitchen videos, and providing users with video segment content at mutation time points.

[0157] S7: Use the video key frame as an index mark of the video segment and output the video segment.

[0158] It should be noted that the video keyframes are used as index marks of the video segments to provide users with previews, and more efficiently extract the required kitchen video data. This allows users to quickly access and view specific video clips containing key activities, thereby improving the access efficiency and practicality of video data. This method is particularly suitable for quickly browsing a large amount of video materials, directly locating the most critical events, saving time and improving work efficiency.

[0159] S8: Display the video key frames and complete the retrieval of the kitchen video.

[0160] It can be understood that the video keyframe represents the main content of the corresponding video segment. When the kitchen manager clicks on the video keyframe, the corresponding video segment will be presented directly, and the video segment can directly skip the video trigger frame and locate to the video keyframe, quickly presenting the video content that the caller wants to obtain.

[0161] In a possible implementation manner, there are multiple video key frames, and the method further includes:

[0162] Receive a click instruction of a video key frame and display the video segment corresponding to the video key frame.

[0163] It should be noted that users can directly access the corresponding specific video segment by clicking on the video keyframe displayed on the interface. This function enhances the user interaction experience and allows users to quickly and directly view the important video content represented by each keyframe without manually searching the entire video recording. This method greatly improves the efficiency of the video surveillance system and the convenience of user operation. It is especially suitable for kitchen scenes where key events need to be quickly reviewed, ensuring the safety of the kitchen environment, and quickly focusing on and holding accountable for irregular behaviors in the kitchen based on this video extraction method.

[0164] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0165] In the present invention, a video trigger frame is determined according to a moving target appearing in a back kitchen monitoring area, and the video trigger frame at the moment when the moving target enters the back kitchen is quickly located. The entire video segment containing the same moving target is extracted by using a constant feature that does not change across frames, and the video segment is regarded as a dynamic system. The Lyapunov exponent of this dynamic system is calculated to select video key frames, identify critical point moments with significant content changes, that is, moments corresponding to important events or dynamic changes in the video, and finally display these video key frames to provide a fast and representative content preview for video retrieval personnel, ensure that all key information is effectively displayed, avoid the problem of easily missing key point information caused by manual video retrieval, and then help video retrieval personnel quickly locate and intercept the required video content. Through these steps, fast, efficient and accurate retrieval of back kitchen video data can be achieved, effectively reducing the workload of retrieval of back kitchen video data, which is suitable for back kitchen scenes that need to quickly review key events, ensure the safety of the back kitchen environment, and quickly pay attention to and hold accountable for irregular behaviors in the back kitchen based on this video extraction method.

[0166] Reference Manual Attached Figure 2 , showing a schematic structural diagram of a system for quickly retrieving kitchen video data in the catering industry provided by the present invention.

[0167] The present invention also provides a fast retrieval system 20 for catering industry back kitchen video data, which is applied to the above-mentioned fast retrieval method for catering industry back kitchen video data, comprising:

[0168] Processor 201.

[0169] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201, a method for quickly retrieving video data of a catering industry kitchen is implemented as in the method embodiment.

[0170] The fast retrieval system 20 for catering industry back kitchen video data provided by the present invention can execute the above-mentioned fast retrieval method for catering industry back kitchen video data and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0171] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0172] In the present invention, a video trigger frame is determined according to a moving target appearing in a back kitchen monitoring area, and the video trigger frame at the moment when the moving target enters the back kitchen is quickly located. The entire video segment containing the same moving target is extracted by using a constant feature that does not change across frames, and the video segment is regarded as a dynamic system. The Lyapunov exponent of this dynamic system is calculated to select video key frames, identify critical point moments with significant content changes, that is, moments corresponding to important events or dynamic changes in the video, and finally display these video key frames to provide a fast and representative content preview for video retrieval personnel, ensure that all key information is effectively displayed, avoid the problem of easily missing key point information caused by manual retrieval of video segments, and then help video retrieval personnel quickly locate and intercept the required video content. Through these steps, fast, efficient and accurate retrieval of back kitchen video data can be achieved, effectively reducing the workload of retrieval of back kitchen video data, which is suitable for back kitchen scenes that need to quickly review key events, ensure the safety of the back kitchen environment, and quickly pay attention to and hold accountable for irregular behaviors in the back kitchen based on this video extraction method.

[0173] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0174] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0175] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0176] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0177] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0178] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0179] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0180] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0181] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0182] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0183] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0184] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0185] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method for quickly retrieving back-kitchen video data of the catering industry as described in the method embodiment is implemented.

[0186] A computer-readable storage medium provided by the present invention can implement the steps and effects of the method for quickly retrieving video data of the catering industry back kitchen in the above method embodiment. To avoid repetition, the present invention will not go into details.

[0187] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0188] In the present invention, a video trigger frame is determined according to a moving target appearing in a back kitchen monitoring area, and the video trigger frame at the moment when the moving target enters the back kitchen is quickly located. The entire video segment containing the same moving target is extracted by using a constant feature that does not change across frames, and the video segment is regarded as a dynamic system. The Lyapunov exponent of this dynamic system is calculated to select video key frames, identify critical point moments with significant content changes, that is, moments corresponding to important events or dynamic changes in the video, and finally display these video key frames to provide a fast and representative content preview for video retrieval personnel, ensure that all key information is effectively displayed, avoid the problem of easily missing key point information caused by manual video retrieval, and then help video retrieval personnel quickly locate and intercept the required video content. Through these steps, fast, efficient and accurate retrieval of back kitchen video data can be achieved, effectively reducing the workload of retrieval of back kitchen video data, which is suitable for back kitchen scenes that need to quickly review key events, ensure the safety of the back kitchen environment, and quickly pay attention to and hold accountable for irregular behaviors in the back kitchen based on this video extraction method.

[0189] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0190] There are a few points to note:

[0191] (1) The drawings of the embodiments of the present invention only involve structures related to the embodiments of the present invention. Other structures may refer to conventional designs.

[0192] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0193] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0194] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for quickly retrieving video data from a catering kitchen, characterized in that: include: S1: Receive the kitchen video retrieval instruction; S2: In response to the kitchen video retrieval instruction, a video trigger frame is determined according to a moving target appearing in the kitchen monitoring area; S3: capturing the constant features of the moving target in the video trigger frame, wherein the constant features include key points based on image gradient extreme points and key point main directions; S4: extracting a video segment from the kitchen video using the constant feature as a start identifier and an end identifier, wherein the video segment includes multiple video frames having the same moving target; S5: Calculate the Lyapunov exponent of the video segment; S6: extracting the critical point of the Lyapunov exponent, and taking the video frame corresponding to the critical point as the video key frame; S7: using the video key frame as an index mark of the video segment, and outputting the video segment; S8: Display the video key frame and complete the retrieval of the kitchen video; Wherein, the S2 specifically includes: S201: Determine the initial kitchen background frame: in, Represents the initial kitchen background frame, Indicates i The grayscale value of the video frame behind the frame, N Indicates the number of kitchen video frames involved in the construction of kitchen background frames; S202: Calculate the pixel difference between the current kitchen video frame and the initial kitchen background frame: in, Indicates time t The pixel difference is Indicates taking the absolute value; S203: Preprocessing the pixel difference including binarization and morphological dilation in sequence, and outputting a moving target represented by morphology: Binarization: Morphological expansion: in, Representing coordinates The binary value at Representing coordinates At the moment t The pixel difference is Representing coordinates The pixel difference threshold at represents the morphological dilation operator, express t The moving target represented by the morphology collected at every moment; S204: Update the initial kitchen video frame and return to step S202: in, represents the learning rate, Indicates the updated initial kitchen background frame; S205: Outputting the video frame having the moving target as the video trigger frame; The S3 specifically includes: S301: Constructing a scale space of the video trigger frame at multiple scales: in, express The video trigger frame at this scale is Scale space under scale, represents the Gaussian kernel, represents the video trigger frame, represents the pixel coordinates of the video trigger frame, Represents the convolution operation; S302: Acquire the maximum value point and the minimum value point of each pixel point in each scale space within a preset neighborhood, and use the maximum value point and the minimum value point as the key point; S303: Performing Taylor series expansion on the key points to relocate the key points, wherein the key points obtained by relocation are coincident points of the gradient optimum point and the curvature optimum point: in, Indicates key points p The differential scale space value at describes the key point p A function of the change in pixel value of surrounding pixels, Indicates key points The difference scale space value at , Indicates the key point To the real key point The offset of the key point The gradient value at Indicates key points The Hessian matrix at the key point The curvature of the scale space at that point. The subscript T indicates transposition. S304: Determine the main direction of the key point according to the distribution of the relocated key point in the direction of the local image gradient: in, Represents the coordinates of the key points obtained by relocation The main direction of the key points, represents the gradient direction of each pixel point in the preset neighborhood of the key point, Represents the inverse tangent function, sin represents the calculation of the sine value, and cos represents the calculation of the cosine value. represents the summation symbol; The S5 specifically includes: S501: extracting video frame intensity features of each video frame in the video segment, wherein the video frame intensity features form a state vector in chronological order; S502: Perform time delay embedding on the state vector to reconstruct the state space of the video segment: in, express t The state vector of the video frame at time instant, Indicates time delay, m represents the embedding dimension, represents the state space; S503: Calculate the Lyapunov index of the state space, that is, the Lyapunov index of the video segment: in, represents the Lyapunov exponent, Indicates infinity, and They represent the initial state space and t In the state space at a certain moment, log represents the natural logarithmic function and lim represents the limit; The S6 specifically includes: S601: Perform sliding average on the Lyapunov exponent to smooth the Lyapunov exponent. in, express t The Lyapunov exponent at the moment is obtained by smoothing the Lyapunov exponent. express t Time increase i After a time unit, , express Lyapunov exponent of the moment; S602: extracting extreme values ​​from the Lyapunov exponents obtained after smoothing by using a findpeaks function, wherein the moment corresponding to the extreme value is the critical point, wherein the extreme value includes a peak value and a valley value; S603: Output the video frame at the time corresponding to the critical point as the video key frame.

2. The method for quickly retrieving video data from the back kitchen of the catering industry according to claim 1 is characterized in that: The kitchen video retrieval instructions include voice instructions, touch screen instructions, API instructions and GUI button instructions.

3. The method for quickly retrieving video data from the back kitchen of a catering industry according to claim 1 is characterized in that: The S4 specifically includes: S401: Retrieving the constant feature in each video frame in the kitchen video; S402: When the constant feature exists in the video frame, retain the video frame; otherwise, return to S401 until the video frames in the kitchen video are retrieved.

4. The method for quickly retrieving video data from the back kitchen of a catering industry according to claim 1 is characterized in that: There are multiple video key frames, and the method further includes: A click instruction of the video key frame is received, and a video segment corresponding to the video key frame is displayed.

5. A fast retrieval system for kitchen video data in the catering industry, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, a method for quickly retrieving catering industry back-kitchen video data as described in any one of claims 1 to 4 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a method for quickly retrieving catering industry back-kitchen video data as described in any one of claims 1 to 4 is implemented.

Images