A business state analysis method and system based on computer vision and signal processing

Abstract

The application discloses a kind of based on computer vision and signal processing's business state analysis method and system, it is related to artificial intelligence technical field, video stream is collected by camera, and using pre-training visual model identifies supervision object state, generates state sequence;Further discrete state sequence is converted into numerical multidimensional time series signal, it is converted from logic judgment to signal processing domain;Then frequency domain analysis is carried out to signal, including generating the scene behavior mode fingerprint map of frequency time-varying characteristics and the scene operation rhythm map reflecting long-period frequency structure;Finally, pattern recognition and business state judgment are carried out based on atlas.The application expands analysis dimension from isolated time point to continuous time period, constructs unified analysis foundation, can simultaneously capture instantaneous abnormality and system law, gets rid of the dependence on pre-defined rule and hard-coded threshold, significantly improves state perception depth and the autonomous ability of abnormality discovery.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a business status analysis method and system based on computer vision and signal processing. Background Technology

[0002] With the development of artificial intelligence technology, using computer vision (CV) for automated monitoring in scenarios such as production, security, and transportation has become a mainstream trend. The typical technical process involves: acquiring video streams through cameras, and using AI visual recognition algorithms (such as object detection, image classification, and behavior recognition) to transform unstructured video data into structured, computer-understandable "factual" information, such as identifying "the conveyor belt is running," "there are 5 workers in the workshop," and "the material box is empty." This technical solution is based on a "fact recognition + rule-based judgment" model, and its effectiveness depends heavily on a large number of rigid business rules and fixed thresholds. Specifically, a system implementing this solution includes the following modules:

[0003] Fact Acquisition Layer: Images of the production workshop are captured via cameras. Target detection algorithms are used to identify monitored objects, such as "conveyor belt" and "workers," and image classification algorithms are then used to determine their status, such as "conveyor belt: running / stopped" and "workers: on duty / off duty."

[0004] Rule Judgment Layer: The system has a built-in rule engine that pre-stores business rules. These rules typically consist of logical judgments (AND, OR, NOT) and time thresholds. For example: "Rule 1 (Leave of Post Exceeding Time Violation): IF (Production Status = In Progress) AND (Employee = Leaving Post) AND (Duration > 180 seconds) THEN Determined as 'Leave of Post Exceeding Time Violation'...".

[0005] Business conclusion layer: The rule engine matches the status data passed from the fact acquisition layer in real time. Once a rule is triggered, the system outputs the corresponding business conclusion, such as recording violations, issuing alarms, and calculating job duration.

[0006] The implementation process of the above-mentioned "fact recognition + rule judgment" scheme is intuitive and can effectively handle deterministic problems, but its inherent technical paradigm leads to the following significant drawbacks:

[0007] (1) Limitations in the analysis dimensions: The essence of the above scheme is to make logical combinations of state snapshots at discrete time points, which completely ignores the dynamic information contained in the state change process itself. For example, the above system can determine whether the device is "on" or "off", but it cannot analyze whether the device's "start-stop mode" is periodic, sudden, or random.

[0008] (2) Fixed detection capability: Because the rules are predefined by humans, the system can only identify "known" violations. The system will not be able to identify new violations that circumvent the rules by changing behavioral patterns. That is, the existing technology lacks the ability to learn from data and discover new patterns.

[0009] (3) Insufficient cross-scenario adaptability of threshold setting: Existing technologies attempt to constrain a dynamic and complex real-world behavior through static numbers or fixed numerical ranges, such as defining rules like "180-second absence violation" or "5 repeated operation warnings". However, these empirical values ​​often need to be manually adjusted in different scenarios and working conditions, otherwise it will lead to high false alarm rate or high false alarm rate.

[0010] (4) Lack of correlation analysis: The rule judgments provided by existing technologies are usually based on a simple combination of a single or a few object states. For the complex correlation patterns hidden between multiple object states, rule engines have difficulty analyzing, extracting and describing them.

[0011] (5) Lack of scalability and deep historical analysis capabilities on a time scale. Existing technologies are essentially event-driven and real-time response-oriented. Their design goal is to make "immediate judgments" rather than "long-term insights." The analysis window is usually limited to the second, minute, or hour level, and cannot perform long-term analysis and extended learning on a monthly or yearly time scale.

[0012] Furthermore, the system's output is typically only discrete "alarm logs" rather than continuous process data available for further analysis, leading to a loss of data value. Valuable historical process information is essentially "evaporated," and the data fails to become an asset with added value, instead becoming merely an immediate consumable.

[0013] Therefore, there is an urgent need to provide a business status analysis method and system based on computer vision and signal processing to solve the above-mentioned technical problems. Summary of the Invention

[0014] To address the problems existing in current technologies, this invention provides a business status analysis method and system based on computer vision and signal processing. It elevates the analytical perspective from isolated, discrete state snapshots to time-frequency domain pattern analysis of continuous, dynamic behavioral processes. This enables the analysis to reveal deeper information such as rhythms, cycles, and instantaneous abrupt changes in state changes, achieving in-depth mining of historical data and enhancing anomaly detection capabilities. Furthermore, it improves detection sensitivity and effectively reduces manual maintenance costs.

[0015] The technical solution of this invention is implemented as follows:

[0016] A business status analysis method based on computer vision and signal processing includes the following steps:

[0017] S1. Data Acquisition: Images are acquired using image acquisition equipment in a designated environment, and a video stream is formed from multiple images; the designated environment includes several monitored objects; in this application, "several" refers to one, two, or more. Specifically, the image acquisition equipment consists of several high-definition (surveillance) cameras deployed in the designated environment.

[0018] S2, Object State Recognition: The video stream is input into a pre-trained visual recognition model in real time to identify the real-time state of the monitored object and generate a state sequence;

[0019] S3. State sequence signaling: Generate a numericalized multidimensional time series signal based on the state sequence;

[0020] Existing technologies typically use discrete states as direct input to rule engines, performing isolated logical judgments at each time point. This invention, however, transforms and maps state sequences into numerical, unified multi-dimensional time-series signals, introducing subsequent analysis from the logical domain to the mathematical domain of signal processing, thus laying a unified mathematical foundation for handling complex dynamic processes of mixed types.

[0021] S4. Signal Analysis and Transformation: Perform frequency domain analysis and transformation on the multidimensional time series signal; the frequency domain analysis and transformation includes time-frequency analysis and global spectrum analysis.

[0022] The time-frequency analysis involves performing a two-dimensional time-frequency transformation on the multidimensional time series signal to generate a scene behavior pattern fingerprint map representing the time-frequency changes, which is used to characterize the frequency component change pattern of the signal over time.

[0023] The global spectrum analysis involves generating a scene operation rhythm diagram with a periodic overall frequency distribution for the multidimensional time series signal collected within a specified long time window, in order to characterize the overall frequency distribution structure of the signal within the time window.

[0024] Existing technologies focus on the state of a target at a specific moment, while the solution proposed in this application focuses on the change pattern of the target over a period of time. Specifically, frequency diagrams or spectrograms obtained through frequency domain analysis can more intuitively and clearly compare and identify the degree of change in state. This allows for the precise capture of every instantaneous change and can also be used to observe systematic patterns and trends over long periods.

[0025] S5. Pattern Analysis and Business Status Judgment: Based on the scenario behavior pattern fingerprint or the scenario operation rhythm diagram, the business status is analyzed and obtained.

[0026] S6. Business Status Response and Output.

[0027] This invention transforms discrete state sequences into multidimensional time-series signals and innovatively introduces frequency domain analysis to construct a "scene behavior pattern fingerprint" and a "scene operation rhythm diagram," achieving a fundamental shift in analytical dimensions from isolated point-in-time judgment to continuous time-span pattern recognition. This method can simultaneously and accurately capture instantaneous anomalies and observe long-term patterns, eliminating complete reliance on predefined rules. It provides a unified and autonomous mathematical analysis foundation for judging the business state of complex dynamic processes, significantly improving the depth of state perception and the initiative in anomaly detection.

[0028] As a further optimization of the above scheme, in S2, different state sequences are generated in parallel for different monitored objects; the state sequence includes multiple discrete states or multiple consecutive states, as well as timestamps corresponding to the states.

[0029] As a further optimization of the above scheme, the visual recognition model includes an object detection model and a state recognition model; the visual recognition model can be a pre-trained model or a combination of multiple pre-trained models; the types of pre-trained models are distinguished by their own recognition direction or optimization direction, etc.

[0030] The target detection model is used to identify and locate the monitored object in the image; the state recognition model is used to identify the real-time state based on the located monitored object.

[0031] Specifically, the objects of regulation include industrial production equipment, safety doors, and designated lights.

[0032] Correspondingly, the status of industrial production equipment includes running, standby, and fault; the status of safety doors includes open and closed; and the status of indicator lights includes different colors, brightness, flashing, constant light, and no light. In addition, dynamic attributes such as speed, threshold, or numerical range can also be used as status.

[0033] As a further optimization of the above scheme, the state in the state sequence is in the form of text; in step S3, the multidimensional time series signal is generated by mapping the state of the text to numerical values.

[0034] The mapping relationship between the text and the numerical value includes binary mapping, multi-valued mapping, and continuous value mapping.

[0035] For regulated objects that have only two states, a binary mapping is used, such as the state of a safety door being "open=1, closed=0"; for regulated objects that have more than two states, a multi-value mapping is used, such as the operating state of equipment being "stop=0, standby=1, running=2"; for regulated objects with continuous output state changes, a continuous value mapping is used, such as the instantaneous speed of a product on an assembly line at a specified point.

[0036] In terms of monitoring performance, binary mapping represents a square wave signal, multi-value mapping represents a stepped wave signal, and continuous value mapping represents a continuous curve signal.

[0037] As a further optimization of the above scheme, the time-frequency analysis is to use short-time Fourier transform or wavelet transform to generate a time spectrum or scale map based on the multidimensional time series signal, which is the scene behavior pattern fingerprint map.

[0038] The global spectrum analysis is performed by using Fourier transform to generate a one-dimensional spectrum diagram based on the multi-dimensional time series signal, which is the scene operation rhythm diagram.

[0039] It should be emphasized that the Fourier Transform (FT), Short-Time Fourier Transform (STFT), and Wavelet Transform (WT) described above are typical examples used to illustrate the core ideas of this invention, but the application of this invention should not be limited to these. Those skilled in the art will understand that any signal processing technique capable of revealing hidden dynamic or periodic patterns in time-series signals can be applied within the framework of this invention. These techniques may include, but are not limited to: Hilbert-Huang Transform (HHT) for adaptive signal decomposition, Wigner-Willi distribution (WVD) for high-resolution time-frequency analysis, cross-spectral analysis for analyzing correlations between multiple signals, and other advanced analytical methods based on signal statistics, entropy theory, or state-space models.

[0040] As a further optimization of the above solution, the business status analysis based on the scenario behavior pattern fingerprint is as follows:

[0041] S5A-1. Collect standard business process data to represent normal business status; generate a normal scenario behavior pattern fingerprint based on the business process data.

[0042] S5A-2: Train an anomaly detection model based on the fingerprint map of normal scene behavior patterns;

[0043] S5A-3. Input the real-time obtained scene behavior pattern fingerprint into the anomaly detection model to determine whether there is an abnormal business state and output the result.

[0044] The above business status analysis employs an unsupervised learning method for real-time, dynamic anomaly detection. In real-time monitoring, the reconstruction error of the newly generated time-spectrum / scale map is calculated. If the reconstruction error is significantly higher than the normal level, it is judged as "abnormal" or "violation".

[0045] Existing technologies require pre-defined rules for judging abnormal and normal states. Their expansion relies on human input and definition, lacking self-learning and anomaly detection capabilities. Some rules are limited to specific numerical values ​​and ranges. This application's technical solution offers a novel approach. Its core is enabling the system to autonomously summarize behavioral pattern benchmarks from normal business data and sensitively detect various statistical deviations in real-time data, such as energy mutations, frequency component changes, or pattern rhythm disturbances. This "benchmark-deviation" detection mechanism naturally possesses the ability to cover a wide range of anomaly types without relying entirely on pre-enumeration and definition of anomalies, exhibiting stronger generalization ability, adaptability, and robustness.

[0046] As a further optimization of the above solution, the business status analysis based on the scenario operation rhythm diagram includes periodicity identification, rhythm comparison and baseline deviation analysis;

[0047] The periodicity identification involves filtering out spectral peaks with amplitudes exceeding a preset change amplitude threshold from the spectrum, and recording the frequency positions corresponding to the spectral peaks as candidate periodic components. Hidden repetitive events can be discovered through these candidate periodic components to uncover potential abnormal behavior patterns.

[0048] The rhythm comparison involves comparing the spectrum of different preset time periods to quantitatively assess the similarity and differences of business processes in different time periods.

[0049] The baseline deviation analysis involves collecting standard business process data to represent a normal business state; generating a normal scenario operation rhythm diagram based on the business process data, with the corresponding spectrum recorded as the spectrum baseline; and comparing the deviation between the real-time spectrum and the spectrum baseline to assess whether there are any abnormalities in the overall business process.

[0050] The above business status analysis is mainly used for offline, periodic global anomaly detection.

[0051] As a further optimization of the above solution, the response includes recording the time of violation or abnormal event, issuing alarms, and periodically generating business process analysis reports.

[0052] This invention also provides a business status analysis system based on computer vision and signal processing, which applies the business status analysis method based on computer vision and signal processing described above; including:

[0053] The image acquisition module is used to execute step S1; the state recognition module is used to execute step S2; the signal conversion module is used to execute step S3; the signal analysis module is used to execute step S4; the pattern judgment module is used to execute step S5; and the output processing module is used to execute step S6.

[0054] Further optimizations to the above solutions also include model libraries and databases;

[0055] The model library stores several visual recognition models, including various pre-trained visual recognition models for different industries or scenarios (such as industrial, security, and retail); it may also include various anomaly detection models trained from normal business data. The system can dynamically load the corresponding models according to specific application scenarios. The database is used to store and retrieve the multi-dimensional time series signals, the scenario behavior pattern fingerprint, the scenario operation rhythm diagram, and business status analysis results.

[0056] Compared with the prior art, the present invention achieves the following beneficial effects:

[0057] (1) This invention transforms discrete state sequences into multidimensional time-series signals and, by introducing frequency domain analysis and spectral analysis, constructs a "scene behavior pattern fingerprint" and a "scene operation rhythm diagram," achieving a fundamental shift in the analysis dimension from isolated point-in-time judgment to continuous time-period pattern recognition. This method can simultaneously achieve accurate capture of instantaneous anomalies and observation of long-term patterns, eliminating complete dependence on predefined rules and providing a unified and autonomous mathematical analysis basis for judging the business status of complex dynamic processes, significantly improving the depth of state perception and the initiative in anomaly detection.

[0058] (2) The present invention adopts the unsupervised learning paradigm of "defining normal and detecting abnormal", which is not limited by the known rule base. It can discover new violations or early signs of equipment failure that have never been seen before. That is, it can capture complex, instantaneous and hidden abnormal behaviors that are difficult to describe by traditional rules, providing forward-looking insights and decision-making basis for risk prevention and process optimization.

[0059] (3) The state recognition method of the present invention also gets rid of the dependence on a large number of hard-coded rules and fragile thresholds, which greatly reduces the maintenance cost and complexity of the system. In addition, when the business process changes slightly, as long as the overall behavior pattern (business process) does not change drastically, the system can still work normally, reducing the impact of misjudgment of "abnormal situation" on process interruption.

[0060] (4) This invention transforms discrete, instantaneous states into storable multidimensional time series signals, supporting the overall analysis of data over long time scales such as months and years, in order to uncover ultra-low frequency periodic patterns and macro trends. It effectively overcomes the limitation of traditional behavioral analysis methods that can only perform short-term analysis, and also makes data a valuable asset that continuously increases in value.

[0061] (5) The present invention transforms the definition of state into a visual scene behavior pattern fingerprint map and scene operation rhythm map, which can provide direct input for the system to identify abnormal states, and can also provide it to managers so that they can visually locate and evaluate the time period and degree of change of abnormal time, providing rich and quantifiable in-depth information for subsequent event tracing and management decisions. Attached Figure Description

[0062] Figure 1 This is a flowchart illustrating a business status analysis method based on computer vision and signal processing provided in an embodiment of the present invention;

[0063] Figure 2 This is a data flow diagram of a business status analysis system based on computer vision and signal processing provided in an embodiment of the present invention;

[0064] Figure 3 These are signal waveforms of different monitored objects provided in embodiments of the present invention;

[0065] Figure 4 This is a scene behavior pattern fingerprint map corresponding to the normal business state provided in the embodiments of the present invention;

[0066] Figure 5 This is a scene behavior pattern fingerprint map corresponding to abnormal business states provided in the embodiments of the present invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0068] This embodiment provides a business status analysis system based on computer vision and signal processing, including:

[0069] The system includes an image acquisition module, a status recognition module, a signal conversion module, a signal analysis module, a pattern judgment module, an output processing module, as well as a model library and a database.

[0070] The model library stores visual recognition models and anomaly detection models, among others.

[0071] Specifically, the above system applies a business status analysis method based on computer vision and signal processing, such as... Figure 1 , Figure 2 As shown, it includes the following steps:

[0072] S1. Data Acquisition: The image acquisition module uses an image acquisition device to acquire images in a designated environment and forms a video stream from multiple images. The designated environment includes several monitored objects; in this application, "several" refers to one, two, or more. Specifically, the image acquisition device is several high-definition (surveillance) cameras deployed in the designated environment. Monitored objects include production equipment, security doors, and indicator lights. The image acquisition module sends the video stream to the status recognition module.

[0073] S2. Object Status Recognition: The status recognition module inputs the video stream into a pre-trained visual recognition model in real time to identify the real-time status of the monitored object and generate a status sequence. Different status sequences are generated in parallel for different monitored objects; each status sequence contains multiple discrete or consecutive states, along with a timestamp corresponding to each state. For example, the status of production equipment includes running, standby, and fault states; the status of safety doors includes open and closed states; and the status of indicator lights includes different colors, brightness, flashing, constant light, and no light. Furthermore, dynamic attributes such as speed, threshold values, or numerical ranges can also be used as status parameters.

[0074] Visual recognition models include object detection models and state recognition models. Object detection models are used to identify and locate monitored objects in images; state recognition models are used to identify the real-time state of the located monitored objects.

[0075] The state recognition module sends the state sequence to the signal conversion module.

[0076] In this embodiment, the states in the state sequence are in the form of text.

[0077] S3, State Sequence Signalization: The signal conversion module generates a numerical multidimensional time series signal based on the state sequence; by mapping the state of the text to numerical values, a multidimensional time series signal is generated; the mapping relationship between text and numerical values ​​includes binary mapping, multi-value mapping and continuous value mapping.

[0078] For regulated objects that have only two states, a binary mapping is used, such as the state of a safety door being "open=1, closed=0"; for regulated objects that have more than two states, a multi-value mapping is used, such as the operating state of production equipment being "stop=0, standby=1, running=2"; for regulated objects with continuous output state changes, a continuous value mapping is used, such as the instantaneous speed of a product on an assembly line at a specified point.

[0079] In terms of monitoring performance, binary mapping represents a square wave signal, multi-value mapping represents a stepped wave signal, and continuous value mapping represents a continuous curve signal. For example... Figure 3 It provides signal waveforms for the status of three different monitored objects (safety gate, conveyor belt, and robotic arm).

[0080] The signal conversion module stores multidimensional time series signals into a database.

[0081] S4. Signal Analysis and Transformation: The signal analysis module reads multi-dimensional time-series signals from the database and performs frequency domain analysis and transformation; frequency domain analysis and transformation includes time-frequency analysis and global spectrum analysis.

[0082] Time-frequency analysis involves performing a two-dimensional time-frequency transform on a multi-dimensional time-series signal to generate a scene behavior pattern fingerprint, which characterizes the variation pattern of the signal's frequency components over time. In this embodiment, time-frequency analysis employs wavelet transform to generate a scaling map, i.e., the scene behavior pattern fingerprint, based on the multi-dimensional time-series signal.

[0083] Global spectrum analysis involves generating a scene operation rhythm diagram with a periodic overall frequency distribution from multidimensional time-series signals acquired within a specified long-term window. This diagram characterizes the overall frequency distribution structure of the signal within the time window. In this embodiment, global spectrum analysis employs Fourier transform to generate a one-dimensional spectrum diagram, i.e., the scene operation rhythm diagram, based on the multidimensional time-series signal. The long-term window can be set to one week or one month.

[0084] Frequency maps or spectrum diagrams obtained through frequency domain analysis can more intuitively and clearly compare and identify the degree of change in the state. They can accurately capture every instantaneous change and can also be used to observe the systematic patterns and trends over long periods.

[0085] S5. Pattern Analysis and Business Status Judgment: The pattern judgment module receives the scene behavior pattern fingerprint or scene operation rhythm diagram from the signal analysis module and analyzes it to obtain the business status.

[0086] In this embodiment, the business status analysis based on the scene behavior pattern fingerprint is as follows:

[0087] S5A-1: Collect standard business process data to represent normal business status; generate a normal scenario behavior pattern fingerprint based on the business process data.

[0088] S5A-2: Train an anomaly detection model based on the fingerprint map of normal scene behavior patterns;

[0089] S5A-3: Input the real-time obtained scene behavior pattern fingerprint into the anomaly detection model to determine whether there is an abnormal business state and output the judgment result.

[0090] The above business status analysis employs an unsupervised learning method for real-time, dynamic anomaly detection. In real-time monitoring, the reconstruction error of the newly generated time-spectrum / scale map is calculated. If the reconstruction error is significantly higher than the normal level, it is judged as "abnormal" or "violation".

[0091] like Figure 4 As shown, it displays a scale map (scenario behavior pattern fingerprint map, or production line behavior pattern fingerprint map) after mixing and changing multiple state signals. The figure clearly shows the high-frequency transients caused by the linkage of multiple signals at the start and end of production, i.e., the start-up mode, as well as the low-frequency characteristic band of the performance during steady-state production.

[0092] When an abnormal state is detected, an "abnormal fingerprint" is formed on the scale map. For example... Figure 5 As shown, when the conveyor belt speed fluctuates abnormally and the safety door opens momentarily, a complex and unexpected high-frequency energy cluster is formed on the scale map, which is outside the main frequency band and is composed of two abnormal trajectories (corresponding to the speed fluctuation and the door opening event, respectively). This feature can be easily captured by the pattern judgment module.

[0093] In this embodiment, the business status analysis based on the scene operation rhythm graph includes periodic identification, rhythm comparison, and baseline deviation analysis; it is mainly used for offline, periodic global anomaly detection.

[0094] Periodicity identification involves filtering out spectral peaks with amplitudes exceeding a preset change amplitude threshold from the spectrum and recording the frequency positions corresponding to these peaks as candidate periodic components. Hidden repetitive events can be discovered through these candidate periodic components to uncover potential abnormal behavior patterns.

[0095] Rhythm comparison involves comparing the spectrum of different preset time periods, such as comparing the spectrum of business processes monitored and collected during day shifts and night shifts, in order to quantitatively assess the similarity and differences of business processes in different time periods.

[0096] Baseline deviation analysis involves collecting standard (i.e. standard operating conditions) business process data to represent normal business status; generating a normal scenario operation rhythm diagram based on the business process data, with the corresponding spectrum recorded as the spectrum baseline; and comparing the deviation between the real-time spectrum and the spectrum baseline to assess whether there are any anomalies in the overall business process.

[0097] S6. Business Status Response and Output. In this embodiment, the response includes recording the violation time or abnormal event (stored in the value database), issuing alarms, and periodically generating business process analysis reports.

[0098] This embodiment also provides a deep spectrum analysis process for the scene operation rhythm diagram. Specifically, the scene / environment is an automated parcel sorting center in a large logistics hub. The monitored objects include "area occupancy status", "buffer zone congestion status", and "forklift operation status".

[0099] Therefore, the following signals were collected respectively:

[0100] A semantic segmentation model is used to analyze the panoramic image of the temporary storage area, collecting the area availability signal S_clear(t) (square wave). Specifically, when there are no trays, debris, or people occupying the area, the output is 1; otherwise, the output is 0. Semantic segmentation classifies each pixel in the image according to its semantic category, thereby achieving pixel-level semantic understanding.

[0101] Based on the target detection and tracking model, the number of packages within the buffer is counted and weighted according to their distribution density. The buffer congestion signal S_congestion(t) is collected, where the buffer signal is a continuous value and needs to be normalized to [0,1]. S_congestion(t) reflects the task backlog and the rationality of its spatial distribution.

[0102] The forklift's motion trajectory is obtained based on a multi-target tracking algorithm, and the sliding window variance of its instantaneous acceleration and angular velocity is calculated to obtain the forklift smoothness signal S_fluency(t), which is then normalized. The signal value is closer to 1 if the trajectory is smooth and the forklift operates efficiently; conversely, the signal value is closer to 0 if the forklift frequently starts and stops and hesitates in path selection.

[0103] The above signals are combined into a single signal EES(t) by weighting, i.e.:

[0104] ;

[0105] in, These represent the weight values ​​of the three signals. This embodiment uses a weighted geometric mean for signal fusion. In other embodiments, a weighted arithmetic mean or other nonlinear fusion functions may be used to adapt to different processing requirements for zero-value signals in different business scenarios.

[0106] Performing a Fourier transform on the EES(t) signal over a continuous week yields a spectrum of global performance, also known as a scene operation rhythm diagram. In this embodiment, service status analysis based on the scene operation rhythm diagram includes periodic identification, rhythm comparison, and baseline deviation analysis.

[0107] Periodicity identification: A significant anomalous peak was found in the spectrogram at f = 0.5 h⁻¹ (meaning one cycle is 2 hours). This peak corresponds to regular, brief energy decay events. Further data backtracking revealed that this phenomenon strictly corresponds to the fixed collective rest / recharging strategy of forklift operators. This analysis reveals a systemic process disruption caused by a fixed work rhythm.

[0108] Rhythm Comparison: The EES(t) spectra of the same forklift crew during the midday and morning peak periods were compared. The results showed that the DC component of the midday spectrum decreased by approximately 20% compared to the morning peak, indicating a significant reduction in the overall system throughput efficiency. Simultaneously, the energy in the 0.1–0.3 h⁻¹ frequency band (corresponding to a period of 3–10 minutes) significantly increased, reflecting frequent short-cycle fluctuations in the sorting and handling rhythm. This phenomenon suggests that the system may experience decreased operational rhythm stability and impaired overall operational smoothness during midday due to buffer scheduling lag, slow equipment response, or increased path conflicts. Furthermore, this invention can further diagnose the intrinsic dynamic correlations between multiple sub-signals, such as S_fluency(t) and S_congestion(t), through cross-spectral analysis, for example, identifying hysteresis response patterns in operations.

[0109] Baseline deviation analysis: Comparing the global EES(t) spectrum before and after layout optimization. The spectrum before optimization is denoted as the spectral baseline. Experimental data show that the optimized spectrum exhibits a 15% increase in daily peak power, a 30% narrowing of the peak half-width at half-maximum (FWHM), and an average decrease of 40% in the high-frequency noise floor (>1 h⁻¹). These results indicate that the new site layout effectively enhances the system's daily operating rhythm and significantly improves the stability of forklift operation.

[0110] Based on the disclosure and teachings of the foregoing specification, those skilled in the art can make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and some modifications and changes to the present invention should also fall within the protection scope of the claims of the present invention. Furthermore, although some specific terms are used in this specification, these terms are only for convenience of explanation and do not constitute any limitation on the present invention.

Claims

1. A service status analysis method based on computer vision and signal processing, characterized in that, Includes the following steps: S1. Data Acquisition: Images are acquired in a designated environment using image acquisition equipment, and a video stream is formed from multiple images; the designated environment includes several monitored objects; S2, Object State Recognition: The video stream is input into a pre-trained visual recognition model in real time to identify the real-time state of the monitored object and generate a state sequence; S3. State sequence signaling: Generate a numericalized multidimensional time series signal based on the state sequence; S4. Signal Analysis and Transformation: Perform frequency domain analysis and transformation on the multidimensional time series signal; the frequency domain analysis and transformation includes time-frequency analysis and global spectrum analysis. The time-frequency analysis involves performing a two-dimensional time-frequency transformation on the multidimensional time series signal using short-time Fourier transform or wavelet transform to generate a time spectrum or scale map, which serves as a fingerprint map of scene behavior patterns representing time-frequency changes. The global spectrum analysis involves using Fourier transform to generate a one-dimensional spectrum diagram from the multi-dimensional time series signal collected within a specified long time window, which serves as a scene operation rhythm diagram with a periodic overall frequency distribution. S5. Pattern Analysis and Business Status Judgment: Based on the scenario behavior pattern fingerprint or the scenario operation rhythm diagram, the business status is analyzed and obtained. The business status analysis based on the scenario's operating rhythm diagram includes periodic identification, rhythm comparison, and baseline deviation analysis. The periodicity identification is based on a preset change amplitude threshold, filtering out spectral peaks whose amplitude exceeds the change amplitude threshold from the spectrum, and recording the frequency position corresponding to the spectral peak as a candidate periodic component. The rhythm comparison involves comparing the spectrum of different preset time periods to quantitatively assess the similarity and differences of business processes in different time periods. The baseline deviation analysis involves collecting standard business process data to represent a normal business state; generating a normal scenario operation rhythm diagram based on the business process data, with the corresponding spectrum recorded as the spectrum baseline; and comparing the deviation between the real-time spectrum and the spectrum baseline to assess whether there are any abnormalities in the overall business process. S6. Business Status Response and Output.

2. The service status analysis method based on computer vision and signal processing according to claim 1, characterized in that, In step S2, different state sequences are generated in parallel for different monitored objects; the state sequence includes multiple discrete states or multiple consecutive states, and timestamps corresponding to the states.

3. The service status analysis method based on computer vision and signal processing according to claim 1, characterized in that, The visual recognition model includes an object detection model and a state recognition model; The target detection model is used to identify and locate the monitored object in the image; the state recognition model is used to identify the real-time state based on the located monitored object.

4. The service status analysis method based on computer vision and signal processing according to claim 1, characterized in that, The states in the state sequence are in the form of text; in step S3, the multidimensional time series signal is generated by mapping the states of the text to numerical values. The mapping relationship between the text and the numerical value includes binary mapping, multi-valued mapping, and continuous value mapping.

5. The service status analysis method based on computer vision and signal processing according to claim 1, characterized in that, The business status analysis based on the aforementioned scenario behavior pattern fingerprint is as follows: S5A-1. Collect standard business process data to represent normal business status; generate a normal scenario behavior pattern fingerprint based on the business process data. S5A-2: Train an anomaly detection model based on the fingerprint map of normal scene behavior patterns; S5A-3. Input the real-time obtained scene behavior pattern fingerprint into the anomaly detection model to determine whether there is an abnormal business state and output the result.

6. The service status analysis method based on computer vision and signal processing according to claim 1, characterized in that, The response includes recording the time of violation or abnormal event, issuing alarms, and periodically generating business process analysis reports.

7. A business status analysis system based on computer vision and signal processing, employing a business status analysis method based on computer vision and signal processing as described in any one of claims 1 to 6; characterized in that, include: The image acquisition module is used to perform step S1; The status recognition module is used to execute step S2; The signal conversion module is used to perform step S3; The signal analysis module is used to execute step S4; the mode determination module is used to execute step S5. The output processing module is used to execute step S6.

8. A business status analysis system based on computer vision and signal processing according to claim 7, characterized in that, It also includes a model library and a database; The model library stores several of the visual recognition models; the database is used to store and retrieve the multidimensional time series signals, the scene behavior pattern fingerprint, the scene operation rhythm diagram, and the business status analysis results.

Images