A monitoring and anti-theft system for a vending machine

Abstract

The present invention discloses a monitoring and anti-theft system for a vending machine, comprising: a vibration monitoring module, configured to monitor the vibration energy of a vending machine body included in the vending machine, and generate a video acquisition instruction and send it to a video acquisition module when it is determined that the vibration energy is greater than a vibration energy threshold; the video acquisition module, configured to acquire a monitoring video within a preset range of the vending machine body and perform preprocessing to obtain a target video when receiving the video acquisition instruction, and send the target video to a transmission module; the transmission module, configured to send the target video to an alarm platform. Through the monitoring and anti-theft system for a vending machine provided by the present invention, it is possible to effectively collect a clear monitoring video with extremely low jitter degree and send it to the alarm platform when the vending machine vibrates with too strong energy due to being damaged and impacted, which helps the staff quickly determine the theft situation of the damaged vending machine and lock down the lawbreakers.

Description

Technical Field

[0001] The present invention relates to the technical field of vending machines, and particularly to a monitoring and anti-theft system for a vending machine. Background Art

[0002] With the progress of science and technology, the entire service industry shows a trend of automation. The unmanned vending machine is one of the forms. It is not restricted by time and location, can also save labor, and is convenient for both the buyer and the seller. Therefore, it is gradually welcomed by people. However, due to the automation feature of the vending machine, many merchants do not invest manpower to supervise it, giving lawbreakers the opportunity to damage the vending machine to steal goods or the money inside the vending machine. Since there may not be surveillance cameras at the locations where vending machines are installed, many vending machines on the market adopt the method of installing built-in cameras to monitor the images near the vending machine. However, when facing violent damage, the camera will vibrate due to the vibration of the vending machine body, and it is impossible to obtain a clear picture, resulting in the staff being unable to see the situation when the vending machine is stolen, and thus it is difficult to trace the lawbreakers. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems in the above technologies to some extent. For this purpose, the object of the present invention is to propose a monitoring and anti-theft system for a vending machine, aiming to provide a vending machine monitoring system that can collect the monitoring video near the vending machine when the vibration of the vending machine is too strong, and at the same time, by combining real-time motion anti-shake and post-video anti-shake, ensure that the monitoring video can still maintain a high clarity when the vending machine vibrates.

[0004] To achieve the above object, an embodiment of the present invention proposes a monitoring and anti-theft system for a vending machine, including:

[0005] A vibration monitoring module, configured to monitor the vibration energy of the vending machine body included in the vending machine, and generate a video acquisition instruction when it is determined that the vibration energy is greater than the vibration energy threshold;

[0006] A video acquisition module, configured to collect the monitoring video within a preset range of the vending machine body and perform preprocessing when receiving the video acquisition instruction sent by the vibration monitoring module to obtain a target video, and send the target video to the alarm platform through a transmission module.

[0007] Preferably, the vibration monitoring module includes:

[0008] A vibration sensor, configured to detect the mechanical vibration of the vending machine body and convert the mechanical vibration into a vibration signal;

[0009] A vibration data processing sub-module, configured to receive the vibration signal sent by the vibration sensor, calculate the vibration energy of the vending machine body according to the vibration signal, and determine whether it is greater than the vibration energy threshold;

[0010] An instruction generation sub-module, configured to:

[0011] When the vibration data processing sub-module determines that the vibration energy is greater than the vibration energy threshold, generate a video acquisition instruction and send it to the video acquisition module.

[0012] Preferably, the video acquisition module includes:

[0013] A video recording sub-module, configured to enter the working state after receiving the video acquisition instruction sent by the instruction generation sub-module, and acquire the monitoring video within the preset range of the vending machine body;

[0014] An anti-shake sub-module, configured to perform real-time motion compensation on the optical elements included in the video recording sub-module according to the vibration signal sent by the vibration sensor;

[0015] A video processing sub-module, configured to perform post anti-shake processing on the monitoring video to obtain an anti-shake video;

[0016] A video recognition sub-module, configured to receive the anti-shake video sent by the video processing sub-module, recognize the content of the anti-shake video, and annotate the recognition result to several frame pictures in the anti-shake video to obtain a target video, and send it to the transmission module.

[0017] Preferably, the anti-shake sub-module includes:

[0018] A vibration transfer function calculation unit, configured to generate a test vibration signal and transmit the test vibration signal from the vending machine body to the video acquisition module, and simultaneously obtain several feedback signals of the device on the transmission path to the test vibration signal;

[0019] Perform signal synthesis on the several feedback signals to obtain a synthesized signal;

[0020] According to the vibration transfer function when the obtained test vibration signal is transmitted from the vending machine body to the video acquisition module, and transmit it to the vibration signal analysis unit;

[0021] A vibration signal analysis unit, configured to perform real-time motion compensation on the optical elements in the video recording sub-module according to the vibration signal and the vibration transfer function.

[0022] Preferably, the video processing sub-module includes:

[0023] A video preprocessing unit for equally dividing the monitoring video into a plurality of monitoring sub-videos;

[0024] A first image extraction unit for extracting the starting frame image of the monitoring sub-video as the starting image; extracting the ending frame image of the monitoring sub-video as the ending image;

[0025] A feature information extraction unit for extracting the feature information shared by the starting image and the ending image as the target feature information of the monitoring sub-video;

[0026] A first calculation and comparison unit for calculating the first displacement distance of the target feature information in the starting image and the ending image, comparing it with a preset first displacement distance threshold, and when it is determined that the first displacement distance is greater than the displacement distance threshold, marking the monitoring sub-video as a sub-video to be processed;

[0027] An original displacement trajectory determination unit for calculating the original displacement trajectory of the target feature information in the sub-video to be processed;

[0028] A displacement trajectory smoothing unit for smoothing the original displacement trajectory to obtain a smoothed displacement trajectory;

[0029] A displacement vector calculation unit for calculating the displacement vector from the original trajectory scatter points in the original displacement trajectory to the smoothed trajectory scatter points in the smoothed displacement trajectory according to the original displacement trajectory and the smoothed displacement trajectory;

[0030] An image displacement unit for displacing the frame image corresponding to the displacement vector in the monitoring sub-video according to the displacement vector;

[0031] A first image splicing unit for orderly splicing a plurality of frame images obtained after being processed by the image displacement unit to obtain a processed sub-video to be processed;

[0032] A video splicing unit for orderly splicing a plurality of processed sub-videos to be processed sent by the first image splicing unit and the original monitoring sub-videos not marked as sub-videos to be processed to obtain an anti-shake video.

[0033] Preferably, the first image splicing unit is further configured to adaptively adjust the size of the video window according to the displacement of the plurality of frame images when orderly splicing the plurality of frame images.

[0034] Preferably, the video recognition sub-module includes:

[0035] A second image extraction unit for extracting the frame images of the anti-shake video and using the extracted frame images as images to be recognized;

[0036] An image recognition unit, configured to:

[0037] Perform edge enhancement and image block recognition on the image to be recognized;

[0038] An image marking unit, configured to mark the several adjacent image blocks when the image recognition unit recognizes the several adjacent image blocks as violent instruments;

[0039] A second image splicing unit, configured to orderly splice the key frame image and the unmarked frame images into a target video and send it to the transmission module.

[0040] Preferably, the image marking unit is further configured to, when the image recognition unit determines that the several adjacent image blocks are jointly recognized as a human portrait, recognize the image block where the face corresponding to the human portrait is located as a human portrait block, and add a human portrait mark to the human portrait block.

[0041] Preferably, when the second image splicing unit orderly splices the key frame image and the unmarked frame images into a target video, a blank frame is added to the next frame of the frame where the human portrait block is located, and an enlarged human portrait block is added to the several blank frames.

[0042] Preferably, the transmission module is further configured to, before sending the target video to the alarm platform, obtain the recording time of the monitoring video and the unique identifier of the vending machine body, and add them to the attribute information of the target video.

[0043] Compared with the prior art, the beneficial effects of the present invention are:

[0044] 1. When the vibration energy of the vending machine is greater than the threshold, the video acquisition module is activated to obtain the monitoring video within the preset range of the vending machine body, instead of collecting the monitoring video in real time, saving storage resources and computing resources.

[0045] 2. Before and after the video acquisition module collects the monitoring video within the preset range of the vending machine body, software and hardware combination is used to perform anti-shake processing on the monitoring video to ensure the clarity of the monitoring video, which helps the staff to determine the situation when the vending machine is stolen.

[0046] 3. When performing anti-shake processing on the monitoring video, the image is recognized and processed, and the information of the recognized image is added to the processed monitoring video, further helping the staff to quickly confirm the situation of the stolen vending machine when it is stolen.

[0047] Other features and advantages of the present invention will be described in the following specification, and in part will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in the written specification and the drawings.

[0048] The technical solution of the present invention will be further described in detail below with reference to the drawings and embodiments. Description of the Drawings

[0049] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, but do not constitute a limitation to the present invention. In the drawings:

[0050] Figure 1 is a schematic diagram of a monitoring and anti-theft system for a vending machine according to an embodiment of the present invention;

[0051] Figure 2 is a method for frame image displacement of a sub-video to be processed according to an embodiment of the present invention;

[0052] Figure 3 is a schematic diagram of the first image stitching unit adaptively adjusting the size of a video window according to an embodiment of the present invention. Detailed Embodiments

[0053] The following describes the preferred embodiments of the present invention with reference to the drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0054] As Figure 1 shown, the present invention provides a monitoring and anti-theft system for a vending machine, including:

[0055] A vibration monitoring module, configured to monitor the vibration energy of the vending machine body included in the vending machine, and generate a video acquisition instruction when it is determined that the vibration energy is greater than a vibration energy threshold;

[0056] A video acquisition module, configured to acquire a monitoring video within a preset range of the vending machine body and perform preprocessing when receiving the video acquisition instruction sent by the vibration monitoring module, obtain a target video, and send the target video to an alarm platform through a transmission module.

[0057] Working principle of the above technical solution: Inside the body of the vending machine, there is a vibration monitoring module for monitoring the vibration energy of the vending machine body. When the vibration energy of the vending machine body is greater than the threshold value, it indicates that the vending machine is suspected of being impacted by an external force. The vibration monitoring module generates a video acquisition instruction and sends it to the video acquisition module. After receiving the video acquisition instruction, the video acquisition module enters the working state, acquires the image data near the vending machine body as the monitoring video, and preprocesses the monitoring video (including hardware-level video anti-shake processing, software-level video anti-shake processing, and video image recognition processing) to obtain a target video with annotation information and a relatively small jitter amplitude in the video frame, which is transmitted to the transmission module and then transmitted by the transmission module to the alarm platform.

[0058] Beneficial effects of the above technical solution: It is determined whether the vending machine is impacted by an external force based on the vibration energy of the vending machine body. Only when the vending machine is suspected of being impacted by an external force (that is, the vibration energy is greater than the threshold value), the video acquisition module is activated to acquire the image data near the vending machine as the monitoring video. The advantage of this is that the video acquisition module does not need to be always on, but only turns on when the vending machine is suspected of being damaged, which can save storage resources and computing resources and reduce the workload of software maintenance personnel. Moreover, the present invention also provides a method for performing hardware-level video anti-shake processing, software-level video anti-shake processing, and video image recognition processing on the monitoring video. The target video obtained by this method has a relatively small jitter amplitude, which is convenient for the staff to clearly see the situation around the vending machine body when the vending machine alarms. At the same time, the target video contains annotation information, which can better help the staff quickly confirm the situation and contribute to holding the lawbreakers accountable after the vending machine is stolen.

[0059] According to some embodiments of the present invention, the vibration monitoring module includes:

[0060] A vibration sensor for detecting the mechanical vibration of the vending machine body and converting the mechanical vibration into a vibration signal;

[0061] A vibration data processing sub-module for receiving the vibration signal sent by the vibration sensor, calculating the vibration energy of the vending machine body according to the vibration signal, and determining whether it is greater than the vibration energy threshold;

[0062] An instruction generation sub-module for:

[0063] When the vibration data processing sub-module determines that the vibration energy is greater than the vibration energy threshold, generating a video acquisition instruction and sending it to the video acquisition module;

[0064] The vibration data processing sub-module calculates the vibration energy of the vending machine body according to the vibration signal, including:

[0065] Decompose the vibration signal S(t) of the vending machine body using FFT technology to obtain the decomposed signal Sc(t) corresponding to the vibration signal S(t);

[0066]

[0067] where t is the time variable; X i (t) is the i-th cosine wave signal obtained after decomposing the vibration signal S(t); In the formula of X i (t), A i is the amplitude of the i-th cosine wave signal; ω i is the angular frequency of the i-th cosine wave signal; is the phase shift of the i-th cosine wave signal; n is the number of cosine wave signals in the decomposed signal Sc(t); i = 1, 2, 3...;

[0068] Calculate the energy of each cosine wave signal in turn and establish the cosine wave signal energy set ES = {E1, E2, E3... E n};

[0069] Normalize the cosine wave signal energy set ES to obtain the normalized cosine wave signal energy set NES = {N1, N2, N3... N n}, where N n is the weight corresponding to the energy E n of the n-th cosine wave signal;

[0070] Set the weight compensation threshold of the cosine wave signal energy

[0071] Compare the weight of each cosine wave signal energy with the weight compensation threshold and determine the weight compensation parameter C(E i ) according to the comparison result,

[0072]

[0073] Substitute the cosine wave signal energy, the weight of the cosine wave signal energy, and the weight compensation parameter of the cosine wave signal energy into the vibration energy calculation formula to calculate the vibration energy corresponding to the vibration signal S(t);

[0074] The vibration energy calculation formula is:

[0075] Working principle of the above technical solution: The vibration monitoring module assembled in the vending machine body includes a vibration sensor sub-module, which is used to measure the mechanical vibration of the vending machine body, convert the mechanical vibration into a vibration signal, and hand it over to the data processing sub-module for processing. The vibration energy of the vending machine body is calculated. When the vibration energy is greater than the threshold, the instruction generation sub-module generates a video acquisition instruction and sends it to the video acquisition module. Further, the present invention provides a vibration energy calculation method. Different from the existing vibration energy calculation methods, the total energy of the vibration signal is obtained by weighted summation of the energies of several cosine wave vibration signals obtained by decomposing the vibration signal. The weights of the energies of the above-mentioned several cosine wave vibration signals are equal to those obtained after normalizing their energies. Before weighted averaging the energies of the above-mentioned several cosine wave vibration signals, it is determined whether the energy of the corresponding normalized signal is less than the threshold. If so, it means that the signal is small and energy compensation is required.

[0076] Advantages of the above technical solution: The vibration signal of the vending machine is measured by the vibration sensor. This technology is relatively mature and has a high reliability. The video acquisition instruction for activating the video acquisition sub-module is generated only when the vibration energy is greater than the threshold, and the video acquisition sub-module does not work at other times, which can save resources to a certain extent. In the vibration energy calculation method provided by the present invention, the energies of several cosine wave vibration signals obtained by decomposing the vibration signal are weighted and summed to obtain the vibration energy. The energy of the cosine wave signal whose normalized signal energy is less than the threshold is compensated, which improves the sensitivity of the vibration monitoring module to a certain extent. And the staff can adjust the sensitivity of the vending machine by adjusting the weight compensation threshold. At the same time, it can effectively avoid the situation that the energy of the main cosine wave signal is too large during the calculation process, resulting in the energy of the secondary cosine wave signal being ignored, ensuring the integrity of the vibration energy spectrum of the vending machine. According to the prior art, the integrity of the vibration energy spectrum can be used for fault determination of mechanical equipment. In practice, the complete vibration energy spectrum can help the staff to monitor the hardware faults of the vending machine.

[0077] According to some embodiments of the present invention, the video acquisition module includes:

[0078] A video recording sub-module, which is used to enter the working state after receiving the video acquisition instruction sent by the instruction generation sub-module, and collect the monitoring video within the preset range of the vending machine body;

[0079] An anti-shake sub-module, which is used to perform real-time motion compensation on the optical elements included in the video recording sub-module according to the vibration signal sent by the vibration sensor;

[0080] A video processing sub-module, which is used to perform post anti-shake processing on the monitoring video to obtain an anti-shake video;

[0081] A video recognition sub-module, configured to receive the anti-shake video sent by the video processing sub-module, recognize the content of the anti-shake video, annotate the recognition result to several frames of the anti-shake video to obtain a target video, and send it to the transmission module.

[0082] The anti-shake sub-module performs real-time motion compensation on the optical elements included in the video recording sub-module according to the vibration signal sent by the vibration sensor, including:

[0083] Receiving the vibration signal sent by the vibration sensor and generating motion compensation parameters according to the vibration signal;

[0084] After the video recording sub-module enters the working state, performing real-time motion compensation on the optical elements included in the video recording sub-module according to the motion compensation parameters.

[0085] The working principle of the above technical solution: After receiving the video acquisition instruction, the video recording module in the video acquisition module will enter the working state. At the same time, the video anti-shake module will obtain the vibration signal from the vibration monitoring module, generate motion compensation parameters according to the vibration signal, and perform real-time motion compensation on the optical components such as the camera in the video recording sub-module to reduce the jitter during the monitoring video recording; after the video recording sub-module finishes recording, it will send the monitoring video to the video processing sub-module, and the video processing sub-module will perform post anti-shake processing on the monitoring video to obtain an anti-shake video and send it to the video recognition sub-module. The video recognition sub-module will recognize the anti-shake video, add the recognition to the picture of the anti-shake video to obtain a target video, and transmit it to the transmission module.

[0086] The beneficial effects of the above technical solution: Generate motion compensation parameters according to the vibration signal, perform motion compensation on the video recording sub-module according to the motion compensation parameters, reduce the jitter during the video recording process, and ensure the stability and clarity of the video. After the video recording is completed, perform post anti-shake processing on the monitoring video to further improve the stability of the video, ensure that the video staff can clearly see the monitoring video, analyze the cause of excessive vibration energy of the vending machine, recognize and annotate the anti-shake video, and further help the video staff quickly lock in the lawbreakers.

[0087] According to some embodiments of the present invention, the anti-shake sub-module includes:

[0088] A vibration transfer function calculation unit, configured to generate a test vibration signal and transmit the test vibration signal from the vending machine body to the video acquisition module, and at the same time obtain several feedback signals of the components on the transmission path to the test vibration signal;

[0089] Perform signal synthesis on the plurality of feedback signals to obtain a synthesized signal;

[0090] Based on the synthesized signal and the test vibration signal, obtain the vibration transfer function when the test vibration signal is transmitted from the vending machine body to the video acquisition module for transmission, and transmit it to the vibration signal analysis unit;

[0091] The vibration signal analysis unit is configured to perform real-time motion compensation on the optical elements in the video recording sub-module according to the vibration signal and the vibration transfer function.

[0092] The transfer function calculation unit obtains the vibration transfer function when the test vibration signal is transmitted from the vending machine body to the video acquisition module for transmission based on the synthesized signal and the test vibration signal, including:

[0093] Set the sampling period according to the period of the test vibration signal, and sample the test vibration signal and the synthesized signal respectively according to the sampling period to obtain a plurality of discrete test vibration signals and a plurality of discrete synthesized signals;

[0094] Successively calculate the ratio of the relative intensity of each discrete test vibration signal to the relative intensity of the corresponding discrete synthesized signal, and establish a signal ratio sequence including a plurality of the ratios;

[0095] Taking the sampling time points corresponding to the signal ratio sequence as the horizontal axis and the ratio of the discrete test vibration signal to the discrete synthesized signal as the vertical axis, establish a rectangular coordinate system, convert the signal ratio sequence into a plurality of scatter points in the rectangular coordinate system, and fit the plurality of scatter points to obtain the vibration transfer function when the test vibration signal is transmitted from the vending machine body to the video acquisition module for transmission;

[0096] The vibration signal analysis unit performs real-time motion compensation on the optical elements in the video recording sub-module according to the vibration signal and the vibration transfer function, including:

[0097] Analyze the vibration signal to obtain the real-time motion vector of the vending machine body;

[0098] Calculate the motion parameters of the vending machine body according to the real-time motion vector;

[0099] Substitute the motion parameters into the vibration transfer function to generate motion compensation parameters;

[0100] Transmit the motion compensation parameters to the motion compensation execution unit;

[0101] The motion compensation execution unit is configured to perform real-time motion compensation on the optical elements in the video recording sub-module according to the motion compensation parameters.

[0102] The working principle of the above technical solution is as follows: the anti-shake submodule includes a vibration transfer function calculation unit, which is used to generate a test vibration signal, and transmit the test vibration signal from the vending machine body to the video acquisition module, and collect a number of signals fed back by the transmission path to the test vibration signal in the transmission path. According to the several feedback signals, the vibration transfer function between the vending machine body and the video acquisition module can be determined, including: synthesizing all feedback signals to obtain a synthesized signal; setting a sampling period to synthesize the synthesized signal and the test vibration signal respectively to obtain a discrete synthesized signal sequence and a discrete test vibration signal sequence; and sequentially calculating the ratio of each discrete synthesized signal in the discrete synthesized signal sequence to the corresponding discrete test vibration signal in the discrete test vibration sequence, as shown in the following table:

[0103] T 1 2 3 4 5 6 7 I1 0.8749 0.8459 0.8858 0.8721 0.8612 0.8656 0.8743 I2 0.5783 0.6815 0.7685 0.7399 0.512 0.5587 0.4558 R 0.6610 0.8057 0.8676 0.8484 0.5945 0.6454 0.5213

[0104] In this table, T is the sampling time point, I1 is the relative intensity of the discrete test vibration signal, I2 is the relative intensity of the discrete synthetic signal, and R is the ratio of I2 to I1; the sampling time point T is used as the independent variable to establish the horizontal axis, and the relative intensity R is used as the dependent variable to establish the vertical axis to obtain a rectangular coordinate system. According to the above table, several scattered points are obtained in the rectangular coordinate system. After fitting the obtained scattered points, the vibration transfer function between the vending machine body and the video acquisition module can be obtained. After that, the motion parameters of the vending machine body (that is, parameters such as vibration direction, vibration period, and vibration amplitude) are obtained in the vibration signal analysis unit, and motion compensation parameters are generated according to the motion parameters and the vibration transfer function. When the vending machine vibrates, the video acquisition module in the vending machine body will also vibrate with the vibration of the vending machine. The vibration of the video acquisition module is determined by the vibration of the vending machine and the vibration transfer function between the vending machine and the video acquisition module. After obtaining the motion parameters of the vending machine body, the motion compensation parameters of the video acquisition module are determined according to the motion parameters and transfer function of the vending machine. Based on the parameters, we can perform motion compensation on the video acquisition module to keep the video acquisition module as still as possible, or adjust the optical component parameters such as the focal length and aperture size of the camera in the video acquisition module according to the motion compensation parameters.

[0105] The beneficial effects of the above technical solution are: generating a test vibration signal and calculating the vibration transfer function between the video acquisition module and the vending machine body according to the test vibration signal; and the motion compensation parameters generated according to the motion parameters and the vibration transfer function can better compensate for the vibration of the video acquisition module than the motion compensation parameters generated only according to the motion parameters, thereby ensuring its stability and further ensuring the clarity of the monitoring video.

[0106] According to some embodiments of the present invention, the video processing submodule includes:

[0107] A video preprocessing unit, configured to equally divide the monitoring video into a plurality of monitoring sub-videos;

[0108] A first image extraction unit, configured to extract the starting frame image of the monitoring sub-video as the starting image; extract the ending frame image of the monitoring sub-video as the ending image;

[0109] A feature information extraction unit, configured to extract the feature information jointly owned by the starting image and the ending image as the target feature information of the monitoring sub-video;

[0110] A first calculation and comparison unit, configured to calculate a first displacement distance of the target feature information in the starting image and the ending image, compare it with a preset first displacement distance threshold, and when it is determined that the first displacement distance is greater than the displacement distance threshold, mark the monitoring sub-video as a sub-video to be processed;

[0111] An original displacement trajectory determination unit, configured to calculate the original displacement trajectory of the target feature information in the sub-video to be processed;

[0112] A displacement trajectory smoothing unit, configured to perform a smoothing process on the original displacement trajectory to obtain a smoothed displacement trajectory;

[0113] A displacement vector calculation unit, configured to calculate a displacement vector from an original trajectory scatter point in the original displacement trajectory to a smoothed trajectory scatter point in the smoothed displacement trajectory according to the original displacement trajectory and the smoothed displacement trajectory;

[0114] An image displacement unit, configured to perform a displacement process on the frame image corresponding to the displacement vector in the monitoring sub-video according to the displacement vector;

[0115] A first image splicing unit, configured to orderly splice a plurality of frame images obtained after being processed by the image displacement unit to obtain a processed sub-video to be processed;

[0116] A video splicing unit, configured to orderly splice a plurality of processed sub-videos to be processed sent by the first image splicing unit and the original monitoring sub-videos not marked as sub-videos to be processed to obtain an anti-shake video.

[0117] The first calculation and comparison unit calculates the first displacement distance of the target feature information in the starting image and the ending image, including:

[0118] Calculating first coordinate information of the target feature information in the starting image;

[0119] Calculating second coordinate information of the target feature information in the ending image;

[0120] Calculate a first displacement distance of the target feature information according to the first coordinate information and the second coordinate information;

[0121] The original displacement trajectory determination unit calculates an original displacement trajectory of the target feature information in the sub-video to be processed;

[0122] Receive the sub-video to be processed sent by the first calculation and comparison unit;

[0123] Calculate coordinate information of the target feature information in each frame of the sub-video to be processed;

[0124] Determine an original displacement trajectory of the target feature information in the sub-video to be processed according to the coordinate information; the original displacement trajectory is a broken line graph including a plurality of original trajectory scatter points;

[0125] After the displacement trajectory smoothing unit performs smoothing processing on the original displacement trajectory to obtain a smoothed displacement trajectory, it further includes:

[0126] Place the original displacement trajectory and the smoothed displacement trajectory in a rectangular coordinate system, and determine a smoothed trajectory scatter point corresponding to each original trajectory scatter point in the smoothed displacement trajectory to obtain a plurality of smoothed trajectory scatter points;

[0127] Working principle of the above technical solution: After receiving the original video, the video processing sub-module divides the original video into several original sub-videos in equal proportion. Then, the first image extraction unit selects any one of the original sub-videos, extracts its starting image and ending image, and the feature information extraction unit performs feature extraction and feature matching on the above-mentioned starting image and ending image, and takes the feature information shared by the two as the target feature information of the original sub-video. For example, when the starting image contains a basketball, feature extraction of the starting image will obtain a circle containing lines, and the above-mentioned circle containing lines is the feature information of the starting image. According to the same method, feature extraction is performed on the ending image to obtain the feature information of the ending image. Assuming that the feature information of the ending image also has the above-mentioned circle containing lines, when performing feature matching on the starting image and the ending image later, the above-mentioned circle containing lines shared by the two is used as the target feature information. Then, the first calculation and comparison module calculates the first coordinate information of the target feature information in the starting image and the second coordinate information in the ending image respectively. Here, it should be noted that if the target feature information is a graphic, the coordinates of the center point of the graphic are used as its coordinate information. After obtaining the first coordinate information and the second coordinate information, the first displacement distance of the target feature information is calculated according to the first coordinate information and the second coordinate information (that is, the distance between the first coordinate information and the second coordinate information). Assuming that the first coordinate information is (589.3551, 997.3541) and the second coordinate information is (621.6654, 825.4785), the displacement distance of the target feature information is 174.8861; the first displacement distance is compared with the preset first displacement distance threshold. When the first displacement distance is greater than the preset first displacement distance threshold, it proves that the target feature information has a significant displacement from the starting image to the ending image of the original sub-video, indicating that the video jitters significantly, and the original sub-video is marked as a sub-video to be processed. Here, there are two points to note. First, due to the periodicity of the mechanical vibration of the vending machine, the length of the original sub-video should not be too long, otherwise, although the starting image and the ending image are in different periods, due to too small a phase difference, the displacement distance of the target feature information is not obvious, resulting in misjudgment; second, when performing feature matching on the starting image and the ending image, single target feature information cannot be selected for local matching only, and multiple target features should be used for overall matching to grasp the overall displacement (picture jitter) of the image. Then, the original displacement trajectory determination unit sequentially obtains the coordinate information of the target feature information of the sub-video to be processed, and calculates the original displacement trajectory of the target feature information in each frame of the image. The original displacement trajectory is a broken line graph containing several original trajectory scatter points.After that, the displacement trajectory smoothing unit smooths the displacement trajectory to obtain a smooth displacement trajectory, places the smooth displacement trajectory and the original displacement trajectory in a rectangular coordinate system, determines the smooth trajectory scatter points corresponding to each original trajectory scatter point, and then the displacement vector calculation unit calculates the displacement vector between each original trajectory scatter point and the corresponding smooth trajectory scatter point. After that, the image displacement unit performs displacement processing on the frame image corresponding to the displacement vector. As. Figure 2 As shown, the present invention provides a method for displacing frame images of a sub-video to be processed, including: 2-1. Obtain the original displacement trajectory of the target feature information of the sub-video to be processed. The original displacement trajectory is a broken line graph containing original trajectory points, and the original trajectory points are shown as solid dots in the figure; 2-2. Smooth the original displacement trajectory to obtain a smooth displacement trajectory. The smooth displacement trajectory is a curve graph containing smooth trajectory points, and the smooth trajectory scatter points are shown as solid pentagrams in the figure; 2-3. Calculate the displacement vector of the frame image according to the original displacement trajectory and the smooth displacement trajectory. As shown in the third point in the figure, this displacement vector points from the original trajectory point to the smooth trajectory point; 2-4. Perform displacement processing on the frame image corresponding to the displacement vector. As shown in the figure, S3 is the third frame image corresponding to the target feature information corresponding to the original trajectory point, and this image is the image before displacement. S3' is the third frame image corresponding to the target feature information corresponding to the smooth trajectory point, and this image is the image after displacement according to the displacement vector corresponding to the third frame image. After performing displacement processing on the frame image, the first image splicing unit splices the displaced frame images in order according to the original order of the frame images to obtain the processed sub-video to be processed. Then, the video splicing unit splices the processed sub-video to be processed and the original monitoring sub-video that is not marked as the sub-video to be processed (without the need to process the original monitoring sub-video) in order to obtain a shake-proof video.

[0128] Beneficial effects of the above technical solution: Obtain the target feature information shared by the starting image and the ending image of the monitoring sub-video, determine the jitter situation of the monitoring sub-video according to the displacement distance of the target feature information from the starting image to the ending image, and perform a preliminary screening on the monitoring sub-video. Only the monitoring sub-video with a more obvious jitter situation is processed as the sub-video to be processed, rather than processing all monitoring sub-videos, which improves the video processing speed and saves computer resources, making this solution more suitable for embedded devices such as vending machines; Perform displacement processing on the sub-video to be processed frame by frame, which can effectively ensure that the processed sub-video to be processed has a high clarity and is coherent between frames, reducing the image tearing feeling of the shake-proof video; The displacement smoothing method consumes less resources for shake-proof processing of the video, and there will be no lag when processing multiple sub-videos to be processed in parallel. It is suitable for embedded devices such as vending machines, and the processing speed is relatively fast, which can ensure that the staff can receive the target video in time, helping to stop illegal elements in time or hold them accountable afterwards, and ensuring the property safety of the vending machine owner.

[0129] According to some embodiments of the present invention, the first image stitching unit is further configured to adaptively adjust the size of the video window according to the displacement of several frames of images when the several frames of images are stitched in order.

[0130] The working principle of the above technical solution is as follows: As Figure 3 shown, the process of the first image stitching unit adaptively adjusting the size of the video window includes: 3-1. Determine the original video window 31; 3-2. Determine the window 33 of the frame image with the largest displacement component in the positive x-axis direction; 3-3. Adjust the edge of the original video window 31 in the positive x-axis direction to the positive x-axis direction until it slightly exceeds the window 33 of the frame image corresponding to the largest displacement component in the positive x-axis direction, to obtain the adjusted video window 32. The adjustment of the original video window in other directions is the same.

[0131] The beneficial effect of the above technical solution: When stitching several frames of pictures after displacement, appropriately increase the video window to prevent the loss of the image area beyond the original window, and ensure the integrity of image data and video data.

[0132] According to some embodiments of the present invention, the video recognition sub-module includes:

[0133] A second image extraction unit, configured to extract the frame images of the anti-shake video, and use the extracted frame images as the images to be recognized;

[0134] An image recognition unit, configured to:

[0135] Perform edge enhancement and image block recognition on the image to be recognized;

[0136] An image marking unit, configured to mark several adjacent image blocks when the image recognition unit recognizes several adjacent image blocks as violent instruments;

[0137] A second image stitching unit, configured to stitch the key frame images and the unmarked frame images in order into a target video and send it to the transmission module.

[0138] The image recognition unit performing edge enhancement and image block recognition on the image to be recognized includes:

[0139] Perform edge enhancement on the image to be recognized to obtain an edge-enhanced image;

[0140] Divide the edge-enhanced image into several image blocks according to the contour lines in the edge-enhanced image, and recognize the several image blocks according to the preset image recognition model in the image recognition unit;

[0141] After the image marking unit marks the several adjacent image blocks, it further includes:

[0142] Mark the image to be recognized where the marked block is located and several frames of images before and after it as key frame images;

[0143] The working principle of the above technical solution: After the video recognition sub-module receives the anti-shake video, the second image extraction module extracts the images of the anti-shake video frame by frame as the images to be recognized. Then, the image recognition unit performs edge enhancement on the images to be recognized to obtain edge-enhanced images. The contour lines of the items in the edge-enhanced images will be significantly enhanced. Therefore, the edge-enhanced images can be segmented into multiple image blocks according to the contour lines. The image recognition unit will recognize the image blocks. If one or several adjacent image blocks are recognized as violent instruments (such as wrenches, hammers, etc., which can cause serious damage to the vending machine), mark the block where the violent instrument is located. This mark can be to change the color of these contour lines, etc. Then, mark the marked image and several frames of images before and after it as several frames of images; The second image splicing unit will orderly splice the marked key frame images and the unmarked frame images into a target video and transmit it to the transmission module.

[0144] The beneficial effects of the above technical solution: Perform image edge enhancement and image recognition on the anti-shake video frame by frame, and annotate the images according to the recognition results, which is convenient for the staff to lock the criminal tools and the time of the crime of the lawbreakers, and helps with accountability. The video processing technology given in this solution is relatively simple and consumes less resources, and is suitable for embedded devices such as vending machines.

[0145] According to some embodiments of the present invention, the image marking unit is further configured to, when the image recognition unit determines that the several adjacent image blocks are jointly recognized as a human portrait, control the image recognition unit to perform secondary recognition on the several adjacent image blocks jointly recognized as a human portrait, use the image block where the face of the human portrait is located as the human portrait block, and add a human portrait mark to the human portrait block.

[0146] According to some embodiments of the present invention, the image marking unit is further configured to, when the image recognition unit determines that the several adjacent image blocks are jointly recognized as a human portrait, recognize the image block where the face of the human portrait is located as the human portrait block, and add a human portrait mark to the human portrait block.

[0147] The working principle of the above technical solution: When the image recognition unit recognizes multiple vector image blocks as a human portrait, perform secondary recognition on the human portrait, use the facial area of the human portrait as the human portrait block, and then the image marking unit adds an additional mark to it. Then, add several blank frames before the next frame of the frame where the human portrait block is located to magnify the human portrait block.

[0148] Advantageous effects of the above technical solution: Focus on identifying and marking the portrait to help the staff determine the situation of the people near the vending machine in a timely manner when the vending machine vibrates, which is convenient for locking up lawbreakers and holding them accountable afterwards.

[0149] According to some embodiments of the present invention, the transmission module is further configured to obtain the recording time of the monitoring video and the unique identifier of the vending machine body before sending the target video to the alarm platform, and add the recording time and the unique identifier to the attribute information of the target video.

[0150] Working principle of the above technical solution: Before sending the video to the alarm platform, the transmission module obtains the recording time of the original monitoring video and the unique identifier of the vending machine that recorded the original monitoring video, and renames the target video based on the recording time and the unique identifier according to a preset rule. For example, before sending the target video to the alarm platform, the transmission module obtains the recording time of the monitoring video (assumed to be 12-21-23:25) and the unique identifier of the vending machine it is in (i.e., the device number of the vending machine or the number customized by the staff, assumed to be VMSED-2539741424), and generates "VMSED-2539741424.12 / 21 / 23:25" according to the preset rule (assumed to be "vending machine number" + "." + "MM / DD / TIME"), and adds it to the attribute information of the target monitoring video.

[0151] Advantageous effects of the above technical solution: Adding the recording time of the original monitoring video and the unique identifier of the vending machine to the name of the target video file helps video staff determine which vending machine has a problem and the time when the vending machine has an abnormality.

[0152] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. A monitoring and anti-theft system for a vending machine, characterized in that, Including: A vibration monitoring module, which is used to monitor the vibration energy of the vending machine body included in the vending machine, and generate a video acquisition instruction when it is determined that the vibration energy is greater than the vibration energy threshold; A video acquisition module, which is used to acquire the monitoring video within the preset range of the vending machine body and perform preprocessing when receiving the video acquisition instruction sent by the vibration monitoring module, obtain the target video, and send the target video to the alarm platform through the transmission module; The vibration monitoring module includes: A vibration sensor, which is used to detect the mechanical vibration of the vending machine body and convert the mechanical vibration into a vibration signal; A vibration data processing sub-module, which is used to receive the vibration signal sent by the vibration sensor, calculate the vibration energy of the vending machine body according to the vibration signal, and judge whether it is greater than the vibration energy threshold; An instruction generation sub-module, which is used for: When the vibration data processing sub-module determines that the vibration energy is greater than the vibration energy threshold, generate a video acquisition instruction and send it to the video acquisition module; The video acquisition module includes: A video recording sub-module, which is used to enter the working state after receiving the video acquisition instruction sent by the instruction generation sub-module, and acquire the monitoring video within the preset range of the vending machine body; An anti-shake sub-module, which is used to perform real-time motion compensation on the optical elements included in the video recording sub-module according to the vibration signal sent by the vibration sensor; A video processing sub-module, which is used to perform post anti-shake processing on the monitoring video to obtain an anti-shake video; A video recognition sub-module, which is used to receive the anti-shake video sent by the video processing sub-module, recognize the content of the anti-shake video, annotate the recognition result to several frame images in the anti-shake video, obtain the target video, and send it to the transmission module; The anti-shake sub-module includes: A vibration transfer function calculation unit, which is used to generate a test vibration signal and transmit the test vibration signal from the vending machine body to the video acquisition module, and at the same time obtain several feedback signals of the devices on the transmission path to the test vibration signal; Perform signal synthesis on the several feedback signals to obtain a synthesized signal; Obtain the vibration transfer function when the test vibration signal is transmitted from the vending machine body to the video acquisition module according to the synthesized signal and the test vibration signal, and transmit it to the vibration signal analysis unit; A vibration signal analysis unit, which is used to perform real-time motion compensation on the optical elements in the video recording sub-module according to the vibration signal and the vibration transfer function.

2. The monitoring and anti-theft system of the vending machine according to claim 1, characterized in that, The video processing sub-module includes: A video preprocessing unit, which is used to equally divide the monitoring video into several monitoring sub-videos; A first image extraction unit, which is used to extract the starting frame image of the monitoring sub-video as the starting image; extract the ending frame image of the monitoring sub-video as the ending image; A feature information extraction unit, which is used to extract the feature information shared by the starting image and the ending image as the target feature information of the monitoring sub-video; The first calculation and comparison unit is configured to calculate a first displacement distance of the target feature information in the start image and the end image, compare it with a preset first displacement distance threshold, and mark the monitored sub-video as a sub-video to be processed when it is determined that the first displacement distance is greater than the displacement distance threshold; The original displacement trajectory determination unit is configured to calculate the original displacement trajectory of the target feature information in the sub-video to be processed; The displacement trajectory smoothing unit is configured to perform a smoothing process on the original displacement trajectory to obtain a smoothed displacement trajectory; The displacement vector calculation unit is configured to calculate a displacement vector from an original trajectory scatter point in the original displacement trajectory to a smoothed trajectory scatter point in the smoothed displacement trajectory according to the original displacement trajectory and the smoothed displacement trajectory; The image displacement unit is configured to perform a displacement process on the frame image corresponding to the displacement vector in the monitored sub-video according to the displacement vector; The first image splicing unit is configured to orderly splice a plurality of frame images obtained after being processed by the image displacement unit to obtain a processed sub-video to be processed; The video splicing unit is configured to orderly splice a plurality of processed sub-videos to be processed sent by the first image splicing unit and the original monitored sub-videos not marked as sub-videos to be processed to obtain an anti-shake video.

3. The monitoring and anti-theft system of the vending machine according to claim 2, characterized in that, The first image splicing unit is further configured to adaptively adjust the size of the video window according to the displacements of the plurality of frame images when orderly splicing the plurality of frame images.

4. The monitoring and anti-theft system of the vending machine according to claim 2, characterized in that, The video recognition sub-module includes: The second image extraction unit is configured to extract frame images of the anti-shake video and use the extracted frame images as images to be recognized; The image recognition unit is configured to: Perform edge enhancement and image block recognition on the image to be recognized; The image marking unit is configured to mark a plurality of adjacent image blocks when the image recognition unit recognizes the plurality of adjacent image blocks as violent instruments; The second image splicing unit is configured to orderly splice key frame images and unmarked frame images into a target video and send it to the transmission module.

5. The monitoring and anti-theft system of the vending machine according to claim 4, characterized in that, The image marking unit is further configured to, when the image recognition unit determines that the plurality of adjacent image blocks are jointly recognized as a human portrait, recognize the image block where the face corresponding to the human portrait is located as a human portrait block and add a human portrait mark to the human portrait block.

6. The monitoring and anti-theft system of the vending machine according to claim 5, characterized in that, When the second image splicing unit orderly splices key frame images and unmarked frame images into a target video, a blank frame is added to the next frame of the frame where the human portrait block is located, and an enlarged human portrait block is added to the plurality of blank frames.

7. The monitoring and anti-theft system of the vending machine according to claim 2, characterized in that, The transmission module is further configured to obtain the recording time of the monitored video and the unique identifier of the vending machine body and add them to the attribute information of the target video before sending the target video to the alarm platform.

Images