Method and equipment for fusing production process data and video monitoring data
By integrating production process data and video surveillance data in industrial production, the inefficiency problem caused by data independence is solved, real-time and historical data are synchronized display, and production management and equipment maintenance efficiency is improved.
Patent Information
- Application Number
- CN202411162323.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-12-17
AI Technical Summary
In industrial production, the collection and display system of production process data and video surveillance data is usually run independently, resulting in the query and matching data of the same time period that requires manual screening, which affects the efficiency of production management and equipment maintenance.
Through a method of fusing production process data with video surveillance data, real-time and historical data are obtained and integrated display is performed on the display interface. VLC library and FFmpeg tools are used to obtain video surveillance data, and data acquisition is improved using shared memory blocks and multi-threading technology. and update efficiency to realize the synchronous display of real-time and historical data.
Improve the efficiency of production management and equipment maintenance. Through real-time and synchronous display of historical data, it facilitates monitoring and fault analysis at the production site, and reduces hardware resource consumption.
Smart Images
Figure CN119148675A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of data processing technology, and in particular, relates to a method and device for fusing production process data with video surveillance data. Background Art
[0002] In industrial production, it is usually necessary to collect and display production process data (such as rolling mill transmission side pressure, high-pressure water flow, servo valve opening, etc.) and video monitoring data (such as equipment monitoring video, etc.). In related technologies, the collection and display systems of production process data and video monitoring data are often operated independently. When it is necessary to query the production process data and video monitoring data in the production process, it is necessary to use their respective systems to query them separately, and then manually screen and match the production process data and video monitoring data in the same time period. This data processing method affects the efficiency of production management and equipment maintenance. Summary of the invention
[0003] The embodiments of the present application provide a method and device for fusing industrial production data with video data, thereby improving the efficiency of production management and equipment maintenance at least to a certain extent.
[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.
[0005] According to a first aspect of an embodiment of the present application, a method for fusing production process data with video surveillance data is provided, comprising:
[0006] Acquire real-time video surveillance data in the production area and its corresponding real-time production process data, and historical video surveillance data in the production area and its corresponding historical production process data in the same time period;
[0007] The real-time video monitoring data and the real-time production process data are integrated and displayed on a first display interface;
[0008] The historical video monitoring data and the historical production process data are integrated and displayed on the second display interface.
[0009] In some embodiments, the method for fusing production process data with video surveillance data further includes:
[0010] In response to a user's operation of enabling the real-time data service, updating a device list database, wherein the device list database includes device information of all monitoring devices in the production area and location information of all locations monitored by each of the monitoring devices;
[0011] Configuring a shared memory block for each of the monitoring devices;
[0012] The real-time production process data of all points monitored by each monitoring device are stored in the corresponding shared memory block according to a preset sampling period.
[0013] In some embodiments, the step of obtaining real-time video surveillance data in a production area and corresponding real-time production process data includes:
[0014] Acquire the real-time video monitoring data from the first monitoring device in the production area based on the VLC library and the FFmpeg tool;
[0015] The real-time production process data in the shared memory block corresponding to the first monitoring device is determined as the real-time production process data corresponding to the real-time video monitoring data.
[0016] In some embodiments, the fusion display of the real-time video monitoring data and the real-time production process data on the first display interface includes:
[0017] Acquire first point information from the device list database according to the device information of the first monitoring device;
[0018] Displaying the first point information and real-time production process data corresponding to the real-time video monitoring data in the first area of the first display interface;
[0019] Displaying the real-time video monitoring data in a second area of the first display interface;
[0020] The control controls for the real-time video monitoring data are displayed in the third control area of the first display interface.
[0021] In some embodiments, the method for fusing production process data with video surveillance data further includes:
[0022] In response to a user's operation of enabling the historical data service, a device configuration database is created for each monitoring device in the production area, wherein the device configuration database includes point information of all points monitored by the corresponding monitoring device;
[0023] For each monitoring device, read data from the corresponding shared memory block and cache it into the array;
[0024] When the data in the array reaches a preset amount, the data in the array is written into a corresponding equipment configuration database as historical production process data.
[0025] In some embodiments, obtaining the historical video surveillance data in the production area and the historical production process data corresponding to the same time period includes:
[0026] Acquire historical video monitoring data of a second monitoring device in the production area according to the start and end times set by the user;
[0027] The historical production process data of the same time period as the start and end time is obtained from the equipment configuration database corresponding to the second monitoring equipment.
[0028] In some embodiments, the fusing and displaying the historical video monitoring data and the historical production process data on the second display interface includes:
[0029] Acquire second point information from a corresponding device configuration database according to the device information of the second monitoring device;
[0030] Displaying the second point information and the historical production process data of the same time period in the first area of the second display interface;
[0031] Displaying the historical video surveillance data in a second area of the second display interface;
[0032] The control controls for the historical video surveillance data are displayed in the third area of the second display interface.
[0033] In some embodiments, displaying the second point information and the historical production process data of the same time period in the second area of the second display interface includes:
[0034] Displaying the second point information in the first area of the second display interface;
[0035] Determining the update frequency of the playback progress bar of the historical video monitoring data according to the preset sampling period;
[0036] Determine the update times of the playback progress bar according to the start and end times and the update frequency;
[0037] Divide the total amount of data in the historical production process data of the same time period by the number of updates to obtain the amount of data that needs to be displayed each time the playback progress bar is updated;
[0038] When the playback progress bar is updated, a corresponding amount of data is obtained from the historical production process data in the same time period and displayed in the first area of the second display interface.
[0039] In some embodiments, the method for fusing production process data with video surveillance data further includes:
[0040] In response to a trigger point selected by a user, monitoring the production process data of the trigger point;
[0041] When the production process data at the trigger point meets the preset trigger conditions, the steps of obtaining the real-time video surveillance data in the production area and its corresponding real-time production process data, the historical video surveillance data in the production area and its corresponding historical production process data in the same time period are executed.
[0042] According to a second aspect of an embodiment of the present application, a device for fusing production process data with video surveillance data is provided, comprising a processor and a memory, wherein the memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, the steps of the method described in any one of the first aspects above are implemented.
[0043] In the present application, by obtaining the real-time video surveillance data in the production area and its corresponding real-time production process data, the historical video surveillance data in the production area and its corresponding historical production process data in the same time period; the real-time video surveillance data and the real-time production process data are integrated and displayed on the first display interface; the historical video surveillance data and the historical production process data are integrated and displayed on the second display interface. Through the above scheme, the real-time production process data and the real-time video surveillance data can be displayed simultaneously on the display interface, and the historical production process data and the historical video surveillance data of the same time period can also be synchronously displayed on the display interface, thereby improving the efficiency of production management and equipment maintenance.
[0044] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0046] Figure 1 A schematic flow chart of a method for fusing production process data with video surveillance data in one embodiment is shown;
[0047] Figure 2 A software interface diagram for collecting production process data and video monitoring data in one embodiment is shown;
[0048] Figure 3 A software interface diagram for acquiring and displaying real-time production process data and real-time video monitoring data in one embodiment is shown;
[0049] Figure 4 A diagram showing a software interface for acquiring and displaying historical production process data and historical video monitoring data in one embodiment is shown;
[0050] Figure 5 A software interface diagram related to an event triggering function in one embodiment is shown;
[0051] Figure 6 A block diagram of a device for fusing production process data with video surveillance data in one embodiment is shown;
[0052] Figure 7 A schematic diagram of the structure of a device for fusing production process data with video surveillance data in one embodiment is shown. DETAILED DESCRIPTION
[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0054] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the present application.
[0055] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0056] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.
[0057] It should also be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the objects used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those shown or described.
[0058] In order to make those skilled in the art better understand the present application, firstly, Figure 1 The application scenarios involved in this application are briefly described.
[0059] Figure 1 FIG. 1 is a flow chart showing a method for fusing production process data with video surveillance data in one embodiment. Figure 1 As shown, the method for fusing production process data with video surveillance data may include the following steps 101 to 103.
[0060] In step 101, real-time video surveillance data in a production area and its corresponding real-time production process data, historical video surveillance data in the production area and its corresponding historical production process data in the same time period are obtained.
[0061] The embodiment of the present application integrates production process data and video surveillance data from two aspects: real-time monitoring and historical backtracking. By monitoring the relevant production process data while monitoring the production site conditions in real time, it is convenient to grasp the on-site conditions and data changes in the production process in real time. Through historical backtracking, when a fault alarm occurs, the video surveillance data and production process data for a period of time before the fault occurs can be automatically retrieved, and the two can be bound and played by time. Users can view the video surveillance data and corresponding production process data at any time by adjusting the playback progress bar, which provides great convenience for production and management.
[0062] It is understandable that before obtaining the video surveillance data and the production process data, the video surveillance data and the production process data need to be collected and stored.
[0063] Figure 2 FIG. 1 shows a software interface diagram for collecting production process data and video monitoring data in one embodiment. Figure 2As shown, a data service software Data_Server can be developed to collect and store production process data. The software can be developed using Python programming, and the software interface can be designed using the Pyqt5 library. The data service interface can use the OPC UA communication protocol, and the user interface has the functions of inputting the data server address, setting the sampling period of real-time production process data and historical production process data, setting the time difference adjustment, and data caching of the data storage service. The connection with the data server is realized through the opcua library. After the connection, the configured channel name is obtained, and the point information of all monitoring devices and all points monitored by each monitoring device is traversed in a loop.
[0064] In some embodiments, the process of storing real-time production process data can be: in response to the user's operation of enabling the real-time data service, updating the equipment list database, wherein the equipment list database includes the equipment information of all monitoring devices in the production area and the point information of all points monitored by each monitoring device; configuring a shared memory block for each monitoring device; and storing the real-time production process data of all points monitored by each monitoring device in the corresponding shared memory block according to a preset sampling period.
[0065] Among them, taking the point being a rolling mill as an example, the point information may be the pressure on the transmission side of the rolling mill; taking the point being a servo valve as an example, the point information may be the opening degree of the servo valve.
[0066] It is understandable that when the user enables the real-time data service in the software interface, it is necessary to create a device list database, store the device information of all traversed monitoring devices and the point information of all points monitored by each monitoring device in the device list database, and check whether there are any updates to the device list database after each real-time data service is enabled. If there are any updates, delete them and re-establish them.
[0067] During the implementation process, an independent thread can be started for each monitoring device. After the thread is started, a shared memory block named after the monitoring device is created. The real-time production process data of all points under the corresponding monitoring device is read and updated to the shared memory block according to the sampling period, thereby realizing the storage of real-time production process data.
[0068] In some embodiments, the process of storing historical production process data may be: in response to a user enabling the historical data service, creating a device configuration database for each monitoring device in the production area, the device configuration database including the point information of all points monitored by the corresponding monitoring device; for each monitoring device, reading data from the corresponding shared memory block and caching it in an array; when the data in the array reaches a preset number, writing the data in the array as historical production process data into the corresponding device configuration database.
[0069] It is understandable that when the user enables the historical data service in the software interface, an independent device configuration database can be created for each monitoring device in the data_record folder. The fields in the table of the device configuration database are the point information of all points monitored by the corresponding monitoring device. The device configuration database can be a Sqlite3 database, which can realize the application of installation-free, high-speed and flexible storage. At the same time, multiple independent threads are enabled to read the data in the corresponding shared memory block respectively, parse the read data into a single data array0 in the form of an array, create an empty two-dimensional array array_all, store the first read data array0 into the array array_all, and then wait according to the storage cycle of the historical production process data set by the user. After the waiting is over, read the data array1 from the shared memory block again and store it in the array array_all, and then continue to loop until the data cached in the array array_all reaches the preset number n, start to execute the write command of the device configuration database, write all the data in the cache into the device configuration database, and then clear the array array_all, and continue to perform the operations of reading data, caching data, and writing data, so as to realize the storage of historical production process data.
[0070] By adopting multi-threaded shared memory storage technology, not only high-frequency collection and synchronous update of real-time production process data is achieved, but also the reading rate of real-time production process data is improved. By adopting cache technology, high-frequency writing of equipment configuration database is avoided, and the consumption of computer resources for historical production process data storage is reduced.
[0071] Figure 3 FIG. 1 shows a software interface diagram for acquiring and displaying real-time production process data and real-time video monitoring data in one embodiment. Figure 3 As shown, a video playback and data monitoring software SGAI_VPC can be developed to obtain and display real-time production process data and real-time video monitoring data. The software can communicate with the monitoring equipment through the RTSP general protocol and play the audio and video in real time. For ball cameras with PTZ control functions, remote adjustment of viewing angle and focal length can be achieved, as well as video screenshots, video recording, and remote time calibration.
[0072] In some embodiments, real-time video monitoring data is obtained from a first monitoring device in a production area based on a VLC library and an FFmpeg tool; the real-time production process data in a shared memory block corresponding to the first monitoring device is determined as the real-time production process data corresponding to the real-time video monitoring data.
[0073] It is understandable that the traditional way of using opencv to obtain video streams has frame loss. When only the video stream is obtained, the memory occupies more than 70 MB for a long time, and the CPU occupies 5%-8%. However, when the VLC library is used to obtain the video stream through the FFmpeg tool, when the video and data are updated at the same time, the memory occupies 26 MB, the CPU occupies less than 1%, and the hardware resources consumed are significantly reduced.
[0074] Figure 4 FIG. 1 shows a software interface diagram for acquiring and displaying historical production process data and historical video monitoring data in one embodiment. Figure 4 As shown, a video playback and data monitoring software SGAI_VPB can be developed to obtain and display historical production process data and historical video monitoring data. The data source of historical production process data is usually a hard disk recorder and other devices. When the software is running, the historical video monitoring data of a certain time period can be obtained through the RTSP protocol, and the historical production process data stored in the device configuration database of the time period can be retrieved at the same time. The historical video monitoring data and the historical production process data are bound with time as the axis, so that the historical video monitoring data and the historical production process data of the same time period can be played back synchronously. During the playback process, the user can flexibly adjust the playback progress and pause to view the data.
[0075] In some embodiments, historical video monitoring data of a second monitoring device in a production area may be obtained according to the start and end times set by a user; and historical production process data of the same time period as the start and end times may be obtained from a device configuration database corresponding to the second monitoring device.
[0076] The user can set the start time and end time of historical video surveillance data in the software interface. The start time and end time are used as the start and end time, and the historical video surveillance data is called from the hard disk recorder according to the start and end time through the RTSP protocol.
[0077] In step 102, the real-time video monitoring data and the real-time production process data are integrated and displayed on a first display interface.
[0078] In some embodiments, the first point information can be obtained from the equipment list database according to the equipment information of the first monitoring device; the real-time production process data corresponding to the first point information and the real-time video monitoring data are displayed in the first area of the first display interface; the real-time video monitoring data is displayed in the second area of the first display interface; and the control controls of the real-time video monitoring data are displayed in the third control area of the first display interface.
[0079] Return to reference Figure 3The first preset interface is divided into left and right layouts. The first area on the left (the area corresponding to the real-time data in the figure) displays point information and real-time production process data, the third area (the area corresponding to the video control in the figure) displays control controls, and the second area displays real-time video monitoring data.
[0080] When the monitoring device is a ball camera with PTZ control, the ONVIF protocol and the "up", "down", "left", "right", "focus +", and "focus -" controls in the control controls can be used to control the PTZ movement and focus adjustment. The screenshot control in the control controls can be used to save the current image in the real-time video monitoring data to the local _snapshot folder and name it with time. The video recording control in the control controls can be used to record real-time video monitoring data and save it to the local _recorded_video directory.
[0081] During the implementation process, the data table corresponding to the first monitoring device can be queried from the device list database, and a data page table can be created in the first area according to the number of data in the data table. The data description of the data page table is the point information of each point, and the data unit is filled in the second row and second column of each point. After that, the real-time production process data is obtained from the shared memory block corresponding to the first monitoring device in real time, and the real-time production process data is updated to the first area.
[0082] The control control may also include a parameter configuration control, in which the user may enter the address of other monitoring devices to connect with the other monitoring devices. After connecting with the new monitoring device, the real-time video monitoring data is updated, the data page table is updated accordingly, and the real-time production process data of the new monitoring device is updated to the first area.
[0083] The embodiment of the present application realizes the integrated display of real-time video monitoring data and real-time production process data through steps 101 and 102.
[0084] In step 103, the historical video monitoring data and the historical production process data are integrated and displayed on the second display interface.
[0085] Return to reference Figure 4 The second preset interface is divided into left and right layouts. The first area on the left (the area corresponding to the data backtracking in the figure) displays the point information and historical production process data, the third area (the area corresponding to the video settings in the figure) displays the control controls, and the second area displays the historical video monitoring data and the playback progress bar.
[0086] In some embodiments, the second point information can be obtained from the corresponding equipment configuration database based on the equipment information of the second monitoring device; the second point information and the historical production process data of the same time period can be displayed in the first area of the second display interface; the historical video monitoring data can be displayed in the second area of the second display interface; and the control controls for the historical video monitoring data can be displayed in the third area of the second display interface.
[0087] The control controls include start time, end time, channel, video storage settings and other controls. According to the parameters set by the user in the control controls, the historical video monitoring data of the corresponding monitoring device in a certain period of time can be retrieved.
[0088] Taking the need to retrieve the historical video surveillance data of the second monitoring device as an example, during the implementation process, the point information of the second monitoring device can be queried from the device configuration database of the second monitoring device, and filled into the data page table of the first area according to the specified format.
[0089] It should be noted that for historical video surveillance data, how to bind the historical production process data with it so that it can be displayed synchronously and achieve the effect of dragging the playback progress bar and synchronously updating the historical production process data is one of the difficulties.
[0090] In some embodiments, the update frequency of the playback progress bar of historical video surveillance data can be determined based on a preset sampling period; the number of updates to the playback progress bar can be determined based on the start and end times and the update frequency; the total amount of data in the historical production process data of the same time period is divided by the number of updates to obtain the amount of data that needs to be displayed each time the playback progress bar is updated; when the playback progress bar is updated, a corresponding amount of data is obtained from the historical production process data of the same time period and displayed in the first area of the second display interface.
[0091] It is understandable that the playback progress bar in the second area is updated as the video is played, and the playback progress of the historical video monitoring data can be controlled by adjusting the playback progress bar. The update frequency of the playback progress bar is closely related to the data binding. After continuous research and testing, the update frequency of the playback progress bar is changed to 50ms / time, which can ensure that the historical video monitoring data is played smoothly while the update of the historical production process data is not lost.
[0092] After acquiring the historical production process data of the same time period, a shared memory block can be created and the data can be loaded into the shared memory block, so that the historical production process data can be read and updated at a high frequency during the playback of historical video surveillance data.
[0093] Since the historical video surveillance data is played in the form of video frames, the playback time of each frame is extremely short, while the sampling period of the production process data is relatively long, for example, it can be 100ms. Therefore, in order to achieve smooth playback of the historical video surveillance data while the historical production process data can be synchronously bound according to the sampling period, a playback progress bar that is updated every 50ms can be generated for the historical video surveillance data (that is, the update frequency is 50ms / time). Every time the progress of the historical video surveillance data is updated, the historical production process data is updated once.
[0094] The duration of the historical video surveillance data can be determined based on the start and end time, and the duration can be converted into unix_time in millimeter units, that is, converted into an integer value. For example, the historical video surveillance data is a 3-minute video from 2024-01-01 00:00:00.000 to 2024-01-01 00:03:00.000, with a start value of 0 and an end value of 180000.
[0095] Divide the duration 180000ms by the frequency of the play progress bar 50ms / time, and you can get the update times of the play progress bar b = 3600 times. Assuming that the total number of data in the historical production process data of the same time period is a, the number of data that needs to be displayed each time the play progress bar is updated is k = b / a. When the play progress bar is updated, the index number of the data in the historical production process data can be determined using round(current_position / k) = index, where current_position is the current update times of the play progress bar, round() is rounded to the integer after the decimal point, and index is the index number of the data in the historical production process data.
[0096] During the playback of historical video surveillance data, each time the update number of the playback progress bar changes, the index number of the data is calculated, and the data is taken out from the historical production process data according to the index number, and the value corresponding to the second point information in the first area is updated according to the corresponding relationship, so that the binding playback of historical video surveillance data and historical production process data is realized. At this time, whether the historical video surveillance data is fast-forwarded or rewinded or the playback progress bar is dragged to any position, the historical production process data can be accurately updated to the value corresponding to the playback time of the historical video surveillance data.
[0097] During the playback of historical video surveillance data, you can also add keyboard and mouse monitoring events. When the user clicks the screen with the mouse, the playback can be started / paused. When the user uses the left and right buttons on the keyboard, the playback can be moved forward / backward by 5 seconds each time.
[0098] When the software is closed, the connection with the video source is disconnected and the data in the cache and shared memory block are cleared.
[0099] The embodiment of the present application realizes the integrated display of historical video monitoring data and historical production process data through steps 101 and 103.
[0100] Figure 5 FIG. 1 shows a software interface diagram related to the event triggering function in one embodiment. Figure 5 As shown, an event trigger monitoring software E-Trigger can be developed so that when there is no host computer, the trigger conditions can be manually configured and the corresponding actions can be automatically executed after the trigger conditions are met. The software can be connected to the server through the opcua library. After connecting to the server, first load the configuration file in the local directory, and display the trigger conditions and program contents created in the configuration file in the configuration area. At the same time, the program obtains the point name in the trigger group under the Advanced Tags directory of the OPC UA server, and applies the point to the drop-down list of the trigger point. The reason for choosing to use the points under Advanced Tags is that the points under this function can perform various logical judgments, such as analog comparison, AND or NOT, and more complex logical relationships, and finally output 0 or 1 of the trigger condition.
[0101] In some embodiments, in response to a trigger point selected by a user, the production process data of the trigger point is monitored; when the production process data of the trigger point meets the preset trigger condition, the steps of acquiring real-time video surveillance data in the production area and its corresponding real-time production process data, historical video surveillance data in the production area and its corresponding historical production process data in the same time period are executed.
[0102] It is understandable that when a fault alarm occurs, the user can select the corresponding trigger point, and after selecting the program path to execute steps 101, 102 and 103, click the start control, the program will enable an independent thread to monitor the production process data changes of the trigger point in real time. When the production process data meets the preset trigger conditions, the real-time video surveillance data and real-time production process data can be displayed in a fusion by executing steps 101 and 102. By executing steps 101 and 103, the historical video surveillance data and historical production process data within a period of time before the fault occurs can be retrieved for fusion display.
[0103] In the above method for fusing production process data with video surveillance data, real-time video surveillance data in the production area and its corresponding real-time production process data, historical video surveillance data in the production area and its corresponding historical production process data in the same time period are obtained; real-time video surveillance data and real-time production process data are fused and displayed on a first display interface; historical video surveillance data and historical production process data are fused and displayed on a second display interface. Through the above scheme, real-time production process data and real-time video surveillance data can be displayed simultaneously on the display interface, and historical production process data and historical video surveillance data of the same time period can also be synchronously displayed on the display interface, thereby improving the efficiency of production management and equipment maintenance.
[0104] The following describes an apparatus embodiment of the present application, which can be used to execute the method for fusing production process data with video surveillance data in the above-mentioned embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the embodiment of the method for fusing production process data with video surveillance data in the above-mentioned embodiment of the present application.
[0105] Figure 6 FIG. 1 is a block diagram of a device for fusing production process data with video surveillance data in one embodiment. Figure 6 As shown, the device for fusing production process data and video surveillance data of an embodiment of the present application includes: a data acquisition module 601, a real-time data fusion module 602 and a historical data fusion module 603, wherein the data acquisition module 601 is used to acquire real-time video surveillance data in a production area and its corresponding real-time production process data, historical video surveillance data in the production area and its corresponding historical production process data in the same time period; the real-time data fusion module 602 is used to fuse the real-time video surveillance data and the real-time production process data on a first display interface; the historical data fusion module 603 is used to fuse the historical video surveillance data and the historical production process data on a second display interface.
[0106] In some embodiments, the data acquisition module 601 is also used to update the equipment list database in response to the user's operation of enabling the real-time data service, wherein the equipment list database includes the equipment information of all monitoring devices in the production area and the point information of all points monitored by each monitoring device; configure a shared memory block for each monitoring device; and store the real-time production process data of all points monitored by each monitoring device in the corresponding shared memory block according to a preset sampling period.
[0107] In some embodiments, the data acquisition module 601 is also used to acquire real-time video monitoring data from a first monitoring device in the production area based on the VLC library and the FFmpeg tool; and determine the real-time production process data in the shared memory block corresponding to the first monitoring device as the real-time production process data corresponding to the real-time video monitoring data.
[0108] In some embodiments, the real-time data fusion module 602 is also used to obtain first point information from the equipment list database based on the equipment information of the first monitoring device; display the real-time production process data corresponding to the first point information and the real-time video monitoring data in the first area of the first display interface; display the real-time video monitoring data in the second area of the first display interface; and display the control controls of the real-time video monitoring data in the third control area of the first display interface.
[0109] In some embodiments, the data acquisition module 601 is also used to create a device configuration database for each monitoring device in the production area in response to a user's operation of enabling the historical data service, and the device configuration database includes the point information of all points monitored by the corresponding monitoring device; for each monitoring device, data is read from the corresponding shared memory block and cached in an array; when the data in the array reaches a preset number, the data in the array is written into the corresponding device configuration database as historical production process data.
[0110] In some embodiments, the data acquisition module 601 is also used to acquire historical video monitoring data of a second monitoring device in the production area according to the start and end times set by the user; and to acquire historical production process data of the same time period as the start and end times from the device configuration database corresponding to the second monitoring device.
[0111] In some embodiments, the historical data fusion module 603 is also used to obtain second point information from the corresponding equipment configuration database based on the equipment information of the second monitoring equipment; display the second point information and the historical production process data of the same time period in the first area of the second display interface; display the historical video monitoring data in the second area of the second display interface; and display the control controls of the historical video monitoring data in the third area of the second display interface.
[0112] In some embodiments, the historical data fusion module 603 is also used to display the second point information in the first area of the second display interface; determine the update frequency of the playback progress bar of the historical video surveillance data according to a preset sampling period; determine the number of updates of the playback progress bar according to the start and end time and the update frequency; divide the total number of data in the historical production process data of the same time period by the number of updates to obtain the number of data that needs to be displayed each time the playback progress bar is updated; when the playback progress bar is updated, obtain a corresponding amount of data from the historical production process data of the same time period and display it in the first area of the second display interface.
[0113] In some embodiments, the device for fusing production process data with video surveillance data also includes an event monitoring module (not shown), which is used to monitor the production process data of the trigger point in response to the trigger point selected by the user; when the production process data of the trigger point meets the preset trigger condition, the steps of obtaining real-time video surveillance data in the production area and its corresponding real-time production process data, historical video surveillance data in the production area and its corresponding historical production process data in the same time period are executed.
[0114] Based on the same inventive concept, the embodiment of the present application also provides a device for fusing production process data with video surveillance data, referring to Figure 7 , showing a schematic diagram of the structure of the device for fusing production process data and video surveillance data in an embodiment of the present application, wherein the device for fusing production process data and video surveillance data comprises one or more memories 704, one or more processors 702, and at least one computer program (computer program instruction) stored in the memory 704 and executable on the processor 702, and the processor 702 implements the method described above when executing the computer program.
[0115] Among them, Figure 7 In the embodiment of the present invention, a bus architecture (represented by bus 700) is shown, which may include any number of interconnected buses and bridges, and bus 700 links various circuits including one or more processors represented by processor 702 and memory represented by memory 704. Bus 700 may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. Bus interface 705 provides an interface between bus 700 and receiver 701 and transmitter 703. Receiver 701 and transmitter 703 may be the same element, namely a transceiver, providing a unit for communicating with various other devices over a transmission medium. Processor 702 is responsible for managing bus 700 and general processing, while memory 704 may be used to store data used by processor 702 when performing operations.
[0116] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor is prompted to implement the steps of the method as described above.
[0117] The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored as one or more instructions or codes on a computer-readable medium or transmitted via a computer-readable medium. Other examples and implementations are within the scope and spirit of the present application and the appended claims. For example, due to the nature of software, the functions described above may be implemented using software executed by a processor, hardware, firmware, hard wiring, or a combination of any of these. In addition, each functional unit may be integrated into a processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit.
[0118] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0119] The units described as separate components may or may not be physically separated, and the components of the control device may or may not be physical units, that is, they may be located in one place or distributed in multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0120] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly or all or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., which can store computer program instructions.
[0121] The above description is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.
Claims
1. A method for fusing production process data with video surveillance data, characterized in that: include: Acquire real-time video surveillance data in the production area and its corresponding real-time production process data, and historical video surveillance data in the production area and its corresponding historical production process data in the same time period; The real-time video monitoring data and the real-time production process data are integrated and displayed on a first display interface; The historical video monitoring data and the historical production process data are integrated and displayed on the second display interface.
2. The method according to claim 1, characterized in that Also includes: In response to a user's operation of enabling the real-time data service, updating a device list database, wherein the device list database includes device information of all monitoring devices in the production area and location information of all locations monitored by each of the monitoring devices; Configuring a shared memory block for each of the monitoring devices; The real-time production process data of all points monitored by each monitoring device are stored in the corresponding shared memory block according to a preset sampling period.
3. The method according to claim 2, characterized in that The obtaining of real-time video monitoring data in the production area and its corresponding real-time production process data includes: Acquire the real-time video monitoring data from the first monitoring device in the production area based on the VLC library and the FFmpeg tool; The real-time production process data in the shared memory block corresponding to the first monitoring device is determined as the real-time production process data corresponding to the real-time video monitoring data.
4. The method according to claim 3, characterized in that The fusion display of the real-time video monitoring data and the real-time production process data on the first display interface includes: Acquire first point information from the device list database according to the device information of the first monitoring device; Displaying the first point information and real-time production process data corresponding to the real-time video monitoring data in the first area of the first display interface; Displaying the real-time video monitoring data in a second area of the first display interface; The control controls for the real-time video monitoring data are displayed in the third control area of the first display interface.
5. The method according to claim 2, characterized in that: Also includes: In response to a user's operation of enabling the historical data service, a device configuration database is created for each monitoring device in the production area, wherein the device configuration database includes point information of all points monitored by the corresponding monitoring device; For each monitoring device, read data from the corresponding shared memory block and cache it into the array; When the data in the array reaches a preset amount, the data in the array is written into a corresponding equipment configuration database as historical production process data.
6. The method according to claim 5, characterized in that Obtaining historical video surveillance data in the production area and historical production process data corresponding to the same time period, including: Acquire historical video monitoring data of a second monitoring device in the production area according to the start and end times set by the user; The historical production process data of the same time period as the start and end time is obtained from the equipment configuration database corresponding to the second monitoring equipment.
7. The method according to claim 6, characterized in that The fusing and displaying the historical video monitoring data and the historical production process data on the second display interface includes: Acquire second point information from a corresponding device configuration database according to the device information of the second monitoring device; Displaying the second point information and the historical production process data of the same time period in the first area of the second display interface; Displaying the historical video surveillance data in a second area of the second display interface; The control controls for the historical video surveillance data are displayed in the third area of the second display interface.
8. The method according to claim 7, characterized in that The displaying of the second point information and the historical production process data of the same time period in the second area of the second display interface includes: Displaying the second point information in the first area of the second display interface; Determining the update frequency of the playback progress bar of the historical video surveillance data according to the preset sampling period; Determine the update times of the playback progress bar according to the start and end times and the update frequency; Divide the total amount of data in the historical production process data of the same time period by the number of updates to obtain the amount of data that needs to be displayed each time the playback progress bar is updated; When the playback progress bar is updated, a corresponding amount of data is obtained from the historical production process data in the same time period and displayed in the first area of the second display interface.
9. The method according to any one of claims 1 to 8, characterized in that Also includes: In response to a trigger point selected by a user, monitoring the production process data of the trigger point; When the production process data at the trigger point meets the preset trigger conditions, the steps of obtaining the real-time video surveillance data in the production area and its corresponding real-time production process data, the historical video surveillance data in the production area and its corresponding historical production process data in the same time period are executed.
10. A device for fusing production process data with video surveillance data, comprising a processor and a memory, characterized in that: The memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, the steps of the method according to any one of claims 1 to 9 are implemented.