Intelligent filling supervision system and method for liquefied petroleum gas cylinders based on AI video
Through an intelligent filling supervision system based on AI video, the use of visual sensors and scene feature databases to monitor and analyze the filling process of liquefied petroleum gas cylinders in real time, solving the problem that existing systems cannot effectively supervise the filling site and improving the filling safety and work efficiency.
Patent Information
- Application Number
- CN202411186240.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-28
AI Technical Summary
The existing liquefied petroleum gas cylinder filling management system lacks effective on-site supervision and cannot monitor and control the filling process in real time, resulting in safety hazards such as non-professional personnel operation, failure to scan the code, and over-charge.
Adopt an intelligent filling supervision system based on AI video, monitor and analyze the filling video through visual sensors, and use the segmentation cycle and scene feature database to process it in blocks to judge abnormal situations during the filling process in real time, including personnel identity and scanning code operations and refill status, reduce misjudgment and improve safety.
Real-time monitoring and abnormal judgment of the filling process of liquefied petroleum gas cylinders is realized, filling safety is improved, mechanized judgment is reduced, humanized analysis is added, dangerous accidents caused by overcharge are avoided, and work efficiency is improved.
Smart Images

Figure CN118692015B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of video analysis, and in particular to an AI video-based intelligent filling supervision system and method for liquefied petroleum gas cylinders. Background Art
[0002] As a clean and efficient fuel, liquefied petroleum gas (LPG) is widely used in households, commerce and industry. However, due to the flammable and explosive characteristics of LPG, its use and management always face high safety requirements. In order to use liquefied petroleum gas safely, it is often filled in gas cylinders. In order to improve the efficiency and safety of filling, early filling management systems gradually introduced automation equipment, such as automatic weighing equipment, filling valve control systems, etc. These systems can partially replace manual operations and reduce the occurrence of human errors. With the development of information technology, in order to increase the filling efficiency, there is a QR code on each gas cylinder. After scanning the QR code, the filling time, version information, gas cylinder number, etc. of the corresponding gas cylinder will be obtained. The system fills the gas cylinder according to the QR code information, which not only improves the filling efficiency, but also eliminates the problems of random filling and multiple uses of one bottle; in the process of use, there are unqualified filling operations such as bypassing the system, not scanning the code for filling, bypassing the filling limit device, and amateurs fill. The existing gas cylinder filling traceability safety management platform only manages the upload of scanned code records before, during and after filling, but does not effectively control the filling site and lacks effective means to supervise on-site filling behavior at gas stations. Summary of the invention
[0003] The purpose of the present invention is to provide an AI video-based intelligent filling supervision system and method for liquefied petroleum gas cylinders to solve the problems raised in the prior art.
[0004] To achieve the above object, the present invention provides the following technical solutions:
[0005] An AI video-based intelligent filling supervision method for liquefied petroleum gas cylinders, the method comprising the following steps:
[0006] S100, using a visual sensor to monitor the filling process of the liquefied petroleum gas cylinder to generate a filling video, collecting historical working data of the filling of the liquefied petroleum gas cylinder, extracting the time when the working data changes, calculating a segmentation cycle, and using the segmentation cycle to segment the filling video to generate a status video;
[0007] Furthermore, the specific steps of using the segmentation cycle to segment the filling video to generate the status video are as follows:
[0008] S101. Install a visual sensor in the liquefied petroleum gas cylinder filling device, and use the visual sensor to monitor the filling process in real time to generate a filling video; collect historical working data when the liquefied petroleum gas cylinder is filled, and extract the time data when each working data changes. , Indicates the time data of the 1st, 2nd, 3rd, ..., nth types of work data changing in the extracted history, where n is a positive integer; calculates the average value of the time data of the n types of work data changing as the segmentation period tp;
[0009] S102, using the calculated segmentation period to segment the filling video generated by the visual sensor monitoring, assuming that the total time of the filling video generated by the visual sensor is tz, and calculating the number of state videos generated after segmentation, the formula is: In the formula, s represents the number of state videos after the filling video is segmented. It is determined whether the calculation result in the formula is divisible. When the calculation result of the formula is not divisible, the number of state videos after segmentation s is added by 1 and rounded up. Finally, the state video obtained after segmenting the filling video using the segmentation cycle is , Represents the 1st, 2nd, 3rd, ..., sth state videos obtained after segmenting the filling video.
[0010] The filling process is captured by visual sensors to generate a filling video. The filling video can be segmented to construct a status video, and the status video can be analyzed and judged separately. When judging abnormalities in the gas cylinder filling process, the segmented status video can be used to make separate judgments, which can achieve more accurate judgment of abnormalities. Different abnormality types can be judged using abnormalities in different status videos.
[0011] S200, collecting scene data of the filling process of liquefied petroleum gas cylinders by staff in history, extracting scene features at the beginning of filling, scene features at the end of filling, and scene features of gas cylinder code scanning; collecting facial data of all staff, and constructing a scene feature database and a face recognition database respectively;
[0012] Furthermore, the specific steps of building a filling database in combination with scene features are as follows:
[0013] S201. Collect the scene data of the staff at the beginning of filling the liquefied petroleum gas cylinder in the history, and extract the area of each functional component in the scene when the liquefied petroleum gas cylinder is filled. , It represents the scene area of the 1st, 2nd, 3rd, ..., jth functional component when the extracted liquefied petroleum gas cylinder is filled, where j is a positive integer; the jth scene areas are compared and the maximum value is selected, and the maximum area is squared to convert the irregular scene area into a square block. The square root processing formula is: , L represents the side length of the square block; determine the square block as the block unit un of the scene data, use the block unit to block the liquefied petroleum gas cylinder filling scene, and divide the liquefied petroleum gas cylinder filling scene into a scene matrix composed of block units. Let the matrix be ;
[0014] In the matrix, represents the block unit at row x and column y in the matrix, , Represents the constructed scene matrix, , A represents the number of rows of the matrix; B represents the number of columns of the matrix, ;
[0015] Extract the data values of each scenario before and at the beginning of filling of liquefied petroleum gas cylinders in the history, and calculate the difference of each scenario data before and at the beginning. The formula is: , in the formula, Represents the calculated data difference of each scene, extracts the maximum value of all scene data differences, and finds the block unit where the maximum data difference is located in the scene matrix. The scene characteristics at the beginning of filling are determined by using the data type and block unit of the maximum difference of scene data. , Indicates the position of the block unit in the scene matrix, ;
[0016] S202: Calculate the scene features at the end of filling by using the method in S201 above. Gas cylinder code scanning scene features , collect facial data of each staff member who fills liquefied petroleum gas cylinders; use the scene features at the beginning of filling, the scene features at the end of filling, and the scene features of gas cylinder scanning to build a scene feature database, and use the collected facial data of each staff member to build a face recognition database.
[0017] The filling scene is divided into blocks and a scene matrix is constructed. When extracting the scene features at the beginning of filling, the scene features at the end of filling, and the scene features of the gas cylinder scanning code, the scene data type that reflects the corresponding feature is found by finding the maximum value of the changed data, and the changing position of the data in the scene matrix is further determined. When using the corresponding scene features to judge the status video, the data type and matrix position can be used to determine whether it is abnormal, reducing the possibility of misjudgment of the system.
[0018] S300, when the staff is filling the liquefied petroleum gas cylinder, the filling video monitored by the visual sensor is analyzed, the real-time facial data of the operator in each state video in the filling video is extracted, the facial data of the staff collected in the face recognition database is compared with the real-time facial data, it is determined whether the operator in each state video is a staff member, and the state video is marked;
[0019] Furthermore, the specific steps for determining whether the operator in each status video is a staff member are as follows:
[0020] S301, collect the facial data of the operator in each state video , Represents the facial data of the 1st, 2nd, 3rd, ..., gth operators in each status video, where g is a positive integer; compares the facial data of the operators in the collected status video with the facial data in the face recognition database. When it is determined that the facial data of the operator in the status video does not exist in the face recognition database, the corresponding status video will be marked as abnormal.
[0021] The facial data of the operator in each status video is judged to find out whether there are non-professional staff performing the cylinder filling operation, so as to avoid the filling failure and danger caused by the illegal operation of non-professional staff;
[0022] S400, when the staff is filling the liquefied petroleum gas cylinder, the scene feature at the start of filling in the scene feature database is used to judge each state video in the real-time collected filling video, and the start state video of the real-time filling is obtained; the start state video is analyzed by using the gas cylinder code scanning scene feature to judge whether the staff performs a code scanning operation before filling the gas cylinder, and the start state video is marked;
[0023] Furthermore, the specific steps for determining whether the staff scans the code before filling the gas cylinder are as follows:
[0024] S401, extract the real-time difference of each scene data in two adjacent state videos, extract the maximum value of the real-time difference of all scene data, and find the block unit position of the real-time difference maximum value in the scene matrix to obtain the real-time state feature , Indicates the position of the block unit of the real-time state feature in the scene matrix;
[0025] S402, using the scene feature at the start of filling in the scene feature database to judge the real-time state feature of each two adjacent state videos, when the scene data type of the real-time state feature is the same as the scene feature type at the start of filling, and When , the latter state video of the two adjacent state videos is determined to be the start state video K;
[0026] S403, extracting scene data corresponding to the gas cylinder code scanning scene feature in the scene feature database from the start state video, and finding the position of the block unit of the extracted real-time code scanning scene data in the scene matrix. , using the scene feature database of gas cylinder code scanning scene features to determine the real-time location, When the operator is judging whether there is a code scanning operation, When the operator does not scan the code, it is determined that the operator does not perform the code scanning operation, and the start state video is marked as not scanned.
[0027] Before filling the gas cylinders, scan the code to obtain the filling information of each gas cylinder, and use the filling information to safely fill the gas cylinders. Use the scanning scene features in the scene feature database to judge the real-time start status video to find out whether there are any unscanned code operations, which can greatly avoid the occurrence of filling abnormalities.
[0028] S500. Before filling the gas cylinder, the staff scans the QR code on the gas cylinder and sends the filling time obtained after scanning to the filling control center. The filling control center uses the filling video to count the number of gas cylinders that need to be filled, and compares it with the number of filling times received, performs a second round of code scanning and judgment, and marks the filling video;
[0029] Furthermore, the specific steps for the second round of code scanning are as follows:
[0030] S501. Before filling the gas cylinder, the staff scans the QR code on the gas cylinder and sends the filling time obtained after the scan to the filling control center. The filling control center obtains the filling time of each filled gas cylinder as , represents the filling time of the 1st, 2nd, 3rd, ..., fth gas cylinder obtained by the filling control center, where f is a positive integer;
[0031] S502: Extract the number of gas cylinders in the filling video, and obtain the number of gas cylinders in the real-time scene as F. When F=f, it is determined that the operator has scanned the code of all gas cylinders; when F≠f, it is determined that there are gas cylinders that have not been scanned, and the filling video is marked as abnormal. The second round of scanning judgment further monitors the scanning operation and accurately finds each operation without scanning.
[0032] S600, using the scene features at the end of filling in the scene feature database to judge each state video in the filling video, and obtain the end state video; calculating the real-time filling time according to the start state video and the end state video, and comparing the real-time filling time with the filling time accepted by the filling control center to determine whether the gas cylinder is overfilled;
[0033] Further, the specific steps for determining whether the gas cylinder is overfilled are:
[0034] S601, using the method described in S402 to judge each state video in the filling video, find out the end state video J, and use the start state video and the end state video in the found filling video to construct a gas cylinder filling process; extract the number of state videos existing between the start state video and the end state video as N, and calculate the real-time gas cylinder filling time, the formula is: , where Ts represents the real-time cylinder filling time;
[0035] S602: Use the filling time of each gas cylinder obtained by the filling control center to determine the real-time filling time. When the liquefied petroleum gas cylinder is overfilled, When the liquefied petroleum gas cylinder is not full, the overfilled and underfilled cylinder filling videos are marked; when When the liquefied petroleum gas cylinder is filled normally.
[0036] S700. The filling control center sends the marked filling video to the supervisor. After the supervisor reviews each mark, he / she cancels and adds marks to generate a review video. The filling control center conducts a comprehensive analysis of the review video to determine whether the filling process is qualified.
[0037] The real-time filling time is calculated using the start status video and the end status video, and the real-time filling time is judged using the cylinder filling time obtained by scanning the code to find overfilling and underfilling anomalies; this increases the filling efficiency and avoids dangerous accidents caused by overfilling.
[0038] Furthermore, the specific steps to determine whether the filling process is compliant are:
[0039] S701. Collect the number of abnormalities in the normal filling of liquefied petroleum gas cylinders in the history. The abnormalities include cylinders not scanning codes, overfilling, incomplete filling, and personnel abnormalities. Suppose the number of abnormalities in the normal filling collected in the history is , represents the number of abnormalities during the 1st, 2nd, 3rd, ..., rth normal filling in the collected history, where r is a positive integer; the average value of the number of r abnormalities is calculated as the abnormal threshold ay;
[0040] S702, the filling control center sends the marked filling video to the supervisor, who reviews each mark and cancels and adds marks to generate a review video. Let the number of marks in the review video be E. When the filling process is judged to be unqualified; when When the filling process is qualified.
[0041] After the automated system finishes processing the filling video, the supervisor will further review the filling video for abnormalities, reducing the system's mechanized judgment of the filling process and increasing its humanized judgment. The supervisor will then conduct a comprehensive analysis of the reviewed video to determine whether the filling process is qualified. If it is judged to be unqualified, the supervisor will optimize the filling process to reduce the occurrence of abnormalities and avoid unnecessary economic losses.
[0042] The intelligent filling supervision system of liquefied petroleum gas cylinders based on AI video includes a data collection module, a filling video segmentation module, a database construction module, a filling abnormality analysis module and a comprehensive judgment module;
[0043] The data collection module is used to collect scene data, time data and the number of abnormalities when filling the liquefied petroleum gas cylinder in history;
[0044] The filling video segmentation module is used to calculate the segmentation period according to the change time of the working data when the gas cylinder is filled in the history, and use the segmentation period to segment the filling video collected by the visual sensor to form a state video;
[0045] The database construction module is used to analyze the filling scene data, generate block units, construct a scene matrix using the block units for the filling scene, calculate the scene data difference and the position in the matrix to generate the scene features at the beginning of filling, the scene features at the end of filling and the scene features of the gas cylinder code scanning to construct a scene feature database, and collect the facial data of the staff to construct a face recognition database;
[0046] The filling abnormality analysis module is used to use the feature data in the scene feature database and the face recognition database to determine in real time whether there is a personnel abnormality, charging without scanning the code, overcharging and incomplete charging in each status video, and mark them;
[0047] The comprehensive judgment module is used for the staff to review the marked filling video, cancel the mark and add the mark to generate the review video, and conduct a comprehensive analysis on the review video to judge whether the filling process is qualified.
[0048] The filling abnormality analysis module includes a personnel abnormality unit, a code scanning analysis unit and a filling time analysis unit;
[0049] The personnel abnormality unit is used to determine whether the real-time operator is a staff member by using the facial data of the staff member in the face recognition database;
[0050] The code scanning analysis unit is used to use the code scanning features in the scene feature database to determine whether the operator in the status video has scanned the code, and then compare the number of code scans received by the filling control center with the number of gas cylinders in the filling video to perform a second round of code scanning judgment;
[0051] The filling time analysis unit is used to calculate the real-time filling time of a gas cylinder based on the filling start video and the filling end video, and use the gas cylinder filling time received by the filling control center to determine whether the real-time filling is full or overfilled.
[0052] Compared with the prior art, the present invention has the following beneficial effects:
[0053] The present invention utilizes visual sensors to collect filling videos of the liquefied petroleum gas cylinder filling process, and performs artificial intelligence automated AI analysis on the filling videos to determine whether there are problems such as personnel abnormalities, failure to scan codes, overcharging and underfilling during the filling process. Automated means are used to achieve real-time monitoring of the filling process, thereby avoiding the problem of being unable to promptly discover abnormalities in the filling process, and greatly improving the safety of filling.
[0054] 2. The present invention performs segmentation processing on the filling video and block processing on the filling scene, which increases the accuracy of abnormal judgment in the working process, avoids misjudgment of abnormalities by the automation system, and increases the trust of the system among the staff.
[0055] 3. After analyzing the filling process using the automated system, the present invention allows supervisors to conduct further review, which reduces the mechanization of the system's judgment of the filling process and increases humanized judgment; then, a comprehensive analysis is conducted on the reviewed video to determine whether the filling process is qualified. If it is determined to be unqualified, the filling process is optimized to reduce the occurrence of abnormalities and avoid unnecessary economic losses. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] Figure 1 This is a module distribution diagram of the AI video-based liquefied petroleum gas cylinder intelligent filling supervision system of the present invention;
[0057] Figure 2 This is a schematic diagram of the steps of the AI video-based intelligent filling supervision method for liquefied petroleum gas cylinders of the present invention. DETAILED DESCRIPTION
[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0059] Example: Figure 1-Figure 2 As shown, the present invention provides a technical solution.
[0060] An AI video-based intelligent filling supervision method for liquefied petroleum gas cylinders, the method comprising the following steps:
[0061] S100, using a visual sensor to monitor the filling process of the liquefied petroleum gas cylinder to generate a filling video, collecting historical working data of the filling of the liquefied petroleum gas cylinder, extracting the time when the working data changes, calculating a segmentation cycle, and using the segmentation cycle to segment the filling video to generate a status video;
[0062] The specific steps of using the segmentation cycle to segment the filling video and generate the status video are as follows:
[0063] S101. Install a visual sensor in the liquefied petroleum gas cylinder filling device, and use the visual sensor to monitor the filling process in real time to generate a filling video; collect historical working data when the liquefied petroleum gas cylinder is filled, and extract the time data when each working data changes. , Indicates the time data of the 1st, 2nd, 3rd, ..., nth types of work data changing in the extracted history, where n is a positive integer; calculates the average value of the time data of the n types of work data changing as the segmentation period tp;
[0064] S102, using the calculated segmentation period to segment the filling video generated by the visual sensor monitoring, assuming that the total time of the filling video generated by the visual sensor is tz, and calculating the number of state videos generated after segmentation, the formula is: In the formula, s represents the number of state videos after the filling video is segmented. It is determined whether the calculation result in the formula is divisible. When the calculation result of the formula is not divisible, the number of state videos after segmentation s is added by 1 and rounded up. Finally, the state video obtained after the filling video is segmented using the segmentation cycle is , Represents the 1st, 2nd, 3rd, ..., sth state videos obtained after segmenting the filling video.
[0065] The filling process is captured by visual sensors to generate a filling video. The filling video can be segmented to construct a status video, and the status video can be analyzed and judged separately. When judging abnormalities in the gas cylinder filling process, the segmented status video can be used to make separate judgments, which can achieve more accurate judgment of abnormalities. Different abnormality types can be judged using abnormalities in different status videos.
[0066] For example, a certain section of filling video is segmented. According to the time data of the historical work data changes, the segmentation period is calculated to be 2 minutes. Assuming that the real-time filling video is 37 minutes, the number of status videos after segmentation is calculated. The formula result is 18.5; there is a remainder, so it is rounded up to 19; the number of status videos after segmentation is 19.
[0067] S200, collecting scene data of the filling process of liquefied petroleum gas cylinders by staff in history, extracting scene features at the beginning of filling, scene features at the end of filling, and scene features of gas cylinder code scanning; collecting facial data of all staff, and constructing a scene feature database and a face recognition database respectively;
[0068] The specific steps to build a filling database based on scene features are:
[0069] S201. Collect the scene data of the staff at the beginning of filling the liquefied petroleum gas cylinder in the history, and extract the area of each functional component in the scene when the liquefied petroleum gas cylinder is filled. , It represents the scene area of the 1st, 2nd, 3rd, ..., jth functional component when the extracted liquefied petroleum gas cylinder is filled, where j is a positive integer; the jth scene areas are compared and the maximum value is selected, and the maximum area is squared to convert the irregular scene area into a square block. The square root processing formula is: , L represents the side length of the square block; determine the square block as the block unit un of the scene data, use the block unit to block the liquefied petroleum gas cylinder filling scene, and divide the liquefied petroleum gas cylinder filling scene into a scene matrix composed of block units. Let the matrix be ;
[0070] In the matrix, represents the block unit at row x and column y in the matrix, , Represents the constructed scene matrix, , A represents the number of rows of the matrix; B represents the number of columns of the matrix, ;
[0071] Extract the data values of each scenario before and at the beginning of filling of liquefied petroleum gas cylinders in the history, and calculate the difference of each scenario data before and at the beginning. The formula is: , in the formula, Represents the calculated data difference of each scene, extracts the maximum value of all scene data differences, and finds the block unit where the maximum data difference is located in the scene matrix. The scene characteristics at the beginning of filling are determined by using the data type and block unit of the maximum difference of scene data. , Indicates the position of the block unit in the scene matrix, ;
[0072] The filling scene is divided into blocks. According to the scene area of each function in the history and after square root processing, the side length of the block unit is obtained as 1m. Assuming that the length and width of the existing filling scene are 20 and 15, the scene is divided into 300 blocks, A=15, B=20; the scene data at the beginning of filling is analyzed, and the data type with the largest change is the displacement of the gas cylinder, and the displacement data of the gas cylinder changes in the 3rd row and 8th column of the matrix, which constitutes the scene characteristics at the beginning of filling; the scene characteristics at the end of filling and the scene characteristics of the gas cylinder scanning code are analyzed in turn.
[0073] S202: Calculate the scene features at the end of filling by using the method in S201 above. Gas cylinder code scanning scene features , collect facial data of each staff member who fills liquefied petroleum gas cylinders; use the scene features at the beginning of filling, the scene features at the end of filling, and the scene features of gas cylinder scanning to build a scene feature database, and use the collected facial data of each staff member to build a face recognition database.
[0074] The filling scene is divided into blocks and a scene matrix is constructed. When extracting the scene features at the beginning of filling, the scene features at the end of filling, and the scene features of the gas cylinder scanning code, the scene data type that reflects the corresponding feature is found by finding the maximum value of the changed data, and the changing position of the data in the scene matrix is further determined. When using the corresponding scene features to judge the status video, the data type and matrix position can be used to determine whether it is abnormal, reducing the possibility of misjudgment of the system.
[0075] S300, when the staff is filling the liquefied petroleum gas cylinder, the filling video monitored by the visual sensor is analyzed, the real-time facial data of the operator in each state video in the filling video is extracted, the facial data of the staff collected in the face recognition database is compared with the real-time facial data, it is determined whether the operator in each state video is a staff member, and the state video is marked;
[0076] The specific steps for determining whether the operator in each status video is a staff member are as follows:
[0077] S301, collect the facial data of the operator in each state video , Represents the facial data of the 1st, 2nd, 3rd, ..., gth operators in each status video, where g is a positive integer; compares the facial data of the operators in the collected status video with the facial data in the face recognition database. When it is determined that the facial data of the operator in the status video does not exist in the face recognition database, the corresponding status video will be marked as abnormal.
[0078] The facial data of the operator in each status video is judged to find out whether there are non-professional staff performing the cylinder filling operation, so as to avoid the filling failure and danger caused by the illegal operation of non-professional staff;
[0079] S400, when the staff is filling the liquefied petroleum gas cylinder, the scene feature at the start of filling in the scene feature database is used to judge each state video in the real-time collected filling video, and the start state video of the real-time filling is obtained; the start state video is analyzed by using the gas cylinder code scanning scene feature to judge whether the staff performs a code scanning operation before filling the gas cylinder, and the start state video is marked;
[0080] The specific steps to determine whether the staff scans the code before filling the gas cylinder are as follows:
[0081] S401, extract the real-time difference of each scene data in two adjacent state videos, extract the maximum value of the real-time difference of all scene data, and find the block unit position of the real-time difference maximum value in the scene matrix to obtain the real-time state feature , Indicates the position of the block unit of the real-time state feature in the scene matrix;
[0082] S402, using the scene feature at the start of filling in the scene feature database to judge the real-time state feature of each two adjacent state videos, when the scene data type of the real-time state feature is the same as the scene feature type at the start of filling, and When , the latter state video of the two adjacent state videos is determined to be the start state video K;
[0083] S403, extracting scene data corresponding to the gas cylinder code scanning scene feature in the scene feature database from the start state video, and finding the position of the block unit of the extracted real-time code scanning scene data in the scene matrix. , using the scene feature database of gas cylinder code scanning scene features to determine the real-time location, When the operator is judging whether there is a code scanning operation, When the operator does not scan the code, it is determined that the operator does not perform the code scanning operation, and the start state video is marked as not scanned.
[0084] Before filling the gas cylinders, scan the code to obtain the filling information of each gas cylinder, and use the filling information to safely fill the gas cylinders. Use the scanning scene features in the scene feature database to judge the real-time start status video to find out whether there are any unscanned code operations, which can greatly avoid the occurrence of filling abnormalities.
[0085] S500. Before filling the gas cylinder, the staff scans the QR code on the gas cylinder and sends the filling time obtained after scanning to the filling control center. The filling control center uses the filling video to count the number of gas cylinders that need to be filled, and compares it with the number of filling times received, performs a second round of code scanning and judgment, and marks the filling video;
[0086] The specific steps for the second round of code scanning are as follows:
[0087] S501. Before filling the gas cylinder, the staff scans the QR code on the gas cylinder and sends the filling time obtained after the scan to the filling control center. The filling control center obtains the filling time of each filled gas cylinder as , represents the filling time of the 1st, 2nd, 3rd, ..., fth gas cylinder obtained by the filling control center, where f is a positive integer;
[0088] S502: Extract the number of gas cylinders in the filling video, and obtain the number of gas cylinders in the real-time scene as F. When F=f, it is determined that the operator has scanned the code of all gas cylinders; when F≠f, it is determined that there are gas cylinders that have not been scanned, and the filling video is marked as abnormal. The second round of scanning judgment further monitors the scanning operation and accurately finds each operation without scanning.
[0089] S600, using the scene features at the end of filling in the scene feature database to judge each state video in the filling video, and obtain the end state video; calculating the real-time filling time according to the start state video and the end state video, and comparing the real-time filling time with the filling time accepted by the filling control center to determine whether the gas cylinder is overfilled;
[0090] The specific steps to determine whether the gas cylinder is overfilled are:
[0091] S601, using the method described in S402 to judge each state video in the filling video, find out the end state video J, and use the start state video and the end state video in the found filling video to construct a gas cylinder filling process; extract the number of state videos existing between the start state video and the end state video as N, and calculate the real-time gas cylinder filling time, the formula is: , where Ts represents the real-time cylinder filling time;
[0092] S602: Use the filling time of each gas cylinder obtained by the filling control center to determine the real-time filling time. When the liquefied petroleum gas cylinder is overfilled, When the liquefied petroleum gas cylinder is not full, the overfilled and underfilled cylinder filling videos are marked; when When the liquefied petroleum gas cylinder is filled normally.
[0093] S700. The filling control center sends the marked filling video to the supervisor. After the supervisor reviews each mark, he / she cancels and adds marks to generate a review video. The filling control center conducts a comprehensive analysis of the review video to determine whether the filling process is qualified.
[0094] The real-time filling time is calculated using the start status video and the end status video, and the real-time filling time is judged using the cylinder filling time obtained by scanning the code to find overfilling and underfilling anomalies; this increases the filling efficiency and avoids dangerous accidents caused by overfilling.
[0095] The specific steps to determine whether the filling process is compliant are:
[0096] S701. Collect the number of abnormalities in the normal filling of liquefied petroleum gas cylinders in the history. The abnormalities include cylinders not scanning codes, overfilling, incomplete filling, and personnel abnormalities. Suppose the number of abnormalities in the normal filling collected in the history is , represents the number of abnormalities during the 1st, 2nd, 3rd, ..., rth normal filling in the collected history, where r is a positive integer; the average value of the number of r abnormalities is calculated as the abnormal threshold ay;
[0097] S702, the filling control center sends the marked filling video to the supervisor, who reviews each mark and cancels and adds marks to generate a review video. Let the number of marks in the review video be E. When the filling process is judged to be unqualified; when When the filling process is qualified.
[0098] After the automated system finishes processing the filling video, the supervisor will further review the filling video for abnormalities, reducing the system's mechanized judgment of the filling process and increasing its humanized judgment. The supervisor will then conduct a comprehensive analysis of the reviewed video to determine whether the filling process is qualified. If it is judged to be unqualified, the supervisor will optimize the filling process to reduce the occurrence of abnormalities and avoid unnecessary economic losses.
[0099] The intelligent filling supervision system of liquefied petroleum gas cylinders based on AI video includes a data collection module, a filling video segmentation module, a database construction module, a filling abnormality analysis module and a comprehensive judgment module;
[0100] The data collection module is used to collect scene data, time data and the number of abnormalities when filling the liquefied petroleum gas cylinder in history;
[0101] The filling video segmentation module is used to calculate the segmentation period according to the change time of the working data when the gas cylinder is filled in the history, and use the segmentation period to segment the filling video collected by the visual sensor to form a state video;
[0102] The database construction module is used to analyze the filling scene data, generate block units, construct a scene matrix using the block units for the filling scene, calculate the scene data difference and the position in the matrix to generate the scene features at the beginning of filling, the scene features at the end of filling and the scene features of the gas cylinder code scanning to construct a scene feature database, and collect the facial data of the staff to construct a face recognition database;
[0103] The filling abnormality analysis module is used to use the feature data in the scene feature database and the face recognition database to determine in real time whether there is a personnel abnormality, charging without scanning the code, overcharging and incomplete charging in each status video, and mark them;
[0104] The comprehensive judgment module is used for the staff to review the marked filling video, cancel the mark and add the mark to generate the review video, and conduct a comprehensive analysis on the review video to judge whether the filling process is qualified.
[0105] The filling abnormality analysis module includes a personnel abnormality unit, a code scanning analysis unit and a filling time analysis unit;
[0106] The personnel abnormality unit is used to determine whether the real-time operator is a staff member by using the facial data of the staff member in the face recognition database;
[0107] The code scanning analysis unit is used to use the code scanning features in the scene feature database to determine whether the operator in the status video has scanned the code, and then compare the number of code scans received by the filling control center with the number of gas cylinders in the filling video to perform a second round of code scanning judgment;
[0108] The filling time analysis unit is used to calculate the real-time filling time of a gas cylinder based on the filling start video and the filling end video, and use the gas cylinder filling time received by the filling control center to determine whether the real-time filling is full or overfilled.
[0109] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.
Claims
1. An AI video-based intelligent filling supervision method for liquefied petroleum gas cylinders, characterized by: The method comprises the following steps: S100, using a visual sensor to monitor the filling process of the liquefied petroleum gas cylinder to generate a filling video, collecting historical working data of the filling of the liquefied petroleum gas cylinder, extracting the time when the working data changes, calculating a segmentation cycle, and using the segmentation cycle to segment the filling video to generate a status video; S200, collecting scene data of the filling process of liquefied petroleum gas cylinders by staff in history, extracting scene features at the beginning of filling, scene features at the end of filling, and scene features of gas cylinder code scanning; collecting facial data of all staff, and constructing a scene feature database and a face recognition database respectively; S300, when the staff is filling the liquefied petroleum gas cylinder, the filling video monitored by the visual sensor is analyzed, the real-time facial data of the operator in each state video in the filling video is extracted, the facial data of the staff collected in the face recognition database is compared with the real-time facial data, it is determined whether the operator in each state video is a staff member, and the state video is marked; S400, when the staff is filling the liquefied petroleum gas cylinder, the scene feature at the start of filling in the scene feature database is used to judge each state video in the real-time collected filling video, and the start state video of the real-time filling is obtained; the start state video is analyzed by using the gas cylinder code scanning scene feature to judge whether the staff performs a code scanning operation before filling the gas cylinder, and the start state video is marked; S500. Before filling the gas cylinder, the staff scans the QR code on the gas cylinder and sends the filling time obtained after scanning to the filling control center. The filling control center uses the filling video to count the number of gas cylinders that need to be filled, and compares it with the number of filling times received, performs a second round of code scanning and judgment, and marks the filling video; S600, using the scene features at the end of filling in the scene feature database to judge each state video in the filling video, and obtain the end state video; calculating the real-time filling time according to the start state video and the end state video, and comparing the real-time filling time with the filling time accepted by the filling control center to determine whether the gas cylinder is overfilled; S700. The filling control center sends the marked filling video to the supervisor. After the supervisor reviews each mark, he / she cancels and adds marks to generate a review video. The filling control center conducts a comprehensive analysis of the review video to determine whether the filling process is qualified.
2. The AI video-based intelligent filling supervision method for liquefied petroleum gas cylinders according to claim 1 is characterized in that: The specific steps of using the segmentation cycle to segment the filling video to generate the status video in S100 are: S101. Install a visual sensor in the liquefied petroleum gas cylinder filling device, and use the visual sensor to monitor the filling process in real time to generate a filling video; collect historical working data when the liquefied petroleum gas cylinder is filled, and extract the time data when each working data changes. , Indicates the time data of the 1st, 2nd, 3rd, ..., nth types of work data changing in the extracted history, where n is a positive integer; calculates the average value of the time data of the n types of work data changing as the segmentation period tp; S102, using the calculated segmentation period to segment the filling video generated by the visual sensor monitoring, assuming that the total time of the filling video generated by the visual sensor is tz, and calculating the number of state videos generated after segmentation, the formula is: In the formula, s represents the number of state videos after the filling video is segmented. It is determined whether the calculation result in the formula is divisible. When the calculation result of the formula is not divisible, the number of state videos after segmentation s is added by 1 and rounded up. Finally, the state video obtained after the filling video is segmented using the segmentation cycle is , Represents the 1st, 2nd, 3rd, ..., sth state videos obtained after segmenting the filling video.
3. The AI video-based intelligent filling supervision method for liquefied petroleum gas cylinders according to claim 1 is characterized in that: The specific steps of constructing the filling database in combination with the scene features in S200 are: S201. Collect the scene data of the staff at the beginning of filling the liquefied petroleum gas cylinder in the history, and extract the area of each functional component in the scene when the liquefied petroleum gas cylinder is filled. , It represents the scene area of the 1st, 2nd, 3rd, ..., jth functional component when the extracted liquefied petroleum gas cylinder is filled, where j is a positive integer; the jth scene areas are compared and the maximum value is selected, and the maximum area is squared to convert the irregular scene area into a square block. The square root processing formula is: , L represents the side length of the square block; determine the square block as the block unit un of the scene data, use the block unit to block the liquefied petroleum gas cylinder filling scene, and divide the liquefied petroleum gas cylinder filling scene into a scene matrix composed of block units. Let the matrix be ; In the matrix, represents the block unit at row x and column y in the matrix, , Represents the constructed scene matrix, , A represents the number of rows of the matrix; B represents the number of columns of the matrix, ; Extract the data values of each scenario before and at the beginning of filling of liquefied petroleum gas cylinders in the history, and calculate the difference of each scenario data before and at the beginning. The formula is: , in the formula, Represents the calculated data difference of each scene, extracts the maximum value of all scene data differences, and finds the block unit where the maximum data difference is located in the scene matrix. The scene characteristics at the beginning of filling are determined by using the data type and block unit of the maximum difference of scene data. , Indicates the position of the block unit in the scene matrix, ; S202: Calculate the scene features at the end of filling by using the method in S201 above. Gas cylinder code scanning scene features , collect facial data of each staff member who fills liquefied petroleum gas cylinders; use the scene features at the beginning of filling, the scene features at the end of filling, and the scene features of gas cylinder scanning to build a scene feature database, and use the collected facial data of each staff member to build a face recognition database.
4. The AI video-based intelligent filling supervision method for liquefied petroleum gas cylinders according to claim 3 is characterized in that: The specific steps of determining whether the operator in each status video is a staff member in S300 are: S301, collect the facial data of the operator in each state video , Represents the facial data of the 1st, 2nd, 3rd, ..., gth operators in each status video, where g is a positive integer; compares the facial data of the operators in the collected status video with the facial data in the face recognition database. When it is determined that the facial data of the operator in the status video does not exist in the face recognition database, the corresponding status video will be marked as abnormal.
5. The AI video-based intelligent filling supervision method for liquefied petroleum gas cylinders according to claim 3 is characterized in that: The specific steps of determining whether the staff scans the code before filling the gas cylinder in S400 are: S401, extract the real-time difference of each scene data in two adjacent state videos, extract the maximum value of the real-time difference of all scene data, and find the block unit position of the real-time difference maximum value in the scene matrix to obtain the real-time state feature , Indicates the position of the block unit of the real-time state feature in the scene matrix; S402, using the scene feature at the start of filling in the scene feature database to judge the real-time state feature of each two adjacent state videos, when the scene data type of the real-time state feature is the same as the scene feature type at the start of filling, and When , the latter state video of the two adjacent state videos is determined to be the start state video K; S403, extracting scene data corresponding to the gas cylinder code scanning scene feature in the scene feature database from the start state video, and finding the position of the block unit of the extracted real-time code scanning scene data in the scene matrix. , using the scene feature database of gas cylinder code scanning scene features to determine the real-time location, When the operator is judging whether there is a code scanning operation, When the operator does not scan the code, it is determined that the operator does not perform the code scanning operation, and the start state video is marked as not scanned.
6. The AI video-based intelligent filling supervision method for liquefied petroleum gas cylinders according to claim 3 is characterized in that: The specific steps of performing the second round of code scanning judgment in S500 are: S501. Before filling the gas cylinder, the staff scans the QR code on the gas cylinder and sends the filling time obtained after the scan to the filling control center. The filling control center obtains the filling time of each filled gas cylinder as , represents the filling time of the 1st, 2nd, 3rd, ..., fth gas cylinder obtained by the filling control center, where f is a positive integer; S502. Extract the number of gas cylinders in the filling video and obtain the number of gas cylinders in the real-time scene as F. When F=f, it is determined that the operator has scanned the codes of all gas cylinders. When F≠f, it is determined that there are gas cylinders that have not been scanned, and the filling video is marked as abnormal.
7. The AI video-based intelligent filling supervision method for liquefied petroleum gas cylinders according to claim 5 is characterized in that: The specific steps of determining whether the gas cylinder is overfilled in S600 are: S601, using the method described in S402 to judge each state video in the filling video, find out the end state video J, and use the start state video and the end state video in the found filling video to construct a gas cylinder filling process; extract the number of state videos existing between the start state video and the end state video as N, and calculate the real-time gas cylinder filling time, the formula is: , where Ts represents the real-time cylinder filling time; S602: Use the filling time of each gas cylinder obtained by the filling control center to determine the real-time filling time. When the liquefied petroleum gas cylinder is overfilled, When the liquefied petroleum gas cylinder is not full, the overfilled and underfilled cylinder filling videos are marked; when When the liquefied petroleum gas cylinder is filled normally.
8. The AI video-based intelligent filling supervision method for liquefied petroleum gas cylinders according to claim 7 is characterized in that: The specific steps of determining whether the filling process is compliant in S700 are: S701. Collect the number of abnormalities in the normal filling of liquefied petroleum gas cylinders in the history. The abnormalities include cylinders not scanning codes, overfilling, incomplete filling, and personnel abnormalities. Suppose the number of abnormalities in the normal filling collected in the history is , represents the number of abnormalities during the 1st, 2nd, 3rd, ..., rth normal filling in the collected history, where r is a positive integer; the average value of the number of abnormalities of r times is calculated as the abnormal threshold ay; S702, the filling control center sends the marked filling video to the supervisor, who reviews each mark and cancels and adds marks to generate a review video. Let the number of marks in the review video be E. When the filling process is judged to be unqualified; when When the filling process is qualified.
9. An AI video-based intelligent filling supervision system for liquefied petroleum gas cylinders using the AI video-based intelligent filling supervision method for liquefied petroleum gas cylinders according to any one of claims 1 to 8, characterized in that: The intelligent filling supervision system for liquefied petroleum gas cylinders includes a data collection module, a filling video segmentation module, a database construction module, a filling abnormality analysis module and a comprehensive judgment module; The data collection module is used to collect scene data, time data and the number of abnormalities when filling the liquefied petroleum gas cylinder in history; The filling video segmentation module is used to calculate the segmentation period according to the change time of the working data when the gas cylinder is filled in the history, and use the segmentation period to segment the filling video collected by the visual sensor to form a state video; The database construction module is used to analyze the filling scene data, generate block units, construct a scene matrix using the block units for the filling scene, calculate the scene data difference and the position in the matrix to generate the scene features at the beginning of filling, the scene features at the end of filling and the scene features of the gas cylinder code scanning to construct a scene feature database, and collect the facial data of the staff to construct a face recognition database; The filling abnormality analysis module is used to use the feature data in the scene feature database and the face recognition database to determine in real time whether there is a personnel abnormality, charging without scanning the code, overcharging and incomplete charging in each status video, and mark them; The comprehensive judgment module is used for the staff to review the marked filling video, cancel the mark and add the mark to generate the review video, and conduct a comprehensive analysis on the review video to judge whether the filling process is qualified.
10. The AI video-based intelligent filling monitoring system for liquefied petroleum gas cylinders according to claim 9 is characterized in that: The filling abnormality analysis module includes a personnel abnormality unit, a code scanning analysis unit and a filling time analysis unit; The personnel abnormality unit is used to determine whether the real-time operator is a staff member by using the facial data of the staff member in the face recognition database; The code scanning analysis unit is used to use the code scanning features in the scene feature database to determine whether the operator in the status video has scanned the code, and then compare the number of code scans received by the filling control center with the number of gas cylinders in the filling video to perform a second round of code scanning judgment; The filling time analysis unit is used to calculate the real-time filling time of a gas cylinder based on the filling start video and the filling end video, and use the gas cylinder filling time received by the filling control center to determine whether the real-time filling is full or overfilled.
