Workshop equipment code scanning inspection anti-cheating method, device and system

Through camera and image recognition technology, the location of inspectors is verified during workshop equipment inspection, and the problem of inspection cheating in dense equipment and multi-floor environments is solved, and the effectiveness verification is achieved without additional hardware is achieved, cost is reduced and increased Security.

CN118968416AActive Publication Date: 2024-11-15HENAN HANWEI ELECTRONICS
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411168394.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-11-15
Estimated Expiration
2044-08-23

AI Technical Summary

Technical Problem

The existing workshop equipment inspection methods have cheating problems in equipment-intensive and multi-floor environments, and require increased hardware configuration and increased costs, which cannot effectively prevent inspection cheating.

Method used

Camera and image recognition technology are used to obtain the position coordinates of equipment and personnel, and by calculating the area coincidence and distance difference, we can determine whether the inspector is near the equipment, so as to achieve inspection effectiveness verification without additional hardware.

Benefits of technology

It saves hardware configuration costs, is suitable for equipment-intensive and multi-floor workshop environments, effectively prevents inspection cheating, and improves the safety of equipment operations and equipment service life.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968416A_ABST
    Figure CN118968416A_ABST
Patent Text Reader

Abstract

The invention provides a workshop equipment code scanning inspection anti-cheating method, device and system. The method comprises the following steps: identifying an initial image acquired by a camera during unmanned inspection by using an image identification technology to obtain position coordinates of each piece of equipment to form a data set S; acquiring a plurality of shot images acquired by a camera in code scanning time, and respectively identifying position coordinates of the head and the human body of the inspection personnel in the shot images to obtain a head data set and a human body data set; calculating the area coincidence degree of the samples in the head data set and the human body data set, and mapping and matching inspection personnel according to the area coincidence degree; calculating the distance from the head of the inspector to the camera according to the width of the head in the head data set, and forming a sample set by screened shot images; and calculating the area overlap ratio of the samples in the human body data set matched with the sample mapping in the sample set and the equipment in the data set S, screening the samples, and judging whether the code scanning inspection of the inspector is effective or not. According to a picture target identification technology and a camera imaging principle, the inspection validity of equipment is inspected through distance measurement and calculation, and the behavior of code scanning cheating is prevented; and under the condition that a new hardware device is not additionally arranged, the cost expenditure is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of workshop inspection and the Internet of Things, and in particular to a method, device and system for preventing cheating during workshop equipment code scanning inspection. Background Art

[0002] With the development of technology, the application of machinery and equipment in various industries is becoming more and more extensive, and the participation of machinery and equipment is getting higher and higher, which greatly improves the production efficiency of enterprises. However, if equipment failure causes abnormal operation or shutdown, it will seriously affect the production process of the enterprise and even cause safety accidents. Therefore, most companies have equipment inspection mechanisms. In petroleum and petrochemical enterprises, due to their high temperature, high pressure, flammable and explosive working environment, the safety management requirements are extremely strict, and equipment inspections are needed to prevent accidents; in the manufacturing industry, especially large-scale production lines, regular inspections are required to prevent equipment failures and ensure production efficiency and product quality; in the chemical industry, it is necessary to regularly check the status of equipment to prevent chemical leaks and other safety accidents.

[0003] The normal operation of equipment plays a vital role in the production of enterprises. Equipment inspection helps enterprises achieve more efficient, safer and more economical production operations. Equipment inspection not only helps to ensure the normal operation of equipment, but also is an important part of continuous improvement and competitiveness of enterprises.

[0004] In order to ensure the actual inspection behavior of the staff, scanning code inspection is currently the lowest-cost method, but there is a widespread problem of cheating, that is, designated inspection personnel take photos of the equipment QR code for backup, and falsely report inspection behavior by scanning paper QR codes, loading photos, etc.

[0005] In order to effectively implement the inspection mechanism, an effective method to prevent inspection cheating is urgently needed.

[0006] There are several existing methods to solve inspection cheating: 1. Only WeChat can be used for scanning: Inspection personnel can only use WeChat’s scanning function to scan the code, and cannot find the QR code image from the mobile phone album to open it. This prevents inspection personnel from saving the QR code image in advance and then operating it elsewhere. However, there are also disadvantages: it is impossible to avoid the use of printed paper QR codes or QR codes on other terminals.

[0007] 2. Limit the scanning location: The system obtains the real-time location of the inspector when scanning the code. If he is not in the specified area, he cannot complete the inspection. It is like an electronic fence, and only people within the fence can inspect. The disadvantages are: the real-time location positioning has an error of more than 10 meters, which is not suitable for situations where multiple devices are close to each other in the workshop; it is not suitable for situations where different floors of the factory are located.

[0008] 3. Dynamic QR code: Use electronic QR code instead of paper QR code, the QR code is updated regularly, and the inspector will scan a different QR code each time, and can only scan the code in front of the equipment. The disadvantages are: electronic QR code requires procurement cost, power supply is required to keep the electronic equipment running normally, and batteries need to be replaced or rewired regularly.

[0009] 4. Hardware Bluetooth: Place a small Bluetooth device near the QR code. When you scan the QR code, the program will automatically find out whether the bound device is near the QR code site. If it is found, it means it is on site, and a sign-in record can be generated. The disadvantage is that configuring Bluetooth requires procurement costs.

[0010] Other related patented technical solutions include: 1. Chinese patent CN202121310135.X discloses a dual positioning system to prevent inspection cheating. It inspects each point on the inspection site. This solution requires setting an NFC electronic tag at each point. The dual positioning method of sending location information through a PDA handheld device and sending real-time positioning of the PDA handheld device through a GPS system is used to determine whether the specified inspection personnel are cheating during the inspection. This solves the problem of inspection personnel being lazy and filling in inspection reports at will without on-site inspections in traditional inspection methods. The above patent requires the configuration of multiple hardware devices such as PDAs and electronic tags, which increases procurement costs and requires the use of GPS positioning. It is not suitable for situations where multiple devices are close to each other in a workshop or on different floors of a factory building.

[0011] 2. Chinese patent CN202210044837.0 discloses a patrol inspection method and related equipment. The invention provides that Bluetooth hardware needs to be installed at the patrol point. When the inspector conducts the patrol inspection, the patrol program in the terminal is located to obtain the real location of the terminal, and then based on the real location of the terminal, it is determined whether the inspector has cheated in the patrol inspection, thereby effectively discovering the patrol inspector's patrol cheating. The above patent also requires the configuration of multiple hardware devices such as Bluetooth, which increases procurement costs and requires the use of GPS positioning. It is not suitable for situations where multiple devices are close to each other in the workshop, and it is not suitable for situations where different floors of the factory building are different.

[0012] 3. Chinese patent CN202211493371.9 discloses a line inspection management method, device, equipment and medium based on ZigBee communication. The invention uses the close-range communication characteristics of the ZigBee communication gateway and the ZigBee communication module, obtains the first optical handover box identification and the first longitude and latitude information through the module unique identification, and prevents inspection cheating through position comparison. This patent requires the configuration of specific hardware equipment, which increases procurement costs and requires the use of GPS positioning. It is not suitable for situations where multiple devices are close to each other in the workshop, and is not suitable for situations on different floors of the factory building. It requires specific identification equipment, which increases the difficulty for inspection personnel to record and fill in equipment failures. Summary of the invention

[0013] In view of the technical problem that the existing inspection method requires additional hardware costs and is not suitable for situations where multiple devices are close to each other in a workshop, the present invention proposes a method, device and system for anti-cheating inspection of workshop equipment by scanning code. It does not require additional hardware configuration, saves costs, and can be applicable to factory workshop environments with densely populated equipment and upper and lower floors.

[0014] In order to achieve the above object, the technical solution of the present invention is implemented as follows: a method for preventing cheating by scanning code inspection of workshop equipment, the steps of which are as follows: Step 1: Use a camera to obtain the initial image of each device within the monitoring range during unmanned inspection, use image recognition technology to identify the initial image to obtain the position coordinates of each device in the initial image, and form a static data set S of the device position; use a rangefinder to measure the distance A from the position of the QR code paper label on the device to the camera; Step 2: When the inspection personnel scans the QR code paper label on the equipment, obtain at least N camera images in the video captured by the camera within N seconds of the scanning time, use image recognition technology to respectively identify the position coordinates of the inspection personnel's head and body in the camera image, and obtain the head position data set S1 and the body position data set S2; Step 3: Calculate the area overlap between each sample in the head position data set S1 and all samples in the human position data set S2, and map and match the same inspector according to the area overlap; Step 4: Calculate the distance from the inspector's head to the camera based on the width of each head in the head position data set S1, and use the difference between the distance A and the distance to select the camera images corresponding to the samples that meet the conditions to form the sample set R1; Step 5: Calculate the area overlap between the samples in the human position data set S2 of the same inspector that match the sample mapping in the sample set R1 and the equipment in the equipment position static data set S, and select the samples with an area overlap of more than 50% to form a set R2; if the number of samples in the set R2 exceeds 0.8N, the inspector's code scanning inspection is valid; otherwise, the inspector's code scanning inspection is invalid.

[0015] Preferably, the image recognition technology is implemented by a deep learning target detection model, and the deep learning target detection model is a deep learning target detection model based on yolov5. The deep learning target detection model based on yolov5 is obtained by training a training sample set X and a sample set Y. The training sample set X includes multiple on-site pictures taken by a camera, and the sample set Y includes multiple on-site pictures of equipment that needs to be inspected. The equipment is marked in the training sample set X, the body of the inspector is marked in the sample set Y, and the head of the inspector is marked in the sample set Y.

[0016] Preferably, each piece of data in the device position static data set S represents the position coordinates of an identified device, and the position coordinates include the number of pixels from the device to the top of the initial image, the number of pixels from the leftmost side of the initial image, the height of the device, and the width of the device; the position coordinates in the head position data set S1 include the number of pixels from the head to the top of the camera image, the number of pixels from the leftmost side of the camera image, the height of the head, and the width of the head; the position coordinates in the human body position data set S2 include the number of pixels from the human body to the top of the camera image, the number of pixels from the leftmost side of the camera image, the height of the human body, and the width of the human body.

[0017] Preferably, the area overlap in step 3 is calculated as follows: 1) The upper left corner coordinates and lower right corner coordinates of the two rectangles where the head in the head position dataset S1 and the human body in the human body position dataset S2 are located are (x1, y1) and (x2, y2) and (a1, b1) and (a2, b2) respectively; then the overlapping area of ​​the two rectangles is: M_coin=(min(x2,a2) - max(x1,a1)) * (min(y2,b2) -max(y1,b1)); 2) The area of ​​the head M_head = width*height=(x2-x1)*(y2- y1); 3) Area overlap = M_coin / M_head.

[0018] Preferably, the method for calculating the distance from the inspection personnel's head to the camera is: obtain the head width M=width of the head in the camera image in the head position data set S1, take the actual head width of the person as 15 cm, and according to the camera imaging principle formula, the distance distance=actual head width of the person 15 cm*camera focal length f / head width M.

[0019] Preferably, the camera focal length f is calculated by: selecting a fixed object in the initial image as a reference object, measuring the height W of the reference object, using a rangefinder to measure the distance d from the middle position of the height of the reference object to the camera, and measuring the height w of the fire hydrant in the initial image in pixels; according to the imaging principle: W / w=d / f, the camera focal length f=w*d / W is obtained in pixels; Inspection personnel are required to stand on the side of the equipment closest to the QR code paper label and scan the QR code paper label. When scanning with a mobile phone, the distance between the person and the QR code paper label should be less than 1 meter. The inspection personnel should fill in the current equipment inspection record on the spot and stay for at least 5 seconds.

[0020] Preferably, the difference between the distance distance and the distance A should be less than 150 cm, and the sample set R1 that meets the conditions is screened out according to A-distince>=0 and A-distince<150.

[0021] Preferably, the acquired position coordinates of the human body are compared with the position coordinates of the device in the static device position data set S, the overlapping area is calculated, and samples with an area overlap of more than 50% are screened out to form a set R2; If the set R2 is not empty, the actual inspection of the current camera image is considered valid. If the set R2 is empty, the actual inspection of the current camera image is considered invalid. Repeat steps 4 to 5 for the 5 camera images of the same device, and calculate the valid number N1. If N1>0.8N, the inspection is considered valid. If N1<=0.8N, the inspection is considered invalid.

[0022] A device for preventing cheating by scanning code inspection for workshop equipment comprises a camera and a QR code paper label, wherein the QR code paper label is fixed on the equipment to be monitored, the camera is arranged at the upper part of the workshop, the equipment is within the monitoring range of the camera, the camera is connected to an edge server, the edge server is connected to a mobile terminal, the mobile terminal matches the QR code paper label on the equipment, and a built-in module of an edge gateway in the edge server executes the method for preventing cheating by scanning code inspection for workshop equipment.

[0023] A system for preventing cheating by scanning code inspection of workshop equipment comprises an edge server, wherein an edge gateway management cloud platform is arranged in the edge server, wherein an image acquisition module, an image recognition module, a data calculation module, a data analysis module and an inspection alarm module are arranged in the edge gateway management cloud platform, wherein the image acquisition module is used to obtain the video images collected by the camera every second after the code scanning time, the image recognition module recognizes the equipment, person or head in the image by image recognition technology and extracts the position coordinates of the equipment, person or head; the data calculation module is used to calculate the area overlap of the head in the head position data set S1 and the human body in the human body position data set S2, the area overlap of the human body in the human body position data set S2 with the highest area overlap and the equipment in the static data set S of the equipment position, and the distance between the head and the camera in the head position data set S1, the data analysis module is used to screen out qualified samples according to the results calculated by the data calculation module and to judge whether the inspection is effective, and the inspection alarm module is used to generate an alarm message of invalid inspection operation according to the judgment result of the data analysis module.

[0024] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention only needs to deploy computing program applications and provide environmental measurement values ​​to prevent inspection cheating. There is no need for additional hardware configuration, such as Bluetooth, PDA, electronic tags, etc., which saves costs.

[0025] 2. The present invention can be applied to factory workshop environments with densely populated equipment, solving the problem that the positioning solution has large errors and cannot be applied to densely populated workshops.

[0026] 3. The present invention can be applied to a factory workshop environment with upper and lower floors, solving the problem that the positioning scheme cannot distinguish and the equipment is in the same position on different floors.

[0027] The present invention belongs to the field of Internet of Things. Based on the image target recognition technology and the camera imaging principle, based on the existing equipment in the workshop, the distance measurement method is used to check the effectiveness of equipment inspection, prevent cheating by scanning codes, ensure the safe operation of enterprise equipment, reduce accidents, and increase the service life of equipment; without adding new hardware devices, the cost expenditure is reduced, and the problem that the positioning commonly used in the existing technical solutions cannot identify the floor is solved. The present invention is specifically manifested in: 1. According to the camera imaging principle, the distance between the human head and the camera in space is calculated, and the spatial distance between the known QR code label and the camera is compared to determine that the device and the human are on a similar spherical surface, and determine that the position of the human to the camera is close to the distance between the device and the camera.

[0028] 2. Use image target recognition technology to calculate the overlap between the coordinates of the person's head and the coordinates of the device. The management requires that the person's position is between the device and the camera. Determine the approximate angles of the person and the device relative to the camera, and determine that the person, camera, and device are in the same straight line.

[0029] 3. By obtaining multiple on-site photos after scanning the code, repeated calculations and multiple verifications can be performed to avoid abnormal results caused by other people passing by.

[0030] 4. The focal length is automatically calculated through the inherent reference object length value. When the camera turns or the focal length changes, the focal length value correction method is immediately triggered to avoid calculation errors caused by focal length changes.

[0031] 5. Use image target recognition technology and camera imaging principles to determine whether the inspector is near the equipment, solving the problem of positioning error and the inability to determine multiple floors in the factory. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 It is a flow chart of the present invention.

[0034] Figure 2 This is an example diagram of monitoring equipment in the equipment location static data set S of the present invention.

[0035] Figure 3 Schematic diagram of the camera imaging principle.

[0036] Figure 4 This is an example diagram of a head in the head position dataset S1 of the present invention.

[0037] Figure 5 This is an example diagram of human body positions in the human body position dataset S2 in the present invention.

[0038] Figure 6 This is an example diagram of the monitoring device and the human body position in the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Example 1 like Figure 1 As shown in the figure, a method for anti-cheating by scanning code inspection of workshop equipment is implemented by using existing items in the factory workshop: QR code paper labels on the equipment and surveillance cameras (most workshops have installed multiple cameras at different angles). It is suitable for scenarios where the positions of workshop equipment and cameras are fixed and there is no large-scale movement.

[0041] The preparatory work of the present invention is: 1. The QR code paper label is affixed to the device on the camera’s shooting surface, and the overlap area between the inspector’s body and the device imaged by the camera is at least 50%; 2. Use a rangefinder to measure the distance A from the QR code paper label on the device to the camera; 3. Prepare multiple on-site pictures taken by the camera as the training sample set X; 4. Prepare multiple on-site pictures of the equipment to be inspected as sample set Y; 5. The image recognition technology used in the present invention is a deep learning target detection model based on yolov5. The target detection model is trained using training sample set X and sample set Y. The equipment is marked in training sample set X, the human body of the staff is marked in sample set Y, and the head of the staff is marked in sample set Y, so as to accurately identify the three target objects with an accuracy rate greater than 95%.

[0042] The above five items are the data preparation stage, which involves obtaining sample sets of human bodies, human heads, and equipment and analyzing features. The purpose is to use these features to identify three specific objects, rather than other objects, in actual daily camera images.

[0043] Flow chart of the technical solution for judging the effectiveness of single equipment inspection, such as Figure 1 As shown, the specific steps of the present invention are: Step 1: Data preparation stage: Use the camera to obtain the initial image of each device within the monitoring range during unmanned inspection, use image recognition technology to identify the initial image to obtain the coordinates of each device in the initial image, and form a static data set S of the device location; use the rangefinder to measure the distance A from the position of the QR code paper label on the device to the camera.

[0044] Step 1 is to obtain the coordinates of the device's location. Since the device's location is not easily changed, it only needs to be obtained once; however, the position of the human body and the head is constantly changing, so the following steps are to obtain in real time and update all the time. The work to be done in step 1 is to obtain the location coordinates of the device at the installed location and match the device ID (automatically or manually maintained) for use in the following steps.

[0045] Use image recognition technology to identify the coordinates of the target device in the camera image, and obtain the device location static data set S. Each piece of data in the device location static data set S represents an identified device sample. Under normal circumstances, this data is only obtained once and is used as the static parameter of the device. The real device code is manually maintained. When the device moves or the camera moves, the device location static data set S needs to be re-obtained. Figure 2 As shown, an example of obtaining the coordinate data of a camera monitoring device: {"attributes": {"name":"Device 1"},{"location": { "top": 702, "left":946, "width": 631, "height": 346}}, {"attributes": {"name":"Device 2"},{"location": { "top": 218, "left":555, "width": 286, "height": 225}}, ...... {"attributes": {"name":"Device sn"},{"location": { "top": tsn,"left":lsn, "width": wsn, "height": hsn}} Among them, "attributes" represents the attribute key name, "name" represents the device name, "location" represents the coordinates, "top": key means the number of pixels from the top of the image, "left": key means the number of pixels from the leftmost side of the image, "height": key means the height of the identified target, "width": key means the width of the identified target, tsn, lsn, wsn, hsn are the actual values.

[0046] Image recognition technology includes: 1. Prepare samples, manually mark in boxes which are people, which are people's heads, and which are devices, and define label categories; 2. Input the prepared samples into the target detection model, and the target detection model will know which graphic features are the required devices, which graphic features are the required people, and which graphic features are the required people's heads; after multiple rounds of training, the accuracy is improved; 3. Then in actual daily shooting, the picture + target detection model can automatically identify the devices, people, and heads that meet the requirements, and the output results include: the coordinates, labels, and accuracy of the identified targets in the picture.

[0047] "top", "left", "width", and "height" are the coordinates of the device, person, or head, which are recognized by the model. The model also recognizes the coordinates and target type, and marks the image according to the coordinates.

[0048] Calculate the focal length f of the camera: Select a reference object to measure key parameters, such as Figure 2 The fire hydrant in the middle is used as a reference, the height W of the fire hydrant is measured, the distance d from the middle position of the fire hydrant to the camera is measured with a rangefinder, and the height w of the fire hydrant in the initial image (in pixels) is measured, such as Figure 3 As shown, according to the imaging principle, W / w=d / f, the focal length f=w*d / W is obtained, and the unit of f is pixel.

[0049] Step 2: When the inspection personnel scan the QR code paper label on the target device, record the scanning time, obtain at least N camera images captured by the camera during the scanning time, identify the coordinates of the inspection personnel's head position and body position in the camera images, and obtain the head position data set S1 and body position data set S2 respectively.

[0050] The management requires the inspection personnel to scan the code, and the system records the scanning time. The inspection personnel are required to stand on the side of the equipment closest to the QR code paper label and scan the QR code paper label. The distance between the person and the QR code paper label when scanning with a mobile phone should be less than 1 meter, and the inspection personnel should fill in the current equipment inspection record on the spot and stay for at least 5 seconds.

[0051] Capture 5 frames of the video according to the scanning time (the quantity will be adjusted according to actual tests), one frame per second for 5 consecutive seconds, and use image recognition technology to identify the position of the human head. As Figure 4 shown, obtain the set S1 of the coordinates of the heads of the inspection personnel in the camera image. Each piece of data represents a human head sample identified: {"attributes": {"name":"Head 1"},{"location": { "top": 815, "left":1041, "width": 36, "height": 52}} {"attributes": {"name":"Head 2"},{"location": { "top": tt2, "left":lt2, "width": wt2, "height": ht2}} ...... {"attributes": {"name":"Head tn"},{"location": { "top": ttn,"left":ltn, "width": wtn, "height": htn}} where tn is the number of human heads, and ttn, ltn, wtn, and htn respectively represent the number of pixels of the human head from the top of the image, the number of pixels from the left side of the image, the height of the human head, and the width of the human head.

[0052] Use image recognition technology to identify the position of the human body. As Figure 5 shown, obtain the human body position dataset S2 of the coordinates of the inspection personnel in each camera image. Each piece of data represents a human sample identified: {"attributes": {"name":"Person 1"},{"location": { "top": 815"left": 1017,"width": 87, "height": 232}} {"attributes": {"name":"Person 2"},{"location": { "top": t2, "left": l2,"width": w2, "height": h2}} ...... {"attributes": {"name":"person rn"},{"location": { "top": trn,"left":lrn, "width": wrn, "height": hrn}} Among them, rn is the number of human bodies, and ttn, ltn, wtn, and htn respectively represent the number of pixels of the human body from the topmost of the image, the number of pixels from the leftmost of the image, the height of the human body, and the width of the human body.

[0053] Step 3: Calculate the area overlap degree of each sample in the head position dataset S1 with all samples in the human body position dataset S2, and map and match the same patrol personnel according to the area overlap degree.

[0054] The sets S1 and S2 are mapped and matched to the same person, and the overlap degree of the two head position datasets S1 and the human body position dataset S2 in Step 2 is calculated. The single sample in the head position dataset S1 is matched and calculated with all samples in the human body position dataset S2, and the sample in the human body position dataset S2 with the highest overlap degree is selected and considered to be the same person. Taking the existing measurement values as an example, the calculation method is as follows: 1) Given that the upper left coordinates and lower right coordinates of the two rectangles of the sample in the head position dataset S1 and the sample in the human body position dataset S2 are (x1, y1) and (x2, y2) and (a1, b1) and (a2, b2) respectively, then the overlapping area of these two rectangles can be calculated by the following formula: M_coin=(min(x2,a2) - max(x1,a1)) * (min(y2,b2) -max(y1,b1)) =(min(1077,1104)-max(1041,1017))*(min(867,1047)-max(815,815))=(1077-1041)*(867-815)=36*52=1872; among them, the upper left coordinates (x1, y1) of "head 1"=(left, top)=(1041, 815), the lower right coordinates (x2, y2) of "head 1"=(left + width, top + height)=(1041 + 36, 815 + 52)=(1077, 867), the upper left coordinates (a1, b1) of "person 1"=(left, top)=(1017, 815), and the upper right coordinates (a2, b2) of "person 1"=(left + width, top + height)=(1017 + 87, 815 + 232)=(1104, 1047). If the value of the overlapping area M_coin is negative, it means that the two rectangles do not overlap.

[0055] 2) Head area M_head = width*height=(x2-x1)*(y2- y1) and = 36*52=1872; 3) Area overlap = M_coin / M_head=1872 / 1872=1.

[0056] Through the calculation of the above area overlap, the matching person coordinates and head coordinates are the same actual person.

[0057] Step 4: Calculate the distance from the inspector's head to the camera based on the head width of the samples in the head position data set S1, and use the difference between the distance A and the distance to filter out samples that meet the conditions to form the sample set R1.

[0058] Step 2: Get the image to identify the width of the inspector's head in the camera image. The actual width of most normal adults' heads is 13-15cm. Since the head image is tilted or the person is wearing a hat, the M value is too large, so the actual head width is taken as 15cm. Calculate the distance from the person's head to the camera. According to the camera imaging principle formula, distance = actual head width 15cm * camera focal length f / inspector's head width M in the camera image.

[0059] The difference between the distance value distance and the distance A in the preparation work should be less than 150cm (it will be fine-tuned when actually optimizing the accuracy), that is, the distance between the human head and the QR code should be less than 150CM. According to A-distince>=0 and A-distince<150, the qualified sample set R1 is selected. The sample set R1 is the head position data set S1 automatically identified in the actual captured image after being filtered by the conditions A-distince>=0 and A-distince<150, so as to determine that the position from the person to the camera is close to the distance from the device to the camera.

[0060] Step 5: Calculate the area overlap between the samples in the human position data set S2 of the same inspector that match the sample mapping in the sample set R1 and the devices in the device position static data set S, and select the samples with an area overlap of more than 50% to form a set R2; if the number of samples in the set R2 exceeds 0.8N, the inspector's code scanning inspection is valid; otherwise, the inspector's code scanning inspection is invalid.

[0061] Based on the screened sample set R1, the obtained human body coordinates are Figure 6 The coordinates of the device in the blue framed area and device location static dataset S are Figure 6Compare the red framed area, calculate the overlapping area, and select the set R2 with an area overlap of more than 50%. The overlap calculation method is the same as step 3.

[0062] If the set R2 is not empty, the current single photo is considered to be valid for actual inspection. If the set R2 is empty, the current single photo is considered to be invalid for actual inspection. Repeat steps 4-5 for the 5 photos of the same device, calculate the valid number of times N1, if N1>3 (valid number ratio>=80%), the current inspection is considered valid, if N1<=3, the current inspection is considered invalid. The purpose of this step is to prevent passers-by from being identified as inspectors and causing result deviation.

[0063] If the factory is equipped with cameras connected to the edge gateway, real-time calculation can be performed to determine whether the current inspection is effective; if the factory is equipped with ordinary cameras, offline timing calculation can be used to periodically batch-judge the effectiveness of the inspection after the inspection. The rotation of the camera requires the re-acquisition of the device coordinate data to determine the static data set S of the device position.

[0064] Example 2 A device for anti-cheating by scanning code inspection for workshop equipment comprises a camera and a QR code paper label, wherein the QR code paper label is fixed on the corresponding equipment, the camera is arranged at the upper part of the workshop, the equipment is within the monitoring range of the camera, the camera is connected to an edge server, the edge server is connected to a mobile terminal, the mobile terminal matches the QR code paper label on the equipment, and the edge gateway built-in module in the edge server executes the method for anti-cheating by scanning code inspection for workshop equipment described in Example 1.

[0065] Example 3 A system for preventing cheating by scanning code inspection of workshop equipment comprises an edge server, wherein an edge gateway management cloud platform is provided in the edge server, wherein an image acquisition module, an image recognition module, a data calculation module, a data analysis module and an inspection alarm module are provided in the edge gateway management cloud platform, wherein the image acquisition module is used to obtain the video images collected by the camera every second after the code scanning time, the image recognition module is used to identify the equipment, person or head in the image and extract its position coordinates; the data calculation module is used to calculate the area overlap of the head in the head position data set S1 and the human body in the human body position data set S2, the area overlap of the human body in the human body position data set S2 with the highest area overlap and the equipment in the static data set S of the equipment position, and the distance between the head and the camera in the head position data set S1, the data analysis module is used to screen out qualified samples and judge whether the inspection is effective according to the results calculated by the data calculation module, and the inspection alarm module is used to generate an alarm message of invalid inspection operation according to the judgment result of the data analysis module.

[0066] The edge gateway management cloud platform also has: Device activation function module: Activate registered edge gateway devices on the edge gateway management platform. Edge gateway devices include cameras.

[0067] Application function module: upload the trained target detection model for other edge servers to download the application on the edge gateway management platform.

[0068] Sub-device function module: used to add camera sub-devices.

[0069] Sub-device viewing function: select the application and enter the application parameters, such as device coordinates, focal length parameters, etc. After the first input and submission, the system automatically stores and submits the settings for viewing; click the alarm to view the detection results.

[0070] After submitting the review settings, open the video page to verify the method of anti-cheating by scanning code for inspection of workshop equipment described in Example 1.

[0071] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for preventing cheating by scanning code for inspection of workshop equipment, characterized in that: The steps are as follows: Step 1: Use a camera to obtain the initial image of each device within the monitoring range during unmanned inspection, use image recognition technology to identify the initial image to obtain the position coordinates of each device in the initial image, and form a static data set S of the device position; use a rangefinder to measure the distance A from the position of the QR code paper label on the device to the camera; Step 2: When the inspection personnel scans the QR code paper label on the equipment, obtain at least N camera images in the video captured by the camera within N seconds of the scanning time, use image recognition technology to respectively identify the position coordinates of the inspection personnel's head and body in the camera image, and obtain the head position data set S1 and the body position data set S2; Step 3: Calculate the area overlap between each sample in the head position data set S1 and all samples in the human position data set S2, and map and match the same inspector according to the area overlap; Step 4: Calculate the distance from the inspector's head to the camera based on the width of each head in the head position data set S1, and use the difference between the distance A and the distance to select the camera images corresponding to the samples that meet the conditions to form the sample set R1; Step 5: Calculate the area overlap between the samples in the human position dataset S2 of the same inspector that match the sample mapping in the sample set R1 and the equipment in the equipment position static dataset S, and select the samples with an area overlap of more than 50% to form the set R2; if the number of samples in the set R2 exceeds 0.8N, the inspection by the inspector is valid; Otherwise, the inspection by the inspector scanning the code will be invalid.

2. The method for preventing cheating by scanning code for workshop equipment inspection according to claim 1 is characterized in that: The image recognition technology is implemented through a deep learning target detection model, which is a deep learning target detection model based on yolov5. The deep learning target detection model based on yolov5 is obtained by training a training sample set X and a sample set Y. The training sample set X includes multiple on-site pictures taken by a camera, and the sample set Y includes multiple on-site pictures of equipment that needs to be inspected. The equipment is marked in the training sample set X, the body of the inspector is marked in the sample set Y, and the head of the inspector is marked in the sample set Y.

3. The method for preventing cheating by scanning code for workshop equipment inspection according to claim 2 is characterized in that: Each piece of data in the device position static data set S represents the position coordinates of an identified device, and the position coordinates include the number of pixels from the device to the top of the initial image, the number of pixels from the leftmost side of the initial image, the height of the device, and the width of the device; the position coordinates in the head position data set S1 include the number of pixels from the head to the top of the camera image, the number of pixels from the leftmost side of the camera image, the height of the head, and the width of the head; the position coordinates in the human body position data set S2 include the number of pixels from the human body to the top of the camera image, the number of pixels from the leftmost side of the camera image, the height of the human body, and the width of the human body.

4. The method for preventing cheating by scanning code for inspection of workshop equipment according to any one of claims 1 to 3, characterized in that: The calculation method of the area overlap in step 3 is: 1) The upper left corner coordinates and lower right corner coordinates of the two rectangles where the head in the head position dataset S1 and the human body in the human body position dataset S2 are located are (x1, y1) and (x2, y2) and (a1, b1) and (a2, b2) respectively; then the overlapping area of ​​the two rectangles is: M_coin=(min(x2,a2) - max(x1,a1)) * (min(y2,b2) -max(y1,b1)); 2) The area of ​​the head M_head = width*height=(x2-x1)*(y2- y1); 3) Area overlap = M_coin / M_head.

5. The method for preventing cheating by scanning code for inspection of workshop equipment according to claim 4 is characterized in that: The method for calculating the distance from the head of the patrol personnel to the camera is: obtain the head width M=width of the head in the camera image in the head position data set S1, take the actual head width of the person as 15 cm, and according to the camera imaging principle formula, the distance distance=actual head width of the person 15 cm*camera focal length f / head width M.

6. The method for preventing cheating by scanning code for inspection of workshop equipment according to claim 5 is characterized in that: The camera focal length f is calculated by: selecting a fixed object in the initial image as a reference object, measuring the height W of the reference object, using a rangefinder to measure the distance d from the middle position of the height of the reference object to the camera, and measuring the height w of the fire hydrant in the initial image in pixels; according to the imaging principle: W / w=d / f, the camera focal length f=w*d / W is obtained in pixels; Inspection personnel are required to stand on the side of the equipment closest to the QR code paper label and scan the QR code paper label. When scanning with a mobile phone, the distance between the person and the QR code paper label should be less than 1 meter. The inspection personnel should fill in the current equipment inspection record on the spot and stay for at least 5 seconds.

7. The method for preventing cheating by scanning code for inspection of workshop equipment according to claim 4 or 5, characterized in that: The difference between the distance distance and the distance A should be less than 150 cm, and the sample set R1 that meets the conditions is screened out according to A-distince>=0 and A-distince<150.

8. The method for preventing cheating by scanning code for inspection of workshop equipment according to claim 7 is characterized in that: In the sample set R1, the acquired position coordinates of the human body are compared with the position coordinates of the device in the static device position data set S, the overlapping area is calculated, and the samples with an area overlap of more than 50% are selected to form the set R2; If the set R2 is not empty, the actual inspection of the current camera image is considered valid. If the set R2 is empty, the actual inspection of the current camera image is considered invalid. Repeat steps 4 to 5 for the 5 camera images of the same device, and calculate the valid number N1. If N1>0.8N, the inspection is considered valid. If N1<=0.8N, the inspection is considered invalid.

9. A device for preventing cheating by scanning code for inspection of workshop equipment, characterized in that: It includes a camera and a QR code paper label, the QR code paper label is fixed on the equipment to be monitored, the camera is set at the upper part of the workshop, the equipment is within the monitoring range of the camera, the camera is connected to the edge server, the edge server is connected to the mobile terminal, the mobile terminal matches the QR code paper label on the equipment, and the edge gateway built-in module in the edge server executes a method for anti-cheating by scanning code inspection for workshop equipment as described in any one of claims 1-8.

10. The system of the workshop equipment code scanning inspection and anti-cheating method according to any one of claims 1 to 8, characterized in that: It includes an edge server, which is equipped with an edge gateway management cloud platform. The edge gateway management cloud platform is equipped with an image acquisition module, an image recognition module, a data calculation module, a data analysis module and an inspection alarm module. Among them, the image acquisition module is used to obtain the camera image collected by the camera every second after the code scanning time. The image recognition module uses image recognition technology to identify the device, person or head in the image and extract the position coordinates of the device, person or head; the data calculation module is used to calculate the area overlap of the head in the head position data set S1 and the human body in the human body position data set S2, the area overlap of the human body in the human body position data set S2 with the highest area overlap and the device in the static data set S of the device position, and the distance between the head and the camera in the head position data set S1. The data analysis module is used to screen out qualified samples according to the results calculated by the data calculation module and judge whether the inspection is effective. The inspection alarm module is used to generate an alarm message that the inspection operation is invalid according to the judgment result of the data analysis module.

Citation Information

Patent Citations

  • Intelligent monitoring method, device and system for manual inspection, and storage medium

    CN110781754A

  • Machine room inspector inspection operation compliance detection method based on computer vision

    CN118397723A

  • Dual positioning system for preventing inspection cheating

    CN214896700U

  • Occupant detection device

    JP2012127811A