A method, device and system for anti-cheating of code scanning inspection of workshop equipment

Abstract

The application provides a method, device and system for preventing cheating in code scanning and inspection of workshop equipment, and the method comprises the following steps: obtaining a data set S by recognizing the position coordinates of each equipment in an initial image collected by a camera during unmanned inspection by using image recognition technology; obtaining multiple camera images collected by the camera at a code scanning time, recognizing the position coordinates of the head and body of an inspector in the camera images respectively, and obtaining a head data set and a body data set; calculating the area coincidence degrees of samples in the head data set and the body data set respectively, and mapping and matching the inspector according to the area coincidence degrees; calculating the distance from the head of the inspector to the camera according to the width of the head in the head data set, and screening camera images to form a sample set; calculating the area coincidence degrees of samples in the sample set and the equipment in the data set S, screening the samples, and judging whether the code scanning and inspection of the inspector is effective. According to the picture target recognition technology and the imaging principle of the camera, the distance is calculated, the effectiveness of equipment inspection is verified, and the code scanning cheating behavior is prevented. In the case that no new hardware device is added, the cost expenditure is reduced.

Description

Technical Field

[0001] This invention relates to the technical fields of workshop inspection and the Internet of Things, and in particular to a method, device and system for preventing cheating in workshop equipment scanning inspection. Background Technology

[0002] With technological advancements, the application of machinery and equipment across various industries is becoming increasingly widespread, significantly improving enterprise production efficiency. However, equipment malfunctions leading to abnormal operation or shutdowns can severely impact production processes and even cause safety accidents. Therefore, most enterprises have established equipment inspection mechanisms. In the oil and petrochemical industry, due to the high-temperature, high-pressure, flammable, and explosive working environment, safety management requirements are extremely stringent, necessitating equipment inspections to prevent accidents. In manufacturing, especially on large-scale production lines, regular inspections are required to prevent equipment failures and ensure production efficiency and product quality. In the chemical industry, regular checks of equipment status are necessary to prevent chemical leaks and other safety accidents.

[0003] The proper functioning of equipment plays a vital role in a company's production. Equipment inspection helps companies achieve more efficient, safer, and more economical production operations. Equipment inspection not only helps ensure the normal operation of equipment but is also an important part of a company's continuous improvement and enhancement of competitiveness.

[0004] To ensure that staff actually conduct inspections, scanning QR codes is currently the lowest-cost method. However, cheating is common, with designated inspectors taking photos of the equipment's QR codes and then falsely reporting inspection activities by scanning paper QR codes or loading photos.

[0005] In order to effectively implement the inspection mechanism, there is an urgent need for an effective method to prevent inspection fraud.

[0006] The existing methods for addressing inspection fraud include the following: 1. Scan only with WeChat: Inspectors can only scan QR codes using WeChat's scan function; they cannot open QR code images from their phone's photo album. This prevents inspectors from saving QR code images in advance and then operating on other devices. However, this also has drawbacks: it cannot prevent the use of printed paper QR codes or QR codes from other terminals.

[0007] 2. Restricted Scanning Location: The system obtains the real-time location of the inspector when scanning the code. If the inspector is not within the designated area, they cannot complete the inspection. It's like an electronic fence; only those within the fence can perform inspections. The drawbacks are: real-time location positioning has errors, exceeding 10 meters, making it unsuitable for situations where multiple devices are close together in a workshop; and unsuitable for situations where equipment is on different floors of a factory building.

[0008] 3. Dynamic QR Codes: Electronic QR codes replace paper QR codes. The QR codes are updated periodically, and each scan by inspection personnel uses a different QR code, requiring them to be scanned only at the device. Disadvantages include: the electronic QR codes require procurement costs, and the electronic equipment requires power to operate normally, necessitating periodic battery replacements or rewiring.

[0009] 4. Hardware Bluetooth: Place a small Bluetooth device near the QR code. When the QR code is scanned, the program automatically searches for the bound device nearby. If found, it indicates the device is present, and a check-in record is generated. The drawback is the incurred purchase cost for configuring Bluetooth.

[0010] Other related patented technical solutions include: 1. Chinese patent CN202121310135.X discloses a dual positioning system to prevent inspection fraud. This system inspects various points at each inspection site. The solution requires an NFC electronic tag at each point and uses a dual positioning method—sending location information via a PDA handheld device and real-time positioning via a GPS system—to determine whether inspection personnel are committing fraud. This addresses the problem of laziness and arbitrary filling out of inspection reports by inspectors who do not conduct on-site inspections, a problem inherent in traditional inspection methods. However, this patent requires multiple hardware devices such as PDAs and electronic tags, increasing procurement costs. It also relies on GPS positioning, making it unsuitable for situations where multiple devices are close together within a workshop or on different floors of a factory building.

[0011] 2. Chinese patent CN202210044837.0 discloses an inspection method and related equipment. This invention requires the installation of Bluetooth hardware at inspection points. When an inspector is conducting an inspection, the system locates the terminal's actual position by tracking the inspection program within the terminal. Based on the terminal's actual position, it determines whether the inspector is engaging in inspection fraud, thus effectively detecting such fraud. However, this patent also requires multiple hardware devices, such as Bluetooth devices, increasing procurement costs. It also relies on GPS positioning, making it unsuitable for situations where multiple devices are close together in a workshop or on different floors of a factory.

[0012] 3. Chinese Patent CN202211493371.9 discloses a method, device, equipment, and medium for line inspection management based on Zifeng Communication. This invention utilizes the close-range communication characteristics of the Zifeng Communication gateway and the Zifeng Communication module, and obtains the first optical junction box identifier and first latitude and longitude information through the module's unique identifier, preventing inspection fraud through location comparison. This patent requires specific hardware equipment, increasing procurement costs. It also requires GPS positioning, making it unsuitable for situations where multiple devices are close together in a workshop or on different floors of a factory. Furthermore, it requires specific equipment identification devices, increasing the difficulty for inspection personnel to record and fill in equipment fault information. Summary of the Invention

[0013] To address the technical problem that existing inspection methods require increased hardware costs and are unsuitable for situations where multiple devices are close together in a workshop, this invention proposes a method, device, and system for preventing cheating during workshop equipment scanning inspections. This method does not require additional hardware configuration, saves costs, and is applicable to factory workshop environments with densely packed equipment and multiple floors.

[0014] To achieve the above objectives, the technical solution of the present invention is as follows: a method for preventing cheating during workshop equipment barcode scanning inspection, comprising the following steps: Step 1: Use a camera to acquire initial images of each device within the monitoring range during unmanned inspection; use image recognition technology to identify the position coordinates of each device in the initial image, forming a static dataset S of device positions; use a rangefinder to measure the distance A from the position of the QR code paper label on the device to the camera. Step 2: When the inspection personnel scan the QR code paper label on the device, at least N camera images are captured in the video within the scanning time of N seconds. Image recognition technology is used to identify the position coordinates of the inspection personnel's head and body in the camera images, and the head position dataset S1 and body position dataset S2 are obtained. Step 3: Calculate the area overlap between each sample in the head position dataset S1 and all samples in the human body position dataset S2, and match the same inspector based on the area overlap. Step 4: Calculate the distance from the inspector's head to the camera based on the width of each head in the head position dataset S1. Use the difference between distance A and distance to filter out the camera images corresponding to the samples that meet the conditions, and form a sample set R1. Step 5: Calculate the area overlap between the human body location dataset S2 of the same inspector in the sample set R1 and the equipment location static dataset S. Select samples with an area overlap of more than 50% to form set R2. If the number of samples in set R2 exceeds 0.8N, the inspector's scan inspection is valid; otherwise, the inspector's scan inspection is invalid.

[0015] Preferably, the image recognition technology is implemented through a deep learning object detection model, which is a YOLOv5-based deep learning object detection model. The YOLOv5-based deep learning object detection model is trained using a training sample set X and a sample set Y. The training sample set X includes multiple on-site images captured by a camera, and the sample set Y includes multiple on-site images of the equipment to be inspected. The equipment is labeled in the training sample set X, and the human body of the inspection personnel is labeled in the sample set Y, with the head of the inspection personnel also labeled in the sample set Y.

[0016] Preferably, each data point in the device location static dataset S represents the location coordinates of an identified device, including the number of pixels the device is from the top of the initial image, the number of pixels it is from the leftmost edge of the initial image, the height of the device, and the width of the device; the location coordinates in the head location dataset S1 include the number of pixels the head is from the top of the camera image, the number of pixels it is from the leftmost edge of the camera image, the height of the head, and the width of the head; the location coordinates in the human body location dataset S2 include the number of pixels the human body is from the top of the camera image, the number of pixels it is from the leftmost edge of the camera image, the height of the human body, and the width of the human body.

[0017] Preferably, the method for calculating the area overlap in step three is as follows: 1) Given that the top-left and bottom-right corner coordinates of the two rectangles containing the head in head location dataset S1 and the human body in human body location dataset S2 are (x1, y1) and (x2, y2) respectively, and (a1, b1) and (a2, b2) respectively; then the overlapping area of ​​the two rectangles is: M_coin=(min(x2,a2) - max(x1,a1)) * (min(y2,b2) -max(y1,b1)); 2) The area of ​​the head, M_head = width * height = (x2 - x1) * (y2 - y1); 3) Area overlap = M_coin / M_head.

[0018] Preferably, the method for calculating the distance from the inspector's head to the camera is as follows: obtain the head width M=width in the camera image from the head position dataset S1, take the actual head width as 15cm, and according to the camera imaging principle formula, the distance = actual head width 15cm * camera focal length f / head width M.

[0019] Preferably, the method for calculating the camera focal length f is as follows: a fixed object in the initial image is selected as a reference object, the height W of the reference object is measured, the distance d from the middle position of the reference object to the camera is measured using a rangefinder, and the height w of the fire hydrant in the initial image is measured in pixels; according to the imaging principle: W / w=d / f, the camera focal length f=w*d / W is obtained, in pixels. Inspection personnel are required to stand on the side of the equipment closest to the QR code paper label, scan the QR code paper label directly, keep the distance between the person and the QR code paper label less than 1 meter when scanning with a mobile phone, fill in the current equipment inspection record on the spot, and stay for at least 5 seconds.

[0020] Preferably, the difference between the distance and the distance A should be less than 150cm, and a sample set R1 that meets the conditions is selected based on A-distince>=0 and A-distince<150.

[0021] Preferably, the obtained human body position coordinates are compared with the device position coordinates in the static dataset S of device positions, the overlapping area is calculated, and samples with an area overlap of more than 50% are selected to form a set R2. If set R2 is not empty, the actual inspection of the current camera image is considered valid; if set R2 is empty, the actual inspection of the current camera image is considered invalid. Repeat steps four and five for five camera images from the same device and calculate the number of valid inspections N1. If N1 > 0.8N, the inspection is considered valid; if N1 <= 0.8N, the inspection is considered invalid.

[0022] A device for preventing cheating during workshop equipment scanning and inspection includes a camera and QR code paper labels. The QR code paper labels are fixed to the equipment to be monitored. The camera is set at the top of the workshop, and the equipment is within the camera's monitoring range. The camera is connected to an edge server, and the edge server is connected to a mobile terminal. The mobile terminal is matched with the QR code paper label on the equipment. The edge gateway built-in module in the edge server executes the aforementioned method for preventing cheating during workshop equipment scanning and inspection.

[0023] A system for preventing cheating during workshop equipment barcode scanning inspection includes an edge server. The edge server houses an edge gateway management cloud platform, which in turn includes an image acquisition module, an image recognition module, a data calculation module, a data analysis module, and an inspection alarm module. The image acquisition module acquires images captured by the camera every second after the barcode scanning time. The image recognition module uses image recognition technology to identify equipment, people, or heads in the images and extract their position coordinates. The data calculation module calculates the area overlap between the head in head position dataset S1 and the human body in human body position dataset S2, the area overlap between the human body in human body position dataset S2 with the highest area overlap and the equipment in static equipment position dataset S, and the distance between the head and the camera in head position dataset S1. The data analysis module filters samples that meet the conditions based on the calculation results and determines whether the inspection is valid. The inspection alarm module generates an alarm message indicating that the inspection operation is invalid based on the judgment results of the data analysis module.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention only requires the deployment of a computing application to provide environmental measurement values ​​to prevent inspection fraud, without the need for additional hardware configurations such as Bluetooth, PDA, electronic tags, etc., thus saving costs.

[0025] 2. This invention can be applied to factory workshops with dense equipment, solving the problem that positioning schemes have large errors and cannot be applied to dense workshops.

[0026] 3. This invention can be applied to factory workshop environments with multiple floors, solving the problem that positioning schemes cannot distinguish between different locations and that equipment is in the same position on different floors.

[0027] This invention belongs to the field of the Internet of Things (IoT). Based on image target recognition technology and camera imaging principles, and utilizing existing workshop equipment, it verifies the effectiveness of equipment inspections through distance calculation methods, prevents barcode scanning fraud, ensures the safe operation of enterprise equipment, reduces accidents, and extends equipment lifespan. It reduces costs without adding new hardware and solves the problem of existing technologies failing to identify floor levels when using positioning. Specifically, this invention is characterized by: 1. Based on the imaging principle of a camera, calculate the spatial distance between the person's head and the camera, compare the spatial distance between the known QR code label and the camera, and thus determine that the device and the person are on similar spherical surfaces, thus confirming that the distance between the person and the camera is close to the distance between the device and the camera.

[0028] 2. Using image target recognition technology, calculate the overlap between the coordinates of the person's head and the coordinates of the device. Management requires that the person's position be between the device and the camera. Determine the method of the person and the device being at approximately the same angle relative to the camera to ensure that the person, camera, and device are on the same straight line.

[0029] 3. By obtaining multiple photos of the scene after scanning the code, repeating the calculation, and verifying multiple times, the system avoids situations where other people passing by cause abnormal results.

[0030] 4. The focal length is automatically calculated using the inherent reference object length value. When the camera turns or the focal length changes, the focal length value correction method is immediately triggered to avoid calculation errors caused by changes in focal length.

[0031] 5. By using image target recognition technology and camera imaging principles, it can determine whether the inspector is near the equipment, thus solving the problems of positioning errors and the inability of positioning to determine multiple floors in the factory. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a flowchart of the present invention.

[0034] Figure 2 This is an example diagram of the monitoring devices in the static data set S of device location of the present invention.

[0035] Figure 3 This is a schematic diagram illustrating the imaging principle of a camera.

[0036] Figure 4 This is an example image of a head in the head location dataset S1 of this invention.

[0037] Figure 5 This is an example diagram of human body positions in the human body position dataset S2 of this invention.

[0038] Figure 6 This is an example diagram showing the position of the monitoring device and the human body in this invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Example 1 like Figure 1 As shown, a method for preventing cheating during workshop equipment scanning inspection utilizes existing items in the factory workshop: QR code paper labels on the equipment and surveillance cameras (most workshops have multiple cameras installed at different angles). This method is suitable for scenarios where the positions of workshop equipment and cameras are fixed and do not move significantly.

[0041] The preparatory work for this invention is as follows: 1. The QR code paper label is affixed to the device on the camera's field of view, and the overlap area between the inspector's body and the device in the camera's image is at least 50%; 2. Use a rangefinder to measure the distance A from the location of the QR code paper label on the device to the camera; 3. Prepare multiple on-site images captured by the camera as the training sample set X; 4. Prepare multiple on-site photos of the equipment to be inspected, as a sample set Y; 5. The image recognition technology used in this invention is a deep learning object detection model based on YOLOv5. The object detection model is trained using training sample set X and sample set Y. The training sample set X is labeled with devices, the sample set Y is labeled with the human body of the staff, and the sample set Y is labeled with the head of the staff, so as to accurately identify three types of target objects with an accuracy rate of more than 95%.

[0042] The above five items constitute the data preparation stage, which involves acquiring sample sets of human bodies, human heads, and devices, and analyzing their features. The purpose is to identify three specific objects, rather than other objects, based on these features in actual everyday camera footage.

[0043] A flowchart of a technical solution for determining the effectiveness of a single piece of equipment inspection, such as... Figure 1 As shown, the specific steps of the present invention are as follows: Step 1: Data Preparation Stage: Use cameras to acquire initial images of each device within the monitoring range during unmanned inspections, use image recognition technology to identify the coordinates of each device in the initial images, and form a static dataset S of device locations; use a rangefinder to measure the distance A from the location of the QR code paper label on the device to the camera.

[0044] The first step is to obtain the coordinates of the device's location. Since the device's location is relatively fixed, this only needs to be obtained once. However, the human body and head are constantly in motion, so subsequent steps require real-time acquisition and continuous updating. The task in step one is to obtain the device's location coordinates at its installed location and match them to the device ID (maintained automatically or manually) for use in subsequent steps.

[0045] Image recognition technology is used to identify the coordinates of a target device within the camera's view, resulting in a static device location dataset S. Each data point in the static device location dataset S represents a identified device sample. This data is normally acquired only once as the device's static parameters, and the actual device encoding is manually maintained. When the device or camera moves, the static device location dataset S needs to be acquired again. Figure 2 The image shows an example of obtaining the coordinate data of a specific camera monitoring device: {"attributes": {"name":"Device 1"},{"location": { "top": 702, "left":946, "width": 631, "height": 346}}, {"attributes": {"name":"Device 2"},{"location": { "top": 218, "left":555, "width": 286, "height": 225}}, ...... {"attributes": {"name":"Device sn"},{"location": { "top": tsn,"left":lsn, "width": wsn, "height": hsn}} In this context, "attributes" represents the attribute key name, "name" represents the device name, "location" represents the coordinates, "top": the key means the number of pixels from the top of the image, "left": the key means the number of pixels from the leftmost edge of the image, "height": the key means the height of the target, "width": the key means the width of the target, and tsn, lsn, wsn, and hsn are the actual values.

[0046] Image recognition technology includes: 1. Preparing samples, manually labeling which objects are people, which are people's heads, and which are devices, and defining labels for classification; 2. Inputting the prepared samples into the object detection model, which will then identify which graphic features represent the required devices, people, and heads; after multiple rounds of training, accuracy is improved; 3. In actual daily shooting, the image + object detection model can automatically identify the required devices, people, and heads, and the output results include: the coordinates of the identified target in the image, the label, and the accuracy rate.

[0047] "top", "left", "width", and "height" are the coordinates of the device, person, or head, which are identified by the model. The model also identifies the coordinates and target type, and marks them on the image based on the coordinates.

[0048] Calculate the camera focal length f: Select a reference object to measure key parameters, such as... Figure 2 Using the fire hydrant as a reference point, measure the height W of the fire hydrant. Use a rangefinder to measure the distance d from the midpoint of the fire hydrant's height to the camera. Measure the height w (in pixels) of the fire hydrant in the initial image. Figure 3 As shown, according to the imaging principle, W / w=d / f, we get the focal length f=w*d / W, where the unit of f is pixels.

[0049] Step 2: When the inspection personnel scan the QR code paper label on the target device, record the scanning time, obtain at least N video images captured by the camera during the scanning time, identify the coordinates of the inspection personnel's head position and body position in the video images, and obtain the head position dataset S1 and body position dataset S2 respectively.

[0050] Management requires inspectors to scan the QR code. The system records the scanning time. Inspectors are required to stand on the side of the equipment closest to the QR code paper label, scan the QR code paper label directly, and keep the distance between the person and the QR code paper label less than 1 meter when scanning with their mobile phone. They must fill in the current equipment inspection record in place and stay for at least 5 seconds.

[0051] Capture 5 frames from the video based on the code scanning time (the quantity will be adjusted according to actual tests), one frame per second for 5 consecutive seconds, and use image recognition technology to identify the position of the human head, as Figure 4 shown in the figure, the coordinates of the head of the inspection personnel in the captured image are acquired to form a set S1, and each piece of data represents an identified human head sample: {"attributes": {"name":"Head 1"},{"location": { "top": 815, "left":1041, "width": 36, "height": 52}}, {"attributes": {"name":"Head 2"},{"location": { "top": tt2, "left":lt2, "width": wt2, "height": ht2}}, ...... {"attributes": {"name":"Head tn"},{"location": { "top": ttn,"left":ltn, "width": wtn, "height": htn}} where tn is the number of human heads, and ttn, ltn, wtn, and htn respectively represent the number of pixels of the human head from the topmost edge of the image, the number of pixels from the leftmost edge of the image, the height of the human head, and the width of the human head.

[0052] Use image recognition technology to identify the position of the human body, as Figure 5 shown in the figure, the coordinates of the inspection personnel in each captured image are acquired to form a human body position dataset S2, and each piece of data represents an identified human sample: {"attributes": {"name":"Person 1"},{"location": { "top": 815,"left": 1017,"width": 87, "height": 232}}, {"attributes": {"name":"Person 2"},{"location": { "top": t2, "left": l2,"width": w2, "height": h2}}, ...... {"attributes": {"name":"person rn"},{"location": { "top": trn,"left":lrn, "width": wrn, "height": hrn}} Wherein, rn is the number of human bodies, and ttn, ltn, wtn, and htn respectively represent the number of pixels of the human body from the topmost end of the image, the number of pixels from the leftmost end of the image, the height of the human body, and the width of the human body.

[0053] Step 3: Calculate the area coincidence degree of each sample in the head position dataset S1 with all samples in the human body position dataset S2, and map and match the same inspection personnel according to the area coincidence degree.

[0054] Map and match the set S1 and S2 to the same person, and calculate the coincidence degree between the two head position datasets S1 and the human body position dataset S2 in step 2. A single sample in the head position dataset S1 is matched and calculated with all samples in the human body position dataset S2, and the sample in the human body position dataset S2 with the highest coincidence degree is selected and considered to be the same person. Taking the existing measured values as an example, the calculation method is as follows: 1) It is known that the upper-left corner coordinates and lower-right corner coordinates of the two rectangles of the sample in the head position dataset S1 and the sample in the human body position dataset S2 are (x1, y1) and (x2, y2), and (a1, b1) and (a2, b2), respectively, then the overlapping area of these two rectangles can be calculated by the following formula: M_coin=(min(x2,a2) - max(x1,a1)) * (min(y2,b2) -max(y1,b1)) =(min(1077,1104)-max(1041,1017))*(min(867,1047)-max(815,815))=(1077-1041)*(867-815)=36*52=1872; wherein, the upper-left corner coordinate (x1, y1) of "Head 1" = (left, top) = (1041,815), the lower-right corner coordinate (x2, y2) of "Head 1" = (left+width, top+height) = (1041+36, 815+52) = (1077, 867), the upper-left corner coordinate (a1, b1) of "Person 1" = (left, top) = (1017, 815), the upper-right corner coordinate (a2, b2) of "Person 1" = (left+width, top+height) = (1017+87, 815+232) = (1104, 1047). If the value of the overlapping area M_coin is negative, it means that the two rectangles do not overlap.

[0055] 2) The head area M_head = width * height = (x2 - x1) * (y2 - y1) and = 36 * 52 = 1872; 3) Area overlap = M_coin / M_head = 1872 / 1872 = 1.

[0056] The above calculation of area overlap shows that the person's coordinates and head coordinates match to be the same actual person.

[0057] Step 4: Calculate the distance from the inspector's head to the camera based on the head width of the samples in the head position dataset S1. Use the difference between distance A and distance to filter out the samples that meet the conditions and form the sample set R1.

[0058] In step two, the image is used to identify the width of the inspector's head in the camera image, M (width). The actual head width of most adults is 13-15cm. However, due to tilting of the image or wearing a hat, the M value may appear larger. Therefore, the actual head width is taken as 15cm. The distance from the person's head to the camera is calculated using the camera imaging principle formula: distance = actual head width 15cm * camera focal length f / inspector's head width M in the camera image.

[0059] The difference between the distance value and the distance A in the preparation work should be less than 150cm (this will be fine-tuned when optimizing accuracy). That is, the distance between the person's head and the QR code should be less than 150cm. Based on A-distince>=0 and A-distince<150, a sample set R1 that meets the conditions is selected. The sample set R1 is the head position dataset S1 automatically identified in the actual captured image after being filtered by the conditions A-distince>=0 and A-distince<150, thus determining that the position of the person to the camera is close to the distance between the device and the camera.

[0060] Step 5: Calculate the area overlap between the human body location dataset S2 of the sample mapping and matching of the same inspector in sample set R1 and the area overlap of each device in the static device location dataset S. Select samples with an area overlap of more than 50% to form set R2. If the number of samples in set R2 exceeds 0.8N, the inspection by the inspector is valid; otherwise, the inspection by the inspector is invalid.

[0061] Based on the selected sample set R1, the obtained human body coordinates are... Figure 6 The blue-framed area and the device location are the coordinates of the device in the static dataset S. Figure 6The areas defined in red are compared, the overlapping area is calculated, and the set R2 with an area overlap of more than 50% is selected. The method for calculating the overlap is the same as in step three.

[0062] If set R2 is not empty, the current single photo is considered to have been inspected effectively; if set R2 is empty, the current single photo is considered to have been inspected effectively. Repeat steps four and five for five photos from the same device, calculating the number of valid inspections N1. If N1 > 3 (validity rate >= 80%), the inspection is considered valid; if N1 <= 3, the inspection is considered invalid. This step aims to prevent passersby from being identified as inspectors, thus avoiding result bias.

[0063] If the factory is equipped with cameras connected to an edge gateway, real-time calculations can be used to determine the validity of the current inspection. If the factory is equipped with ordinary cameras, offline timed calculations can be used to periodically batch-judge the validity of the inspection after each inspection. Camera rotation requires re-acquiring device coordinate data to determine the static dataset S of the device position.

[0064] Example 2 A device for preventing cheating during workshop equipment scanning inspection includes a camera and QR code paper labels. The QR code paper labels are fixed to the corresponding equipment. The camera is set at the top of the workshop, and the equipment is within the camera's monitoring range. The camera is connected to an edge server, and the edge server is connected to a mobile terminal. The mobile terminal matches the QR code paper label on the equipment. The edge gateway built-in module in the edge server executes the method for preventing cheating during workshop equipment scanning inspection described in Example 1.

[0065] Example 3 A system for preventing cheating during workshop equipment barcode scanning inspection includes an edge server. The edge server houses an edge gateway management cloud platform, which in turn includes an image acquisition module, an image recognition module, a data calculation module, a data analysis module, and an inspection alarm module. The image acquisition module acquires video images captured by the camera every second after the barcode scanning time. The image recognition module identifies equipment, people, or heads in the images and extracts their position coordinates. The data calculation module calculates the area overlap between the head in head position dataset S1 and the human body in human body position dataset S2, the area overlap between the human body in human body position dataset S2 with the highest area overlap and the equipment in static equipment position dataset S, and the distance between the head and the camera in head position dataset S1. The data analysis module filters samples that meet the conditions based on the calculation results and determines whether the inspection is valid. The inspection alarm module generates an alarm message indicating that the inspection operation is invalid based on the judgment results of the data analysis module.

[0066] The edge gateway management cloud platform also includes: Device activation module: Activates registered edge gateway devices on the edge gateway management platform. Edge gateway devices include cameras.

[0067] Application functional module: Upload the trained object detection model so that other edge servers can download and apply it on the edge gateway management platform.

[0068] Sub-device function module: Used to add camera sub-devices.

[0069] Sub-device viewing function: Select the application, enter the application parameters, such as device coordinates, focal length parameters, etc. After the first submission, the system will automatically save and submit the viewing settings; click on alarm to view the detection results.

[0070] After submitting the viewing settings, open the video page to verify the anti-cheating method for workshop equipment scanning inspection described in Example 1.

[0071] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for preventing cheating during workshop equipment barcode scanning inspection, characterized in that, The steps are as follows: Step 1: Use a camera to acquire initial images of each device within the monitoring range during unmanned inspection; use image recognition technology to identify the position coordinates of each device in the initial image, forming a static dataset S of device positions; use a rangefinder to measure the distance A from the position of the QR code paper label on the device to the camera. Step 2: When the inspection personnel scan the QR code paper label on the device, at least N camera images are captured in the video within the scanning time of N seconds. Image recognition technology is used to identify the position coordinates of the inspection personnel's head and body in the camera images, and the head position dataset S1 and body position dataset S2 are obtained. Step 3: Calculate the area overlap between each sample in the head position dataset S1 and all samples in the human body position dataset S2, and match the same inspector based on the area overlap. Step 4: Calculate the distance from the inspector's head to the camera based on the width of each head in the head position dataset S1. Use the difference between distance A and distance to filter out the camera images corresponding to the samples that meet the conditions, and form a sample set R1. Step 5: Calculate the area overlap between the human body location dataset S2 of the same inspector in the sample set R1 and the equipment location static dataset S. Select samples with an area overlap of more than 50% to form set R2. If the number of samples in set R2 exceeds 0.8N, the inspection by the inspector is valid. Otherwise, the inspection by scanning the code will be invalid. The difference between the distance and the distance A should be less than 150cm. The sample set R1 that meets the conditions is selected based on A-distince>=0 and A-distince<150. On sample set R1, the obtained human body position coordinates are compared with the device position coordinates in the static device position dataset S, the overlapping area is calculated, and samples with an area overlap of more than 50% are selected to form set R2. If set R2 is not empty, the actual inspection of the current camera image is considered valid; if set R2 is empty, the actual inspection of the current camera image is considered invalid. Repeat steps four and five for five camera images from the same device and calculate the number of valid inspections N1. If N1 > 0.8N, the inspection is considered valid; if N1 <= 0.8N, the inspection is considered invalid.

2. The method for preventing cheating during workshop equipment barcode scanning inspection according to claim 1, characterized in that, The image recognition technology is implemented through a deep learning object detection model, which is a YOLOv5-based deep learning object detection model. The YOLOv5-based deep learning object detection model is trained using a training sample set X and a sample set Y. The training sample set X includes multiple on-site images captured by a camera, and the sample set Y includes multiple on-site images of the equipment to be inspected. The equipment is labeled in the training sample set X, and the human body of the inspection personnel is labeled in the sample set Y, with the head of the inspection personnel also labeled.

3. The method for preventing cheating during workshop equipment barcode scanning inspection according to claim 2, characterized in that, Each data point in the static device location dataset S represents the location coordinates of an identified device. The location coordinates include the number of pixels the device is from the top of the initial image, the number of pixels it is from the leftmost edge of the initial image, the height of the device, and the width of the device. The location coordinates in the head location dataset S1 include the number of pixels the head is from the top of the camera image, the number of pixels it is from the leftmost edge of the camera image, the height of the head, and the width of the head. The location coordinates in the human body location dataset S2 include the number of pixels the human body is from the top of the camera image, the number of pixels it is from the leftmost edge of the camera image, the height of the human body, and the width of the human body.

4. The method for preventing cheating during workshop equipment barcode scanning inspection according to any one of claims 1-3, characterized in that, The method for calculating the area overlap in step three is as follows: 1) Given that the top-left and bottom-right corner coordinates of the two rectangles containing the head in head location dataset S1 and the human body in human body location dataset S2 are (x1, y1) and (x2, y2) respectively, and (a1, b1) and (a2, b2) respectively; then the overlapping area of ​​the two rectangles is: M_coin=(min(x2,a2) - max(x1,a1)) * (min(y2,b2) -max(y1,b1)); 2) The area of ​​the head, M_head = width * height = (x2 - x1) * (y2 - y1); 3) Area overlap = M_coin / M_head.

5. The method for preventing cheating during workshop equipment barcode scanning inspection according to claim 4, characterized in that, The method for calculating the distance from the inspector's head to the camera is as follows: obtain the head width M=width in the camera image from the head position dataset S1, take the actual head width as 15cm, and according to the camera imaging principle formula, the distance = actual head width 15cm * camera focal length f / head width M.

6. The method for preventing cheating during workshop equipment barcode scanning inspection according to claim 5, characterized in that, The method for calculating the camera focal length f is as follows: Select a fixed object in the initial image as a reference object, measure the height W of the reference object, measure the distance d from the middle position of the reference object to the camera using a rangefinder, and measure the height w of the fire hydrant in the initial image, in pixels; according to the imaging principle: W / w=d / f, the camera focal length f=w*d / W, in pixels. Inspection personnel are required to stand on the side of the equipment closest to the QR code paper label, scan the QR code paper label directly, keep the distance between the person and the QR code paper label less than 1 meter when scanning with a mobile phone, fill in the current equipment inspection record on the spot, and stay for at least 5 seconds.

7. A device for preventing cheating during workshop equipment barcode scanning inspection, characterized in that, The system includes a camera and a QR code paper label. The QR code paper label is fixed to the equipment that needs to be monitored. The camera is set at the top of the workshop, and the equipment is within the camera's monitoring range. The camera is connected to an edge server, and the edge server is connected to a mobile terminal. The mobile terminal is matched with the QR code paper label on the equipment. The edge gateway built-in module in the edge server executes the workshop equipment scanning inspection anti-cheating method as described in any one of claims 1-6.

8. A system for preventing cheating during workshop equipment barcode scanning inspection, characterized in that, The method for preventing cheating during workshop equipment barcode scanning inspection according to any one of claims 1-6 includes an edge server. The edge server is equipped with an edge gateway management cloud platform. The edge gateway management cloud platform includes an image acquisition module, an image recognition module, a data calculation module, a data analysis module, and an inspection alarm module. The image acquisition module acquires video images captured by the camera every second after the barcode scanning time. The image recognition module identifies equipment, people, or heads in the images using image recognition technology and extracts the position coordinates of the equipment, people, or heads. The data calculation module calculates the area overlap between the head in head position dataset S1 and the human body in human body position dataset S2, the area overlap between the human body in human body position dataset S2 with the highest area overlap and the equipment in static equipment position dataset S, and the distance between the head and the camera in head position dataset S1. The data analysis module filters out samples that meet the conditions based on the calculation results of the data calculation module and determines whether the inspection is effective. The inspection alarm module generates an alarm message indicating that the inspection operation is invalid based on the judgment results of the data analysis module.

Images