Method for detecting call answering and making behaviors of bid evaluation room based on target detection

Abstract

The invention relates to a bid evaluation room call receiving and making behavior detection method based on target detection. The method comprises the steps of obtaining a target image corresponding to a bid evaluation room scene, performing face area positioning and expansion, performing call receiving and making behavior action detection, and judging whether a call receiving and making behavior exists in the bid evaluation room scene. The method has the beneficial effects that the structure of a YOLOv5 target detection network is optimized, the method is used for detecting targets such as human faces and calls, and the problem of large target scale change is solved; in addition, a data labeling strategy is improved, behavior action samples such as cheek supporting and face touching are added, the behavior action samples serve as a target category, comprehensive training is conducted, and false alarms of call making behaviors are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and more specifically, to a method for detecting phone call behavior in a bid evaluation room based on target detection. Background Art

[0002] With the development of information technology, bidding work has changed from the traditional on-site voting form of paper documents to electronic online bidding. In order to improve the efficiency of bid evaluation, bidding companies have been actively exploring intensive bid evaluation, remote bid evaluation and other methods. Remote bid evaluation can effectively reduce the gathering of bid evaluation personnel and the uneven distribution of bid evaluation expert resources. In order to effectively eliminate violations in the bid evaluation process and provide a more open, fair and just competition environment, it is necessary to establish an information-based supervision and management platform centered on intelligent monitoring and AI intelligent analysis, of which AI-based intelligent bid evaluation personnel's phone behavior detection is an important component.

[0003] The phone call detection algorithm was previously mainly used to monitor the driver's behavior during driving. By alerting the driver of the phone call behavior, accidents can be effectively prevented. At present, the main problems with phone call detection are that the target is blocked, the mobile phone target is small, and the environment is relatively complex. In the past, the phone call detection algorithm used a face positioning algorithm to assist in locating the key target area. However, in the bid evaluation process, the evaluators have different shapes, and the camera may capture the front face or the side face. Due to the differences in terminal device parameters and the distance of video acquisition, there are also differences in the scale of the recognition target. Moreover, in the bid evaluation environment, the monitored object has interference actions such as holding the chin and touching the face, which can easily cause false alarms. Summary of the invention

[0004] The purpose of the present invention is to address the deficiencies of the prior art and to propose a method for detecting the behavior of making and receiving phone calls in a bidding evaluation room based on target detection.

[0005] In the first aspect, a method for detecting the behavior of making and receiving phone calls in a bidding evaluation room based on target detection is provided, comprising: Step 1, obtaining a target image corresponding to the bid evaluation room scene; Step 2: locate the face area of ​​the target image to obtain a collection of face positions; Step 3: Expand the face area to obtain an expanded face area image; Step 4: Detect the action of making or receiving a phone call on the extended face area image, and output a label and a framed area; Step 5: When the output tag type is a phone, the detection result corresponding to the extended face area image is judged to be correct; when the output tag type is a hand, the detection result corresponding to the extended face area image is judged to be wrong; if the extended face area image does not detect any target, the detection result corresponding to the extended face area image is judged to be wrong; Step 6: Based on the detection result of the extended face area image, determine whether there is any behavior of making or receiving phone calls in the bid evaluation room scene.

[0006] Preferably, in step 6, the target image includes multiple frame images within a unit time. If the detection result of the extended face area image corresponding to any frame image is correct, then the detection result corresponding to the target image is correct; if the detection results of the target image within the preset time threshold are all correct, it is determined whether there is any behavior of making or receiving a phone call in the evaluation room scene.

[0007] Preferably, in step 2, the face position set includes face regions of different forms.

[0008] Preferably, in step 2 and step 4, an improved YOLOv5 model is used as the target detection network, a Transformer encoding module is provided at the end of the trunk part of the improved YOLOv5, and a newly added detection head is provided in the prediction part of the improved YOLOv5.

[0009] Preferably, in step 4, the improved YOLOv5 model is trained with the labeled data set to obtain a phone call action detection model; in the labeled data set, images having both hand and phone elements are labeled as phone categories, and images having only hand elements are labeled as hand categories.

[0010] Preferably, in step 4, the image having only hand elements includes a face-touching action image and a chin-supporting action image.

[0011] In a second aspect, a target detection-based system for detecting the behavior of making and receiving phone calls in a bidding evaluation room is provided, which is used to execute any of the methods described in the first aspect, including: An acquisition module, used to acquire a target image corresponding to the bid evaluation room scene; A positioning module, used to locate the face area of ​​the target image and obtain a collection of face positions; An expansion module, used to expand the face area and obtain an expanded face area image; A detection module, used to detect the action of making or receiving a phone call on the extended face area image, and output a label and a framed area; A judgment module, used for judging that the detection result corresponding to the extended face area image is correct when the output label type is a phone; judging that the detection result corresponding to the extended face area image is wrong when the output label type is a hand; and judging that the detection result corresponding to the extended face area image is wrong if no target is detected in the extended face area image; The determination module is used to determine whether there is any behavior of making or receiving a phone call in the bid evaluation room scene based on the detection result of the extended face area image.

[0012] According to a third aspect, a computer storage medium is provided, wherein a computer program is stored in the computer storage medium; when the computer program is executed on a computer, the computer executes any method described in the first aspect.

[0013] In a fourth aspect, an electronic device is provided, including: Memory, used to store computer programs; A processor is used to execute the computer program to implement any method as described in the first aspect.

[0014] The beneficial effects of the present invention are: 1. The present invention optimizes the YOLOv5 target detection network structure for detecting targets such as faces and phones, solving the problem of large changes in target scales; and the present invention improves the data annotation strategy, adds behavioral action samples such as chin-supporting and face-touching, and uses them as a target category for comprehensive training to reduce false alarms for phone calls.

[0015] 2. The present invention expands the face area to include the area where the action of making or receiving a phone call may occur, performs making or receiving a phone call target detection in the area, and comprehensively determines and analyzes whether there is any making or receiving a phone call behavior.

[0016] 3. The present invention adopts a recognition strategy of multiple recognitions within a unit time, and only considers the existence of the behavior of making or receiving a phone call after a certain period of time, so as to reduce the false detection rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flow chart of a method for detecting the behavior of making and receiving phone calls in a bidding evaluation room based on target detection provided by an embodiment of the present invention; Figure 2 An optimized network structure diagram provided for an embodiment of the present invention; Figure 3 A flowchart of the call receiving and making behavior detection provided by an embodiment of the present invention; Figure 4 A schematic diagram of detection effects of different sight distance detection provided by an embodiment of the present invention; Figure 5A schematic diagram of the detection effect of interference item elimination provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The present invention is further described below in conjunction with embodiments. The description of the following embodiments is only used to help understand the present invention. It should be noted that for ordinary persons in the art, without departing from the principle of the present invention, the present invention can also be modified in some ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0019] Embodiment 1: The present invention is mainly used to identify and detect the behavior of bid evaluation personnel making and receiving phone calls in the bid evaluation room. Since there are many types of hardware devices used for monitoring in the bid evaluation room, including computer cameras and surveillance cameras, and the scenes collected are different, the target scale varies greatly and is easily blocked. In view of the above problems, Figure 1 As shown, Example 1 of the present application provides a method for detecting the behavior of making and receiving phone calls in a bidding evaluation room based on target detection, including: Step 1: Obtain the target image corresponding to the bid evaluation room scene.

[0020] In step 1, the target image includes multiple frame images within a unit time (such as one minute).

[0021] Step 2: locate the face area of ​​the target image to obtain a collection of face positions.

[0022] In step 2, the face region is located in any frame of the target image to obtain a collection of face positions. In addition, the face position set includes face regions of different forms, such as frontal face and side face.

[0023] Step 3: Expand the face area to obtain an expanded face area image.

[0024] Because during the process of making or receiving a phone call, the phone may appear on the side of the face or below the face. After the face area is obtained through forward reasoning of a single frame image, the frame selection area output by the model needs to be expanded so that the expanded area includes the location where the phone may appear.

[0025] The expanded formula is:

[0026] in, , are the coordinates of the upper left corner and the lower right corner before the rectangular area is expanded. , are the coordinates of the upper left corner and the lower right corner after the region is expanded. width and height respectively represent the width and height of the rectangle before expansion. The value range of K is [0,1]. For small targets (width or height less than 256), K is usually 1; for large targets (width and height greater than or equal to 256), K is usually 0.3.

[0027] Step 4: Perform phone call action detection on the extended face area image, and output a label and a framed area.

[0028] In step 4, the tag collection obtained is For example, Figure 4 As shown, the target images acquired at different viewing distances are detected. Figure 4 The yellow frame in the figure indicates the detected face area, and the green frame indicates the detected call action area.

[0029] Step 5: When the output label type is phone, the detection result corresponding to the extended face area image is judged to be correct, which is expressed as: ; When the output label type is hand, it is judged that the detection result corresponding to the extended face area image is wrong, which is expressed as: If no target is detected in the extended face region image, the detection result corresponding to the extended face region image is judged to be an error, which is expressed as .

[0030] For example, Figure 5 As shown, interference items such as touching the face and supporting the chin can be eliminated. Figure 5 In the figure, the yellow frame indicates the detected face area, and the green frame indicates the detected call action area.

[0031] Step 6: Based on the detection result of the extended face area image, determine whether there is any behavior of making or receiving phone calls in the bid evaluation room scene.

[0032] In step 6, the target image includes multiple frames of images within a unit time. If the detection result of the extended face area image corresponding to any frame of image is correct, the detection result corresponding to the target image is correct; if the detection results of the target images within the preset time threshold are all correct, it is determined whether there is any behavior of making or receiving a phone call in the bid evaluation room scene. For example, the detection results of multiple frames of images within one second are comprehensively analyzed. If there is a detection result in one frame of image , then the detection result corresponding to that second is ,The collection of test results within five seconds is If the action of making or receiving a phone call is detected within five consecutive seconds, it is determined that the scene contains the action of making or receiving a phone call.

[0033] It should be noted that in step 6, you can also select a three-second or ten-second video for analysis based on actual detection needs, or you can perform algorithm analysis at one-second intervals and conduct a comprehensive analysis after multiple judgments.

[0034] Embodiment 2: Based on Example 1, Example 2 of the present application provides a more specific method for detecting the behavior of making and receiving phone calls in the bidding evaluation room based on target detection, including: Step 1: Based on the improved YOLOv5 model, obtain the face detection model Model1 and the phone call action detection model Model2.

[0035] As a classic target detection network, YOLOv5 adds adaptive anchor frame calculation, uses CSPDarknet53 as the backbone of feature extraction, draws on PANet for feature fusion, and finally detects targets at three different scales. In the evaluation room detection environment, the target detection network is mainly used to identify two key elements, faces and phones, in indoor scenes. Due to different video acquisition devices and shooting distances, the scales of faces and phones in the images vary greatly. And affected by the shooting angle, the collected faces are frontal, side faces, etc., and the phone is easily blocked.

[0036] In order to solve the above problems, in step 1, the present application improves the YOLOv5 model. Specifically, the present invention modifies the original network structure. The YOLOv5 model includes a trunk part, a neck part and a prediction part. The present application adds a Transformer encoding module at the end of the trunk part, which expands the receptive field on the feature map, is conducive to obtaining the global information of the image, and enhances the feature perception ability of the network. In the prediction part, a new detection head is added, the tensor size is 152*152*21, and the output resolution is larger to correspond to smaller recognition targets, so that the network's detection ability for small targets becomes stronger. The specific network structure of the improved YOLOv5 model is as follows: Figure 2 shown.

[0037] In addition, during the bid evaluation process, bid evaluators often touch their faces, support their chins, and other actions, which can easily be misjudged as making or receiving phone calls. Therefore, the present invention improves the labeling strategy of the data set used for model training. In the data set, negative sample targets are artificially added, and actions such as touching the face and supporting the chins are labeled as the "hand" category. Only when both the hand and the phone are present will it be judged as the behavior of making or receiving phone calls, that is, the action of holding the phone is labeled as the "phone" category. Through training, the model can distinguish the differences between different action categories and output the action labels in the image. A comprehensive analysis is performed based on the different types of output labels to eliminate interfering actions, so as to reduce the algorithm's misjudgment rate of making or receiving phone calls.

[0038] The training uses images collected from the actual bidding room scene, annotates and creates a data set, loads the pre-trained weight model file, transfers learning on the improved YOLOv5 model, and fine-tunes the training. According to the different stages of model use in the invention, different training sets are annotated to obtain Model 1 for face detection and Model 2 for phone action detection. Model 1 is used to search for the face position in the entire image (i.e., step 3 of this embodiment), and Model 2 is used to search for phone actions in the cropped image (i.e., step 5 of this embodiment).

[0039] Step 2: Obtain the target image corresponding to the bid evaluation room scene.

[0040] Step 3: locate the face area of ​​the target image to obtain a collection of face positions.

[0041] Step 4: Expand the face region to obtain an expanded face region image.

[0042] like Figure 3 As shown, the present application crops the original image according to the coordinates of the expanded area, and inputs the cropped image into the call receiving and making phone calls detection model Model2 for locating the call receiving and making phone calls actions.

[0043] Step 5: Detect the phone call behavior on the extended face area image, and output a label and a framed area.

[0044] Step 6. When the output label type is a phone, the detection result corresponding to the extended face area image is judged to be correct; when the output label type is a hand, the detection result corresponding to the extended face area image is judged to be wrong; if the extended face area image does not detect any target, the detection result corresponding to the extended face area image is judged to be wrong.

[0045] Specifically, after obtaining the face frame area and the phone call action frame area through a single model or multiple models, the intersection-and-union ratio of the two frame areas is calculated. When the intersection-and-union ratio value is greater than a preset threshold T, it is considered that there is a phone call behavior in the image.

[0046] Step 7: Based on the detection result of the extended face area image, determine whether there is any behavior of making or receiving phone calls in the bid evaluation room scene.

[0047] The action of making or receiving a phone call is generally continuous. In order to improve the accuracy of the algorithm in identifying the action of making or receiving a phone call, the algorithm performs frame extraction analysis on the video stream within one second, extracts multiple frames of images for algorithm analysis and detection, and if one of the multiple frames detects the action of making or receiving a phone call, it is determined that there is a suspected action of making or receiving a phone call. If the action of making or receiving a phone call is detected in every frame extraction detection in the next five seconds, it is determined that there is the behavior of making or receiving a phone call in the detection scene.

[0048] It should be noted that the parts in this embodiment that are the same or similar to those in Embodiment 1 can be referenced to each other and will not be described in detail in this application.

[0049] Embodiment 3: Based on Examples 1 and 2, Example 3 of the present application provides a system for detecting phone calls in a bidding evaluation room based on target detection, including: An acquisition module, used to acquire a target image corresponding to the bid evaluation room scene; A positioning module, used to locate the face area of ​​the target image and obtain a collection of face positions; An expansion module, used to expand the face area and obtain an expanded face area image; A detection module, used to detect the action of making or receiving a phone call on the extended face area image, and output a label and a framed area; A judgment module, used for judging that the detection result corresponding to the extended face area image is correct when the output label type is a phone; judging that the detection result corresponding to the extended face area image is wrong when the output label type is a hand; and judging that the detection result corresponding to the extended face area image is wrong if no target is detected in the extended face area image; The determination module is used to determine whether there is any behavior of making or receiving a phone call in the bid evaluation room scene based on the detection result of the extended face area image.

[0050] Specifically, the system provided in this embodiment is a system corresponding to the method provided in Embodiments 1 and 2. Therefore, the parts in this embodiment that are the same or similar to Embodiments 1 and 2 can be referenced to each other and will not be repeated in this application.

Claims

1. A method for detecting the behavior of making and receiving phone calls in a bidding evaluation room based on target detection, characterized in that: include: Step 1, obtaining a target image corresponding to the bid evaluation room scene; Step 2: locate the face area of ​​the target image to obtain a collection of face positions; Step 3: Expand the face area to obtain an expanded face area image; Step 4: Detect the action of making or receiving a phone call on the extended face area image, and output a label and a framed area; Step 5: When the output tag type is phone, the detection result corresponding to the extended face area image is determined to be correct; when the output tag type is hand, the detection result corresponding to the extended face area image is determined to be wrong; If no target is detected in the extended face region image, determining that the detection result corresponding to the extended face region image is an error; Step 6: Based on the detection result of the extended face area image, determine whether there is any behavior of making or receiving phone calls in the bid evaluation room scene.

2. According to the target detection-based method for detecting the behavior of making and receiving phone calls in the bidding evaluation room of claim 1, it is characterized in that: In step 6, the target image includes multiple frame images within a unit time, and if the detection result of the extended face area image corresponding to any frame image is correct, then the detection result corresponding to the target image is correct; If the detection results of the target image within the preset time threshold are all correct, it is determined whether there is any behavior of making or receiving phone calls in the bid evaluation room scene.

3. The method for detecting the behavior of making and receiving phone calls in a bidding evaluation room based on target detection according to claim 2 is characterized in that: In step 2, the face position set includes face regions of different shapes.

4. The method for detecting the behavior of making and receiving phone calls in the bidding evaluation room based on target detection according to claim 3 is characterized in that: In step 2 and step 4, an improved YOLOv5 model is used as the target detection network, a Transformer encoding module is provided at the end of the trunk part of the improved YOLOv5, and a newly added detection head is provided in the prediction part of the improved YOLOv5.

5. The method for detecting the behavior of making and receiving phone calls in a bidding evaluation room based on target detection according to claim 4 is characterized in that: In step 4, the improved YOLOv5 model is trained with the labeled data set to obtain a phone call action detection model; in the labeled data set, images with both hand and phone elements are labeled as phone categories, and images with only hand elements are labeled as hand categories.

6. The method for detecting the behavior of making and receiving phone calls in a bidding evaluation room based on target detection according to claim 5 is characterized in that: In step 4, the image having only hand elements includes a face-touching action image and a chin-supporting action image.

7. A target detection-based system for detecting phone calls in a bidding room, characterized in that: Used to perform the method according to any one of claims 1 to 6, comprising: An acquisition module, used to acquire a target image corresponding to the bid evaluation room scene; A positioning module, used to locate the face area of ​​the target image and obtain a collection of face positions; An expansion module, used to expand the face area and obtain an expanded face area image; A detection module, used to detect the action of making or receiving a phone call on the extended face area image, and output a label and a framed area; A judgment module, used for judging that the detection result corresponding to the extended face area image is correct when the output label type is a phone; judging that the detection result corresponding to the extended face area image is wrong when the output label type is a hand; and judging that the detection result corresponding to the extended face area image is wrong if no target is detected in the extended face area image; The determination module is used to determine whether there is any behavior of making or receiving a phone call in the bid evaluation room scene based on the detection result of the extended face area image.

8. A computer storage medium, characterized in that The computer storage medium stores a computer program; when the computer program is executed on a computer, the computer executes any one of the methods described in claims 1 to 6.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the method according to any one of claims 1 to 6.

Images