Method and device for determining target behavior, pumping machinery and readable storage medium

Abstract

The application provides a target behavior determination method and device, a pumping machine and a readable storage medium. The target behavior determination method comprises the following steps: identifying a target object and a reference object in a target image, and determining position information of the target object and the reference object; determining whether a target behavior exists in the target image according to the position information of the target object and the reference object, converting the identification of the illegal water adding behavior into the identification of the three targets of a person, a tubular object and a hopper screen surface, improving the identification accuracy through the positions of the three targets in the image space, avoiding the illegal water adding in the pumping machine construction process, and further avoiding the segregation phenomenon caused by the excessive water content of the concrete, and ensuring the safety in the construction process.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target detection, in particular to a target behavior determination method and device, a pumping machine and a readable storage medium. BACKGROUND

[0002] In related technologies, the engineering machinery or equipment has high requirements for the concrete to be pumped when pumping, and the problems of the concrete will affect the operation conditions or the engineering machinery and equipment. For example, improper water adding behavior when the pumping machine pumps concrete will cause the occurrence of segregation phenomenon due to excessive water content in the concrete, and easily cause the occurrence of pump blocking. SUMMARY

[0003] The present application aims to at least solve the technical problems in the prior art or related art.

[0004] To this end, the first aspect of the present application provides a target behavior determination method.

[0005] The second aspect of the present application provides a target behavior determination device.

[0006] The third aspect of the present application provides a readable storage medium.

[0007] The fourth aspect of the present application provides an electronic device.

[0008] The fifth aspect of the present application provides a pumping machine.

[0009] Therefore, the first aspect of the present application provides a target behavior determination method, which comprises: identifying a target object and a reference object in a target image, and determining position information of the target object and the reference object; determining whether a target behavior exists in the target image according to the position information of the target object and the reference object.

[0010] In the technical solution of the present application, the edge computing device first acquires a target image, then identifies a target object and a reference object in the target image, and determines the position information of the target object and the reference object in the target image with the target object and the reference object, and finally the edge computing device determines whether a target behavior exists in the target image according to the position information of the target object and the reference object.

[0011] In the technical solution, the target behavior can be a behavior of adding water to the pump truck in violation of rules, and whether the behavior of adding water to the pump truck in violation of rules exists in the target image is determined according to the position information of the target object and the reference object, wherein the target object can be an object for adding water in violation of rules, such as a water inlet pipe, a spray head or other objects capable of delivering water, and the reference object can include an object related to the target object for adding water in violation of rules, such as a hopper screen surface, a person, a water pipe support or other objects related to the target object for adding water in violation of rules. Further, the edge computing device on the pump truck determines the position information of the object for adding water and the object related to the behavior of adding water in violation of rules in the target image, and the edge computing device determines whether the behavior of adding water to the pump truck in violation of rules exists in the target image according to the relationship between the position information of the object for adding water and the object related to the behavior of adding water in violation of rules in the target image, so as to avoid adding water in violation of rules during the construction of the pump truck, and further avoid the occurrence of segregation phenomenon caused by excessive water content of concrete, and ensure the safety during construction.

[0012] In the technical solution, the target image is collected by an RGB (Red Green Blue, optical three primary colors) camera, and after the edge computing device on the pump truck obtains the target image, the edge computing device identifies the target object and the reference object in the target image, and determines the position information of the two objects, and further determines whether the target behavior exists in the target image according to the position information, wherein the identification of the behavior of adding water in violation of rules is converted into the identification of the three targets of a person, a tubular object and a hopper screen surface, and the position of the three targets in the image space is used to improve the accuracy of identification, and the use of a classical algorithm of traditional computer vision reduces the identification cost.

[0013] The target behavior determination method provided by the application can also have the following additional technical features:

[0014] In any of the above technical solutions, the reference object includes a first reference object, and whether the target behavior exists in the target image is determined according to the position information of the target object and the reference object, specifically including: determining a first calibration area of the target object according to the position information of the target object, and determining a second calibration area of the first reference object according to the position information of the first reference object; and determining that the target behavior exists in the target image based on the first coincidence degree between the first calibration area and the second calibration area being greater than or equal to a first preset threshold.

[0015] In the technical solution, the specific steps of determining whether the target behavior exists in the target image according to the position information of the target object and the reference object are as follows: when the edge computing device identifies that the target image includes the target object and the first reference object, the edge computing device determines a first calibration region of the target object and a second calibration region of the first reference object according to the position information of the target object and the position information of the first reference object respectively, the edge computing device calculates a first coincidence degree of the first calibration region and the second calibration region, and when the first coincidence degree is greater than or equal to a first preset threshold, it is determined that the target behavior exists in the target image.

[0016] In the technical solution, the specific steps of determining whether the target behavior exists in the target image according to the position information of the target object and the reference object are as follows: when the edge computing device identifies that the target image includes the target object and the first reference object, the edge computing device determines a first calibration region of the target object and a second calibration region of the first reference object according to the position information of the target object and the position information of the first reference object respectively, the edge computing device calculates a first coincidence degree of the first calibration region and the second calibration region, and when the first coincidence degree is greater than or equal to a first preset threshold, it is determined that the target behavior exists in the target image.

[0017] Specifically, the calculation of the coincidence degree between the first calibration region and the second calibration region can be achieved by calculating the IOU (Intersection Over Union) of the first calibration region and the second calibration region, or calculating the coverage index of the first calibration region.

[0018] In the technical solution, the calculation formula of the IOU of the first calibration region and the second calibration region is as follows.

[0019] The coincidence degree = (the tube rectangular frame ∩ the screen surface rectangular frame) / (the tube rectangular frame ∪ the screen surface rectangular frame);

[0020] Wherein, the tube rectangular frame is the first calibration region, the screen surface rectangular frame is the second calibration region, and the coincidence degree is a value for indicating whether the target behavior exists in the target image.

[0021] In any of the above technical solutions, the calculation formula of the coverage index of the first calibration region is as follows.

[0022] The coincidence degree = (the tube rectangular frame ∩ the screen surface rectangular frame) / the tube rectangular frame;

[0023] Wherein, the tube rectangular frame is the first calibration region, the screen surface rectangular frame is the second calibration region, and the coincidence degree is a value for indicating whether the target behavior exists in the target image.

[0024] In any of the above technical solutions, the position relationship between the target objects is determined by the coincidence degree calculation, that is, whether the water pipe is above the screen surface is determined by the coincidence degree, so as to simplify the determination of the illegal water adding behavior, improve the efficiency and accuracy of the determination.

[0025] In any of the above technical solutions, the reference object includes a first reference object and a second reference object, and whether the target behavior exists in the target image is determined according to the position information of the target object and the reference object, specifically including: determining a first calibration region of the target object according to the position information of the target object, and determining a second calibration region of the first reference object and a third calibration region of the second reference object according to the position information of the first reference object and the second reference object; in a case where a first coincidence degree of the first calibration region and the second calibration region is greater than or equal to a first preset threshold, a second coincidence degree of the first calibration region and the third calibration region is determined; in a case where the second coincidence degree of the third calibration region and the first calibration region is greater than or equal to a second preset threshold, it is determined that the target behavior exists in the target image.

[0026] In the above technical solution, the specific steps of determining whether the target behavior exists in the target image according to the position information of the target object and the reference object are that, in a case where the edge computing device identifies that the target image includes the target object, the first reference object and the second reference object, the first calibration region of the target object, the second calibration region of the first reference object and the third calibration region of the second reference object are determined according to the position information of the target object, the position information of the first reference object and the position information of the second reference object, respectively, the first coincidence degree of the first calibration region and the second calibration region is calculated, in a case where the first coincidence degree is greater than or equal to the first preset threshold, the edge computing device calculates the second coincidence degree of the first calibration region and the third calibration region, and in a case where the second coincidence degree is greater than or equal to the second preset threshold, it is determined that the target behavior exists in the target image.

[0027] In the technical solution, the position information of the target object, the position information of the first reference object and the position information of the second reference object in the target image are determined as follows: the edge computing device determines a target frame of the target object, the first reference object and the second reference object when the edge computing device identifies the target object, the first reference object and the second reference object in the target image, and the target frame is used to indicate the position information of the target object, the first reference object and the second reference object. The target object can be a tubular object for adding water, the first reference object can be a hopper screen surface related to the illegal water adding, and the second reference object can be a person related to the illegal water adding. The edge computing device further identifies the person on the basis of identifying the tubular object and the hopper screen surface in the first picture. Then, the edge computing device calculates a first calibration region of the tubular object and a third calibration region of the person on the basis of calculating a first calibration region of the tubular object and a second calibration region of the hopper screen surface. The edge computing device determines whether the illegal water adding behavior occurs by calculating the coincidence degree among the tubular object, the hopper screen surface and the person, so that the determination of the illegal water adding behavior is more reliable and has a higher confidence.

[0028] In any of the technical solutions, the target object and the reference object in the target image are identified, and the position information of the target object and the reference object is determined, specifically including: establishing a target detection model; inputting the target image into the target detection model to identify the target object in the target image; identifying the reference object and determining the position information of the target object and the reference object when the target object is identified.

[0029] In the technical solution, the position information of the target object and the reference object in the target image are identified, and the position information of the target object and the reference object is determined, specifically including: establishing a target detection model; inputting the target image into the target detection model to identify the target object in the target image; identifying the reference object and determining the position information of the target object and the reference object when the target object is identified.

[0030] In the technical solution, the target detection model includes but is not limited to a YOLO (You Only Look Once) series, an SSD (Single Shot Multi Box Detector) series, and the like. When the target detection model is trained and inference is performed using the target detection model, the category can be set, and then the target detection model is trained and inference is performed using the set category. For example, the category can be set as a tubular object, a hopper screen surface, and a person, wherein the tubular object, the hopper screen surface, and the person correspond to a target object, a first reference object, and a second reference object, respectively. That is, the target detection model in the edge computing device is used to identify the tubular object, the hopper screen surface, and the person, determine a target frame of the tubular object, the hopper screen surface, and the person, and use the target frame to indicate the position information of the target object and the reference object. The classic algorithm of traditional computer vision is used to reduce the identification cost and effectively avoid or warn the illegal water adding situation, which has a high application value.

[0031] In any of the technical solutions, after determining that the target behavior exists in the target image, the method for determining the target behavior further includes: sending a first signal to a controller, the first signal being used to indicate that the target behavior currently exists, and the controller being used to send the target image in which the target behavior exists to a user; and / or issuing an alarm prompt information, the prompt information being used to prompt processing of the target behavior.

[0032] In the technical solution, after determining that the target behavior exists in the target image, the edge computing device sends a first signal to a controller and / or issues an alarm prompt information to prompt processing of the target behavior.

[0033] In the technical solution, the edge computing device transmits the first signal to the controller in a CAN (Controller Area Network) protocol, and the target image can be transmitted back to a database background through the controller, so that the user can timely obtain the target image of the illegal water adding behavior. The alarm prompt information can use a buzzer to alarm, and the pump truck can be equipped with the buzzer to timely alarm the illegal water adding behavior.

[0034] In any of the technical solutions, the method for determining the target behavior further includes: storing a plurality of target images in which the target behavior exists in a target database.

[0035] In the technical solution, the RGB camera continuously captures a plurality of pictures when capturing, and the edge computing device stores a plurality of first pictures in which the target behavior exists in the target database after identifying that the target behavior exists in the pictures, which is convenient for later tracing and avoids disputes for the illegal water adding behavior in the construction process.

[0036] In any of the above technical solutions, before identifying the target object and the reference object in the target image and determining the position information of the target object and the reference object, the target behavior determination method further comprises: acquiring the target image; wherein the target image comprises the target object and / or the reference object.

[0037] In the above technical solution, before identifying the target image, the edge computing device acquires the target image, wherein the target image comprises the target object and / or the reference object.

[0038] Specifically, the target image is acquired by an RGB camera, and the acquired target image is sent to the edge computing device, wherein the target image acquired by the RGB camera can comprise the target object and / or the reference object that needs to be identified by the edge computing device, and the edge computing device acquires the target image and identifies the target object and the reference object in the target image. In the above technical solution, when the target object and the reference object are included in the target image, the edge computing device needs to identify the target object and the reference object and further judge whether there is a target behavior in the target image; when only the target object or the reference object is included in the target image, the edge computing device identifies the target image and judges that there is no target behavior in the target image.

[0039] In any of the above technical solutions, the target behavior specifically comprises a behavior of using the target object to illegally add water to a hopper of a pumping machine.

[0040] In the above technical solution, the target behavior specifically comprises a behavior of using the target object to illegally add water to a hopper of a pumping machine, wherein the illegal water adding behavior can be used on a pump truck, a vehicle-mounted pump and other pumping machines. For example, when the illegal water adding behavior is used on a pump truck, a screen surface above a hopper, a water pipe tubular object and a person are identified and detected, wherein the water pipe tubular object is the target object, and the screen surface above the hopper and the person are the reference objects, and when the overlapping area of the screen surface above the hopper, the water pipe tubular object and the person exceeds a threshold value set by a user, it is determined that there is an illegal water adding behavior to the pump truck.

[0041] The second aspect of the application provides a target behavior determination device, comprising: an identification unit configured to identify a target object and a reference object in a target image and determine position information of the target object and the reference object; and a processing unit configured to determine whether there is a target behavior in the target image according to the position information of the target object and the reference object.

[0042] In the technical solution of the present application, first, the edge computing device acquires a target image, then the recognition unit recognizes the target object and the reference object in the target image, and determines the position information of the target object and the reference object in the target image with the target object and the reference object, and finally the processing unit determines whether there is a target behavior in the target image according to the position information of the target object and the reference object.

[0043] In the above technical solution, the target behavior can be a rule-breaking water adding behavior to the pump truck, vehicle-mounted pump and other pumping machinery. For example, according to the position information of the target object and the reference object, it is determined whether there is a rule-breaking water adding behavior to the pump truck, wherein the target object can be an object for rule-breaking water adding, such as a water inlet pipe, a spray head or other objects capable of transporting water, and the reference object can include objects related to the rule-breaking water adding of the target object, such as a hopper screen surface, a person, a water pipe support or other objects related to the rule-breaking water adding of the target object. Further, the edge computing device on the pump truck determines the position information of the object for water adding and the object related to the rule-breaking water adding in the target image, and the edge computing device determines whether there is a rule-breaking water adding behavior to the pump truck according to the relationship between the position information of the object for water adding and the object related to the rule-breaking water adding in the target image, so as to avoid rule-breaking water adding during the construction process of the pump truck, and further avoid the occurrence of segregation phenomenon caused by excessive water content of concrete, and ensure the safety during the construction process.

[0044] In the above technical solution, the target image is collected by an RGB (Red Green Blue, optical three primary colors) camera, and after the edge computing device acquires the target image, the edge computing device recognizes the target object and the reference object in the target image, and determines the position information of the two objects, and further determines whether there is a target behavior in the target image according to the position information, wherein the identification of the rule-breaking water adding behavior is converted into the identification of the three targets of the person, the tubular object and the hopper screen surface, and the position of the three in the image space is used to improve the accuracy of identification.

[0045] The target behavior determination device provided by the present application can also have the following additional technical features:

[0046] In any of the above technical solutions, the reference object includes a first reference object, and the target behavior in the target image is determined according to the position information of the target object and the reference object, specifically including: determining a first calibration area of the target object according to the position information of the target object, and determining a second calibration area of the first reference object according to the position information of the first reference object; based on the first coincidence degree of the first calibration area and the second calibration area being greater than or equal to a first preset threshold, it is determined that there is a target behavior in the target image.

[0047] In the technical solution, the specific step of determining whether the target behavior exists in the target image according to the position information of the target object and the reference object is that, when the edge computing device identifies that the target image includes the target object and the first reference object, the edge computing device determines a first calibration region of the target object and a second calibration region of the first reference object according to the position information of the target object and the position information of the first reference object respectively, the edge computing device calculates a first coincidence degree of the first calibration region and the second calibration region, and when the first coincidence degree is greater than or equal to a first preset threshold, it is determined that the target behavior exists in the target image.

[0048] In the technical solution, the specific step of determining whether the target behavior exists in the target image according to the position information of the target object and the reference object is that, when the edge computing device identifies that the target image includes the target object and the first reference object, the edge computing device determines a first calibration region of the target object and a second calibration region of the first reference object according to the position information of the target object and the position information of the first reference object respectively, the edge computing device calculates a first coincidence degree of the first calibration region and the second calibration region, and when the first coincidence degree is greater than or equal to a first preset threshold, it is determined that the target behavior exists in the target image.

[0049] Specifically, the calculation of the coincidence degree between the first calibration region and the second calibration region can be achieved by calculating the IOU (Intersection Over Union) of the first calibration region and the second calibration region, or calculating the coverage index of the first calibration region.

[0050] In the technical solution, the calculation formula of the IOU of the first calibration region and the second calibration region is as follows.

[0051] Coincidence degree = (pipe rectangular frame ∩ screen surface rectangular frame) / (pipe rectangular frame ∪ screen surface rectangular frame);

[0052] Wherein, the pipe rectangular frame is the first calibration region, the screen surface rectangular frame is the second calibration region, and the coincidence degree is a value for indicating whether the target behavior exists in the target image.

[0053] In any of the above technical solutions, the calculation formula of the coverage index of the first calibration region is as follows.

[0054] Coincidence degree = (pipe rectangular frame ∩ screen surface rectangular frame) / pipe rectangular frame;

[0055] Wherein, the pipe rectangular frame is the first calibration region, the screen surface rectangular frame is the second calibration region, and the coincidence degree is a value for indicating whether the target behavior exists in the target image.

[0056] In any of the above technical solutions, the position relationship between the target objects is determined by the coincidence degree calculation, that is, whether the water pipe is above the screen surface is determined by the coincidence degree, so as to simplify the determination of the illegal water adding behavior, improve the efficiency and accuracy of the determination.

[0057] In any of the above technical solutions, the reference object includes a first reference object and a second reference object, and whether the target behavior exists in the target image is determined according to the position information of the target object and the reference object, specifically including: determining a first calibration region of the target object according to the position information of the target object, and determining a second calibration region of the first reference object and a third calibration region of the second reference object according to the position information of the first reference object and the second reference object; in a case where a first coincidence degree of the first calibration region and the second calibration region is greater than or equal to a first preset threshold, determining a second coincidence degree of the first calibration region and the third calibration region; in a case where the second coincidence degree of the third calibration region and the first calibration region is greater than or equal to a second preset threshold, determining that the target behavior exists in the target image.

[0058] In the above technical solution, the specific steps of the processing unit for determining whether the target behavior exists in the target image according to the position information of the target object and the reference object are that, when the edge computing device identifies that the target image includes the target object, the first reference object and the second reference object, the edge computing device determines a first calibration region of the target object, a second calibration region of the first reference object and a third calibration region of the second reference object according to the position information of the target object, the position information of the first reference object and the position information of the second reference object respectively, the edge computing device calculates a first coincidence degree of the first calibration region and the second calibration region, in a case where the first coincidence degree is greater than or equal to a first preset threshold, calculates a second coincidence degree of the first calibration region and the third calibration region, and in a case where the second coincidence degree is greater than or equal to a second preset threshold, determines that the target behavior exists in the target image.

[0059] In the technical solution, the position information of the target object, the position information of the first reference object and the position information of the second reference object in the target image are determined as follows: the edge computing device determines a target frame of the target object, the first reference object and the second reference object when the edge computing device identifies the target object, the first reference object and the second reference object in the target image, and the target frame is used to indicate the position information of the target object, the first reference object and the second reference object. The target object can be a tubular object for adding water, the first reference object can be a hopper screen surface related to the illegal water adding, and the second reference object can be a person related to the illegal water adding. The edge computing device further identifies the person on the basis of identifying the tubular object and the hopper screen surface in the first picture. Then, the edge computing device calculates a first calibration region of the tubular object and a third calibration region of the person on the basis of calculating a first calibration region of the tubular object and a second calibration region of the hopper screen surface. The edge computing device determines whether the illegal water adding behavior occurs by calculating the coincidence degree among the tubular object, the hopper screen surface and the person, so that the determination of the illegal water adding behavior is more reliable and has a higher confidence.

[0060] In any of the technical solutions, the target object and the reference object in the target image are identified, and the position information of the target object and the reference object is determined, specifically including: establishing a target detection model; inputting the target image into the target detection model to identify the target object in the target image; identifying the reference object and determining the position information of the target object and the reference object when the target object is identified.

[0061] In the technical solution, the position information of the target object, the position information of the first reference object and the position information of the second reference object in the target image are determined as follows: the edge computing device determines a target frame of the target object, the first reference object and the second reference object when the edge computing device identifies the target object, the first reference object and the second reference object in the target image, and the target frame is used to indicate the position information of the target object, the first reference object and the second reference object. The target object can be a tubular object for adding water, the first reference object can be a hopper screen surface related to the illegal water adding, and the second reference object can be a person related to the illegal water adding. The edge computing device further identifies the person on the basis of identifying the tubular object and the hopper screen surface in the first picture. Then, the edge computing device calculates a first calibration region of the tubular object and a third calibration region of the person on the basis of calculating a first calibration region of the tubular object and a second calibration region of the hopper screen surface. The edge computing device determines whether the illegal water adding behavior occurs by calculating the coincidence degree among the tubular object, the hopper screen surface and the person, so that the determination of the illegal water adding behavior is more reliable and has a higher confidence.

[0062] In the technical solution, the target detection model includes but is not limited to a YOLO (You Only Look Once) series, an SSD (Single Shot Multi Box Detector) series, and the like. When the target detection model is trained and inference is performed using the target detection model, the category can be set, and then the target detection model is trained and inference is performed using the set category. For example, the category can be set as a tubular object, a hopper screen surface, and a person, wherein the tubular object, the hopper screen surface, and the person correspond to a target object, a first reference object, and a second reference object, respectively. That is, the target detection model in the edge computing device is used to identify the tubular object, the hopper screen surface, and the person, determine a target frame of the tubular object, the hopper screen surface, and the person, and use the target frame to indicate the position information of the target object and the reference object. The classic algorithm of traditional computer vision is used to reduce the identification cost and effectively avoid or warn the illegal water adding situation, which has a high application value.

[0063] In any of the technical solutions, after determining that the target behavior exists in the target image, the method for determining the target behavior further includes: sending a first signal to a controller, the first signal being used to indicate that the target behavior currently exists, and the controller being used to send the target image in which the target behavior exists to a user; and / or issuing an alarm prompt information, the prompt information being used to prompt processing of the target behavior.

[0064] In the technical solution, after determining that the target behavior exists in the target image, the edge computing device sends a first signal to a controller and / or issues an alarm prompt information to prompt processing of the target behavior.

[0065] In the technical solution, the edge computing device transmits the first signal to the controller in a CAN (Controller Area Network) protocol, and the target image can be transmitted back to a database background through the controller, so that the user can timely obtain the target image of the illegal water adding behavior. The alarm prompt information can use a buzzer to alarm, and the pump truck can be equipped with a buzzer to timely alarm the illegal water adding behavior.

[0066] In any of the technical solutions, the method for determining the target behavior further includes: storing a plurality of target images in which the target behavior exists in a target database.

[0067] In the technical solution, the RGB camera continuously captures a plurality of pictures when capturing, and the edge computing device stores a plurality of first pictures in which the target behavior exists in the target database after identifying the target behavior in the pictures, which is convenient for later tracing and avoids disputes for the illegal water adding behavior in the construction process.

[0068] In any of the above technical solutions, before identifying the target object and the reference object in the target image and determining the position information of the target object and the reference object, the method for determining the target behavior further comprises: acquiring the target image; wherein the target image comprises the target object and / or the reference object.

[0069] In the above technical solution, before identifying the target image, the edge computing device acquires the target image, wherein the target image comprises the target object and / or the reference object.

[0070] Specifically, the target image is acquired by an RGB camera, and the acquired target image is sent to the edge computing device, wherein the target image acquired by the RGB camera can comprise the target object and / or the reference object that needs to be identified by the edge computing device, and the edge computing device acquires the target image and identifies the target object and the reference object in the target image.

[0071] In the above technical solution, when the target image comprises the target object and the reference object, the edge computing device needs to identify the target object and the reference object and further judge whether there is a target behavior in the target image; when the target image only comprises the target object or the reference object, the edge computing device identifies the target image and judges that there is no target behavior in the target image. In any of the above technical solutions, the target behavior specifically comprises the behavior of adding water to the hopper of the pumping machinery in violation of regulations using the target object.

[0072] In the above technical solution, the target behavior specifically comprises the behavior of adding water to the hopper of the pumping machinery in violation of regulations using the target object, wherein the behavior of adding water in violation of regulations can be used on the pumping machinery such as a pump truck and a vehicle-mounted pump. For example, when the behavior of adding water in violation of regulations is used on the pump truck, the screen surface above the hopper, the water pipe tubular object and the person are identified and detected, wherein the water pipe tubular object is the target object, and the screen surface above the hopper and the person are the reference objects, and when the overlapping area of the screen surface above the hopper, the water pipe tubular object and the person exceeds the threshold value set by the user, it is determined that there is the behavior of adding water to the pump truck in violation of regulations.

[0073] The third aspect of the present application provides a readable storage medium, and the readable storage medium stores programs or instructions, and the programs or instructions are executed by a processor to realize the steps of the method for determining the target behavior in any of the above technical solutions.

[0074] The readable storage medium provided by the present application stores programs or instructions, and the programs or instructions are executed by a processor to realize the steps of the method for determining the target behavior in any of the above technical solutions. Therefore, the readable storage medium proposed by the present application has all the beneficial effects of the method for determining the target behavior in any of the above technical solutions, which will not be described here.

[0075] The fourth aspect of the present application provides an electronic device, comprising the target behavior determination apparatus in any of the above technical solutions; and / or the readable storage medium in any of the above technical solutions.

[0076] The electronic device provided by the present application comprises the target behavior determination apparatus in any of the above technical solutions, thus having all the advantages of the target behavior determination apparatus in any of the above technical solutions, which will not be repeated here.

[0077] Further, the electronic device provided by the present application can further comprise the readable storage medium defined in any of the above technical solutions. Therefore, the electronic device provided by the present application has all the advantages of the readable storage medium defined in any of the above technical solutions, which will not be repeated here.

[0078] The fifth aspect of the present application provides a pumping machine, comprising: the target behavior determination apparatus in any of the above technical solutions; and / or the readable storage medium in any of the above technical solutions; and / or the electronic device in any of the above technical solutions.

[0079] In the above technical solution, the pumping machine can be a pump truck, a vehicle-mounted pump, a trailer pump, a wet sprayer, etc.

[0080] The pumping machine provided by the present application comprises the target behavior determination apparatus in any of the above technical solutions, thus having all the advantages of the target behavior determination apparatus in any of the above technical solutions, which will not be repeated here.

[0081] Further, the pumping machine provided by the present application can further comprise the readable storage medium defined in any of the above technical solutions. Therefore, the pumping machine provided by the present application has all the advantages of the readable storage medium defined in any of the above technical solutions, which will not be repeated here.

[0082] Further, the pumping machine provided by the present application further comprises the electronic device defined in any of the above technical solutions, thus the pumping machine provided by the present application has all the advantages of the electronic device defined in any of the above technical solutions, which will not be repeated here.

[0083] Additional aspects and advantages of the present application will become apparent from the following description with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0084] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0085] Figure 1 One of the flowcharts of the target behavior determination method of the embodiment of the present application is shown;

[0086] Figure 2 Fig. 2 shows a flowchart of a method for determining a target behavior according to an embodiment of the present application;

[0087] Figure 3 Fig. 3 shows a flowchart of a method for determining a target behavior according to an embodiment of the present application;

[0088] Figure 4 Fig. 4 shows a flowchart of a method for determining a target behavior according to an embodiment of the present application;

[0089] Figure 5 Fig. 5 shows a flowchart of a method for determining a target behavior according to an embodiment of the present application;

[0090] Figure 6 Fig. 6 shows a flowchart of a method for determining a target behavior according to an embodiment of the present application;

[0091] Figure 7 Fig. 7 shows a flowchart of a method for determining a target behavior according to an embodiment of the present application;

[0092] Figure 8 Fig. 8 shows a schematic block diagram of a device for determining a target behavior according to an embodiment of the present application;

[0093] Figure 9 Fig. 9 shows a schematic diagram of identifying a target object and a reference object according to an embodiment of the present application;

[0094] Correspondence between reference signs in Fig. 9 and object names is as follows: Figure 9 Correspondence between reference signs in Fig. 9 and object names is as follows:

[0095] 902 target object, 904 first reference object, 906 second reference object. DETAILED DESCRIPTION

[0096] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0097] The execution subject of the technical solution provided by the embodiments of the present application can be a computer device, and the subject can be determined according to actual use requirements, which is not limited by the embodiments of the present application. In order to more clearly describe the method for determining a target behavior provided by the embodiments of the present application, the execution subject of the method for determining a target behavior is an edge computing device in the following method embodiments.

[0098] Embodiment one

[0099] As Figure 1As shown, the embodiment of the first aspect of the application proposes a target behavior determination method, the target behavior determination method comprises:

[0100] Step 102: identifying the target object and the reference object in the target image, and determining the position information of the target object and the reference object;

[0101] Step 104: determining whether there is a target behavior in the target image according to the position information of the target object and the reference object.

[0102] In the embodiment of the application, first, the edge computing device acquires the target image, then the edge computing device identifies the target object and the reference object in the target image, and determines the position information of the target object and the reference object in the target image with the target object and the reference object, finally the edge computing device determines whether there is a target behavior in the target image according to the position information of the target object and the reference object.

[0103] In the above embodiment, the target behavior can be a rule-breaking water adding behavior of a pump truck, a vehicle-mounted pump, a drag pump and other pumping machines, and whether there is a rule-breaking water adding behavior of the pump truck in the target image is determined according to the position information of the target object and the reference object, wherein the target object can be an object for rule-breaking water adding or an object for rule-breaking oil stealing and other liquid conveying objects, such as a water inlet pipe-shaped object, a spray head or an oil pipe for oil pumping and other liquid conveying objects, and the reference object can include objects related to the rule-breaking water adding or rule-breaking oil stealing and other rule-breaking behaviors of the target object, such as a hopper screen surface, a person, a water pipe support, an oil tank or other objects related to the rule-breaking liquid conveying of the target object. Further, the edge computing device on the pump truck determines the position information of the object for water adding and the object related to the rule-breaking water adding in the target image, and the edge computing device determines whether there is a rule-breaking water adding behavior of the pump truck in the target image according to the relationship between the position information of the object for water adding and the object related to the rule-breaking water adding in the target image, so as to avoid rule-breaking water adding during the pump truck construction process, and further avoid the occurrence of segregation phenomenon caused by excessive water content of concrete, and ensure the safety during the construction process.

[0104] In the above embodiment, the target image is collected by an RGB (Red Green Blue, optical three primary colors) camera, and after the edge computing device acquires the target image, the edge computing device identifies the target object and the reference object in the target image, and determines the position information of the two objects, and further determines whether there is a target behavior in the target image according to the position information, wherein the identification of the rule-breaking water adding behavior is converted into the identification of the three targets of the person, the tubular object and the hopper screen surface, and the position of the three in the image space is used to improve the accuracy of identification.

[0105] Embodiment two

[0106] As Figure 2 shown in any of the above embodiments, the reference object includes a first reference object, and whether the target behavior exists in the target image is determined according to the position information of the target object and the reference object, specifically including:

[0107] Step 202: identifying the target object and the reference object in the target image, and determining the position information of the target object and the reference object;

[0108] Step 204: in the case that the reference object includes the first reference object, determining the first calibration region of the target object according to the position information of the target object, and determining the second calibration region of the first reference object according to the position information of the first reference object;

[0109] Step 206: determining that the target behavior exists in the target image based on the first coincidence degree between the first calibration region and the second calibration region being greater than or equal to the first preset threshold.

[0110] In the above embodiment, the specific steps of determining whether the target behavior exists in the target image according to the position information of the target object and the reference object are that, in the case that the edge computing device identifies that the target image includes the target object and the first reference object, the edge computing device determines the first calibration region of the target object and the second calibration region of the first reference object according to the position information of the target object and the position information of the first reference object respectively, the edge computing device calculates the first coincidence degree of the first calibration region and the second calibration region, and in the case that the first coincidence degree is greater than or equal to the first preset threshold, it is determined that the target behavior exists in the target image.

[0111] In the above embodiment, the determination of the position information of the target object and the position information of the first reference object in the target image is specifically that, in the case that the edge computing device identifies the target object and the first reference object in the target image, the edge computing device determines the target frame of the target object and the first reference object, and the target frame is used to indicate the position information of the target object and the first reference object. Then the edge computing device determines the first calibration region of the calibration frame of the target object and the second calibration region of the calibration frame of the first reference object, and determines the coincidence degree of the two calibration regions, so as to determine whether the target behavior exists in the target image.

[0112] Specifically, the calculation of the coincidence degree between the first calibration region and the second calibration region can be realized by calculating the IOU (Intersection Over Union) of the first calibration region and the second calibration region, or calculating the coverage index of the first calibration region.

[0113] In the above embodiment, the calculation formula of the IOU of the first calibration region and the second calibration region is as follows.

[0114] Degree of overlap = (tube rectangle ∩ screen rectangle) / (tube rectangle ∪ screen rectangle);

[0115] The tube rectangle is the first calibration area, the screen rectangle is the second calibration area, and the degree of overlap is a value used to indicate whether there is target behavior in the target image.

[0116] In any of the above embodiments, the formula for calculating the coverage index of the first calibration area is as follows.

[0117] overlap degree = (tube rectangle ∩ screen rectangle) / tube rectangle;

[0118] The tube rectangle is the first calibration area, the screen rectangle is the second calibration area, and the degree of overlap is a value used to indicate whether there is target behavior in the target image.

[0119] In any of the above embodiments, the degree of overlap is used to replace the determination of the positional relationship between the target objects. That is, the degree of overlap is used to determine whether the water pipe is above the screen surface, thereby simplifying the determination of illegal water addition and improving the efficiency and accuracy of the determination.

[0120] Example 3

[0121] like Figure 3 As shown, in any of the above embodiments, the reference objects include a first reference object and a second reference object. Determining whether a target behavior exists in the target image based on the position information of the target object and the reference objects specifically includes:

[0122] Step 302: Identify the target object and reference object in the target image, and determine the position information of the target object and reference object;

[0123] Step 304: When the reference objects include a first reference object and a second reference object, determine the first calibration area of ​​the target object based on the position information of the target object, and determine the second calibration area of ​​the first reference object and the third calibration area of ​​the second reference object based on the position information of the first reference object and the second reference object.

[0124] Step 306: If the first degree of overlap between the first calibration region and the second calibration region is greater than or equal to the first preset threshold, determine the second degree of overlap between the first calibration region and the third calibration region;

[0125] Step 308: If the second degree of overlap between the third calibration region and the first calibration region is greater than or equal to the second preset threshold, it is determined that there is target behavior in the target image.

[0126] In the above embodiment, the specific step of determining whether the target behavior exists in the target image according to the position information of the target object and the reference objects is that, in a case where the edge computing device identifies that the target image includes the target object, the first reference object and the second reference object, the edge computing device determines a first calibration region of the target object, a second calibration region of the first reference object and a third calibration region of the second reference object according to the position information of the target object, the position information of the first reference object and the position information of the second reference object respectively, calculates a first coincidence degree of the first calibration region and the second calibration region, in a case where the first coincidence degree is greater than or equal to a first preset threshold, the edge computing device calculates a second coincidence degree of the first calibration region and the third calibration region, and in a case where the second coincidence degree is greater than or equal to a second preset threshold, it is determined that the target behavior exists in the target image.

[0127] In the above embodiment, the determination of the position information of the target object, the position information of the first reference object and the position information of the second reference object in the target image is specifically that, in a case where the edge computing device identifies the target object, the first reference object and the second reference object in the target image, the edge computing device determines a target frame of the target object, the first reference object and the second reference object, and the target frame is used to indicate the position information of the target object, the first reference object and the second reference object. Wherein the target object can be a tubular object for adding water, the first reference object can be a hopper screen surface related to the illegal water adding, and the second reference object can be a person related to the illegal water adding. On the basis of identifying the tubular object and the hopper screen surface in the first picture, the edge computing device further identifies the person, and then on the basis of calculating the first calibration region of the tubular object and the second calibration region of the hopper screen surface, the first calibration region of the tubular object and the third calibration region of the person are calculated again. The edge computing device determines whether the illegal water adding behavior occurs by calculating the coincidence degree among the tubular object, the hopper screen surface and the person, so that the judgment of the illegal water adding behavior is more reliable and has higher confidence.

[0128] In any of the above embodiments, as shown in FIG. 9, Figure 9 Figure 9 The target object 902 in the target image is used to indicate the position information of the target object, i.e., the position information of the water pipe; Figure 9 The first reference object 904 in the target image is used to indicate the position information of the first reference object, i.e., the position information of the hopper screen surface; Figure 9 ​The second reference object 906 in the image is used to indicate the position information of the second reference object, i.e., the position information of the person; wherein, in this embodiment, the target object is the water pipe, the first reference object is the screen surface of the hopper, and the second reference object is the person. The calibration regions of the water pipe, the screen surface of the hopper and the person are determined according to the target object 902, the first reference object 904 and the second reference object 906 respectively, and then whether the illegal water adding behavior occurs is determined by calculating the region coincidence degree between the target object 902, the first reference object 904 and the second reference object 906.

[0129] Further, the identification of the illegal water adding behavior is converted into the identification of the three targets of the person, the water pipe and the screen surface, and the position of the three targets in the image space is used to improve the accuracy of the identification. The position relationship between the targets is determined by the coincidence degree calculation, i.e., whether the water pipe is above the screen surface, whether the water pipe is on the hand of the person, etc. The illegal water adding behavior in the construction process of the pump truck is avoided.

[0130] In any of the above embodiments, in addition to identifying the illegal water adding behavior in the construction process of the pump truck, the embodiments of the present application can also be used to identify other illegal behaviors such as stealing gasoline or diesel oil in the fuel tank of the vehicle and other behaviors of transporting liquid. For example, when identifying the behavior of stealing gasoline or diesel oil in the fuel tank of the vehicle, the target object is the oil pipe for extracting oil in the fuel tank, the first reference object is the fuel tank opening of the vehicle, and the second reference object is the person. The identification method is the same as the method of identifying the illegal water adding behavior in the construction process of the pump truck, and will not be described here.

[0131] Embodiment Four

[0132] As shown in the above any embodiment, the target object and the reference object in the target image are identified, and the position information of the target object and the reference object is determined, which specifically includes: Figure 4

[0133] Step 402: establishing a target detection model;

[0134] Step 404: inputting the target image into the target detection model to identify the target object in the target image;

[0135] Step 406: in the case of identifying the target object, identifying the reference object and determining the position information of the target object and the reference object;

[0136] Step 408: determining whether the target behavior exists in the target image according to the position information of the target object and the reference object.

[0137] ​In the above embodiments, the specific steps of identifying the target object and the reference object in the target image and determining the position information of the target object and the reference object are that a target detection model is established in the edge computing device, the edge computing device inputs the acquired target image into the established target detection model, the target detection model starts to identify the target image, and in a case where the target object is identified in the target image, the position information of the target object and the reference object is determined.

[0138] In the above embodiments, the target detection model includes but is not limited to a YOLO (You Only Look Once) series, an SSD (Single Shot Multi Box Detector) series, and the like. When the target detection model is trained and inference is performed using the target detection model, the category can be set, and then the training and inference of the target detection model are performed using the set category. For example, the category can be set as a tubular object, a hopper screen surface, and a person, wherein the tubular object, the hopper screen surface, and the person correspond to the target object, the first reference object, and the second reference object, respectively. That is, the tubular object, the hopper screen surface, and the person are identified using the target detection model in the edge computing device, the target boxes of the tubular object, the hopper screen surface, and the person are determined, and the position information of the target object and the reference object is indicated using the target boxes. Through the classic algorithm of traditional computer vision, the recognition cost is reduced, and the situation of illegal water addition can be effectively avoided or warned, which has extremely high application value.

[0139] Embodiment Five

[0140] As shown in the above any embodiment, after the target behavior is determined to exist in the target image, the method for determining the target behavior further includes: Figure 5

[0141] Step 502: establishing a target detection model;

[0142] Step 504: inputting the target image into the target detection model to identify the target object in the target image;

[0143] Step 506: in a case where the target object is identified, identifying the reference object and determining the position information of the target object and the reference object;

[0144] Step 508: in a case where the reference object includes the first reference object, determining a first calibration region of the target object according to the position information of the target object, and determining a second calibration region of the first reference object according to the position information of the first reference object;

[0145] Step 510: determining that the target behavior exists in the target image based on the first calibration region and the second calibration region having a first coincidence degree greater than or equal to a first preset threshold; ​

[0146] Step 512: In the case that the reference objects include a first reference object and a second reference object, determining a first calibration region of the target object according to the position information of the target object, and determining a second calibration region of the first reference object and a third calibration region of the second reference object according to the position information of the first reference object and the second reference object;

[0147] Step 514: In the case that the first coincidence degree of the first calibration region and the second calibration region is greater than or equal to a first preset threshold, determining a second coincidence degree of the first calibration region and the third calibration region.

[0148] Step 516: In the case that the second coincidence degree of the third calibration region and the first calibration region is greater than or equal to a second preset threshold, determining that the target behavior exists in the target image.

[0149] Step 518: Sending a first signal to the controller, the first signal being used to indicate that the target behavior currently exists; and / or issuing an alarm prompt information, the prompt information being used to prompt processing of the target behavior.

[0150] In the above embodiments, after determining that the target behavior exists in the target image, the edge computing device sends a first signal to the controller, and / or issues an alarm prompt information to prompt processing of the target behavior.

[0151] In the above embodiments, the edge computing device transmits the first signal to the controller in the form of a CAN (Controller Area Network) protocol, and can transmit the target image back to the database background through the controller, so that the user can obtain the target image of the illegal water adding behavior in a timely manner; the alarm prompt information can use a buzzer to alarm, and the pump truck can be equipped with a buzzer to alarm the illegal water adding behavior in a timely manner.

[0152] Embodiment Six

[0153] As shown in any of the above embodiments, the method for determining the target behavior further comprises: Figure 6

[0154] Step 602: Establishing a target detection model;

[0155] Step 604: Inputting the target image into the target detection model to identify the target object in the target image;

[0156] Step 606: In the case that the target object is identified, identifying the reference object and determining the position information of the target object and the reference object.

[0157] ​Step 608: In the case that the reference object includes the first reference object, a first calibration region of the target object is determined according to the position information of the target object, and a second calibration region of the first reference object is determined according to the position information of the first reference object.

[0158] Step 610: In the case that the first coincidence degree of the first calibration region and the second calibration region is greater than or equal to the first preset threshold, it is determined that the target behavior exists in the target image.

[0159] Step 612: In the case that the reference object includes the first reference object and the second reference object, a first calibration region of the target object is determined according to the position information of the target object, and a second calibration region of the first reference object and a third calibration region of the second reference object are determined according to the position information of the first reference object and the second reference object.

[0160] Step 614: In the case that the first coincidence degree of the first calibration region and the second calibration region is greater than or equal to the first preset threshold, a second coincidence degree of the first calibration region and the third calibration region is determined.

[0161] Step 616: In the case that the second coincidence degree of the third calibration region and the first calibration region is greater than or equal to the second preset threshold, it is determined that the target behavior exists in the target image.

[0162] Step 618: The target images in which the target behavior exists are stored into the target database.

[0163] In the above embodiment, the RGB camera continuously captures multiple pictures when capturing, and the edge computing device stores the multiple first pictures into the target database after identifying that the target behavior exists in the pictures, which is convenient for later tracing and avoids disputes for the illegal water adding behavior in the construction process.

[0164] In any of the above embodiments, the target behavior specifically includes the behavior of illegally adding water to the hopper of the pumping machinery by using the target object.

[0165] In the above embodiment, the target behavior specifically includes the behavior of illegally adding water to the hopper of the pumping machinery by using the target object, and the illegal water adding behavior can be used on the pump truck, the vehicle-mounted pump and other pumping machinery. Illustratively, when the illegal water adding behavior is used on the pump truck, the screen surface above the hopper, the water pipe tubular object and the person are identified and detected, wherein the water pipe tubular object is the target object, the screen surface above the hopper and the person are the reference objects, and when the overlapping area of the screen surface above the hopper, the water pipe tubular object and the person exceeds the threshold value set by the user, it is determined that the illegal water adding behavior to the pump truck exists.

[0166] Embodiment Seven

[0167] As Figure 7As shown, in any of the above embodiments, before identifying the target object and reference object in the target image and determining the position information of the target object and reference object, the method for determining the target behavior further includes:

[0168] Step 702: Acquire the target image, wherein the target image includes the target object and / or the reference object;

[0169] Step 704: Identify the target object and reference object in the target image, and determine the position information of the target object and reference object;

[0170] Step 706: Determine whether there is target behavior in the target image based on the position information of the target object and the reference object.

[0171] In the above embodiments, before recognizing the target image, the edge computing device acquires the target image, wherein the target image includes the target object and / or a reference object.

[0172] Specifically, a target image is acquired by an RGB camera and sent to an edge computing device. The target image acquired by the RGB camera may include the target object and / or reference object that the edge computing device needs to identify. The edge computing device acquires the target image and identifies the target object and reference object in the target image.

[0173] In the above embodiments, when the target image includes a target object and a reference object, the edge computing device needs to identify the target object and the reference object, and further determine whether there is a target behavior in the target image; when the target image only has a reference object or a target object, the edge computing device identifies the target image and determines that there is no target behavior in the target image.

[0174] Example 8

[0175] like Figure 8 As shown, a second aspect embodiment of the present invention provides a target behavior determination device 800, which includes: an identification unit 802, used to identify a target object and a reference object in a target image and determine the position information of the target object and the reference object; and a processing unit 804, used to determine whether a target behavior exists in the target image based on the position information of the target object and the reference object.

[0176] In an embodiment of the present invention, a target image is first acquired by an edge computing device. Then, a recognition unit identifies the target object and reference object within the target image and determines the position information of the target object and reference object in the target image. Finally, a processing unit determines whether a target behavior exists in the target image based on the position information of the target object and reference object.

[0177] In the above embodiment, the target behavior can be a behavior of adding water to the pump truck in violation of regulations, and whether the behavior of adding water to the pump truck in violation of regulations exists in the target image is determined according to position information of the target object and the reference object, wherein the target object can be an object for adding water in violation of regulations, such as a water inlet pipe, a spray head or other objects capable of delivering water, and the reference object can include an object related to the target object for adding water in violation of regulations, such as a hopper screen surface, a person, a water pipe support or other objects related to the target object for adding water in violation of regulations. Further, the edge computing device on the pump truck determines position information of the object for adding water and the object related to the behavior of adding water in violation of regulations in the target image, and the edge computing device determines whether the behavior of adding water to the pump truck in violation of regulations exists in the target image according to a relationship between the position information of the object for adding water and the object related to the behavior of adding water in violation of regulations in the target image, so as to avoid adding water in violation of regulations during the construction of the pump truck, and further avoid the occurrence of segregation phenomenon caused by excessive water content of the concrete, and ensure the safety during the construction.

[0178] In the above embodiment, the target image is collected by an RGB (Red Green Blue, optical three primary colors) camera, and after the edge computing device obtains the target image, the edge computing device identifies the target object and the reference object in the target image, and determines position information of the two objects, and further determines whether the target behavior exists in the target image according to the position information, wherein the identification of the behavior of adding water in violation of regulations is converted into the identification of the three targets of the person, the tubular object and the hopper screen surface, and the position of the three targets in the image space is used to improve the accuracy of the identification.

[0179] Embodiment Nine

[0180] In any of the above embodiments, the reference object includes a first reference object, and whether the target behavior exists in the target image is determined according to the position information of the target object and the reference object, specifically including: determining a first calibration region of the target object according to the position information of the target object, and determining a second calibration region of the first reference object according to the position information of the first reference object; and determining that the target behavior exists in the target image based on the first coincidence degree between the first calibration region and the second calibration region being greater than or equal to a first preset threshold.

[0181] In the above embodiments, the specific step of determining, by the processing unit, whether the target behavior exists in the target image according to the position information of the target object and the reference object is that, in a case where the edge computing device identifies that the target image includes the target object and the first reference object, the edge computing device determines a first calibration region of the target object and a second calibration region of the first reference object according to the position information of the target object and the position information of the first reference object respectively, the edge computing device calculates a first coincidence degree of the first calibration region and the second calibration region, and in a case where the first coincidence degree is greater than or equal to a first preset threshold, it is determined that the target behavior exists in the target image.

[0182] In the above embodiments, the determination of the position information of the target object and the position information of the first reference object in the target image is specifically that, in a case where the edge computing device identifies the target object and the first reference object in the target image, the edge computing device determines a target frame of the target object and the first reference object, and the target frame is used to indicate the position information of the target object and the first reference object. Then the edge computing device determines a first calibration region of a calibration frame of the target object and a second calibration region of a calibration frame of the first reference object, and determines the coincidence degree of the two calibration regions, so as to determine whether the target behavior exists in the target image.

[0183] Specifically, the calculation of the coincidence degree between the first calibration region and the second calibration region can be achieved by calculating the IOU (Intersection Over Union) of the first calibration region and the second calibration region, or calculating the coverage index of the first calibration region.

[0184] In the above embodiments, the calculation formula of the IOU of the first calibration region and the second calibration region is as follows.

[0185] The coincidence degree = (the tube rectangular frame ∩ the screen surface rectangular frame) / (the tube rectangular frame ∪ the screen surface rectangular frame);

[0186] Wherein, the tube rectangular frame is the first calibration region, the screen surface rectangular frame is the second calibration region, and the coincidence degree is a value used to indicate whether the target behavior exists in the target image.

[0187] In any of the above embodiments, the calculation formula of the coverage index of the first calibration region is as follows.

[0188] The coincidence degree = (the tube rectangular frame ∩ the screen surface rectangular frame) / the tube rectangular frame;

[0189] Wherein, the tube rectangular frame is the first calibration region, the screen surface rectangular frame is the second calibration region, and the coincidence degree is a value used to indicate whether the target behavior exists in the target image.

[0190] In any of the above embodiments, the position relationship between the target object and the reference object is determined by the coincidence degree calculation, i.e., whether the water pipe is above the screen surface is determined by the coincidence degree, so as to simplify the determination of the illegal water adding behavior, and improve the efficiency and accuracy of the determination.

[0191] Embodiment ten

[0192] In any of the above embodiments, the reference object includes a first reference object and a second reference object, and whether the target behavior exists in the target image is determined according to the position information of the target object and the reference object, specifically including: determining a first calibration region of the target object according to the position information of the target object, and determining a second calibration region of the first reference object and a third calibration region of the second reference object according to the position information of the first reference object and the second reference object; in a case where a first coincidence degree of the first calibration region and the second calibration region is greater than or equal to a first preset threshold, determining a second coincidence degree of the first calibration region and the third calibration region; and in a case where the second coincidence degree of the third calibration region and the first calibration region is greater than or equal to a second preset threshold, determining that the target behavior exists in the target image.

[0193] In the above embodiment, the specific steps of the processing unit for determining whether the target behavior exists in the target image according to the position information of the target object and the reference object are that, in a case where the edge computing device identifies that the target image includes the target object, the first reference object and the second reference object, the edge computing device determines a first calibration region of the target object, a second calibration region of the first reference object and a third calibration region of the second reference object according to the position information of the target object, the position information of the first reference object and the position information of the second reference object, respectively, calculates a first coincidence degree of the first calibration region and the second calibration region, in a case where the first coincidence degree is greater than or equal to a first preset threshold, calculates a second coincidence degree of the first calibration region and the third calibration region, and in a case where the second coincidence degree is greater than or equal to a second preset threshold, determines that the target behavior exists in the target image.

[0194] In the above embodiment, the position information of the target object, the position information of the first reference object and the position information of the second reference object in the target image are specifically determined as follows: the edge computing device determines the target frame of the target object, the first reference object and the second reference object in the case of identifying the target object, the first reference object and the second reference object in the target image, and the target frame is used to indicate the position information of the target object, the first reference object and the second reference object. The target object can be a tubular object for adding water, the first reference object can be a hopper screen surface related to the illegal water adding, and the second reference object can be a person related to the illegal water adding. On the basis of identifying the tubular object and the hopper screen surface in the first picture, the edge computing device further identifies the person, and then calculates the first calibration area of the tubular object and the third calibration area of the person on the basis of calculating the first calibration area of the tubular object and the second calibration area of the hopper screen surface. The edge computing device determines whether the illegal water adding behavior occurs by calculating the coincidence degree among the tubular object, the hopper screen surface and the person, so that the judgment of the illegal water adding behavior is more reliable and has higher confidence.

[0195] Embodiment eleven

[0196] In any of the above embodiments, the target object and the reference object in the target image are identified, and the position information of the target object and the reference object is determined, specifically including: establishing a target detection model; inputting the target image into the target detection model to identify the target object in the target image; identifying the reference object and determining the position information of the target object and the reference object in the case of identifying the target object.

[0197] In the above embodiment, the specific steps of identifying the target object and the reference object in the target image and determining the position information of the target object and the reference object are as follows: a target detection model is established in the edge computing device, the edge computing device inputs the acquired target image into the established target detection model, the target detection model starts to identify the target image, and the position information of the target object and the reference object is determined in the case of identifying the target object in the target image.

[0198] In the above embodiments, the target detection model includes but is not limited to YOLO (You Only Look Once, target detection) series, SSD (Single Shot Multi Box Detector, multi-frame prediction) series, etc. When training the target detection model and using the target detection model for inference, the category can be set, and then the set category is used for training and inference of the target detection model, for example, the category can be set as a tubular object, a hopper screen surface and a person, wherein the tubular object, the hopper screen surface and the person correspond to a target object, a first reference object and a second reference object respectively, that is, the target detection model in the edge computing device is used to identify the tubular object, the hopper screen surface and the person, determine the target frame of the tubular object, the hopper screen surface and the person, use the target frame to indicate the position information of the target object and the reference object, and use the classic algorithm of traditional computer vision to reduce the identification cost and effectively avoid or warn the illegal water adding situation, which has very high application value.

[0199] Embodiment twelve

[0200] In any of the above embodiments, after determining that the target image contains the target behavior, the method for determining the target behavior further comprises: sending a first signal to the controller, the first signal being used to indicate that the target behavior currently exists, and the controller being used to send the target image containing the target behavior to the user; and / or issuing an alarm prompt information, the prompt information being used to prompt the processing of the target behavior.

[0201] In the above embodiments, after determining that the target image contains the target behavior, the edge computing device sends a first signal to the controller and / or issues an alarm prompt information to prompt the processing of the target behavior.

[0202] In the above embodiments, the edge computing device transmits the first signal to the controller in the form of CAN (Controller Area Network, controller area network) protocol transmission, and the target image can be transmitted back to the database background through the controller, so that the user can timely obtain the target image of the illegal water adding behavior; the alarm prompt information can use a buzzer to alarm, and the pump truck can be equipped with a buzzer to timely alarm the illegal water adding behavior.

[0203] In any of the above embodiments, the method for determining the target behavior further comprises: storing a plurality of target images containing the target behavior into a target database.

[0204] In the above embodiments, the RGB camera continuously takes a plurality of pictures when taking pictures, and the edge computing device stores a plurality of first pictures containing the target behavior into the target database after identifying the target behavior in the pictures, which is convenient for later tracing and avoids disputes for the illegal water adding behavior in the construction process.

[0205] In any of the above embodiments, the target behavior specifically includes the behavior of using the target object to add water to the hopper of the pumping machine in violation of regulations.

[0206] In the above embodiments, the target behavior specifically includes the behavior of using the target object to add water to the hopper of the pumping machine in violation of regulations, wherein the behavior of adding water in violation of regulations can be used on a pump truck, a vehicle-mounted pump, or the like. Illustratively, when the behavior of adding water in violation of regulations is used on a pump truck, a screen surface above a hopper of the pump truck, a water pipe tubular object, and a person are identified and detected, wherein the water pipe tubular object is the target object, and the screen surface above the hopper and the person are reference objects. When an overlapping area of the screen surface above the hopper, the water pipe tubular object, and the person exceeds a threshold value set by a user, it is determined that the behavior of adding water in violation of regulations to the pump truck exists.

[0207] Embodiment Thirteen

[0208] In any of the above embodiments, before identifying the target object and the reference object in the target image and determining the position information of the target object and the reference object, the method for determining the target behavior further includes: obtaining the target image; wherein the target image includes the target object and / or the reference object.

[0209] In the above embodiments, before identifying the target image, the edge computing device obtains the target image, wherein the target image includes the target object and / or the reference object.

[0210] Specifically, the target image is collected by an RGB camera, and the collected target image is sent to the edge computing device, wherein the target image collected by the RGB camera can include the target object and / or the reference object that needs to be identified by the edge computing device. The edge computing device obtains the target image and identifies the target object and the reference object in the target image.

[0211] In the above embodiments, when the target object and the reference object are included in the target image, the edge computing device needs to identify the target object and the reference object and further judge whether there is a target behavior in the target image; when only the target object or the reference object is included in the target image, the edge computing device identifies the target image and judges that there is no target behavior in the target image.

[0212] A third aspect of the present application provides a readable storage medium, and the readable storage medium stores programs or instructions. When the programs or instructions are executed by a processor, the steps of the method for determining the target behavior in any of the above technical solutions are implemented.

[0213] The readable storage medium provided by the present application has all the beneficial effects of the target behavior determination method in any of the above technical solutions, and thus details are not repeated here.

[0214] Embodiment fifteen

[0215] The fourth aspect embodiment of the present application provides an electronic device, comprising: the target behavior determination device in any of the above technical solutions; and / or the readable storage medium in the above technical solution.

[0216] The electronic device provided by the present application comprises the target behavior determination device in any of the above technical solutions, and thus has all the beneficial effects of the target behavior determination device in any of the above technical solutions, and thus details are not repeated here.

[0217] Further, the electronic device provided by the present application can further comprise the readable storage medium defined in the above technical solution. Therefore, the electronic device provided by the present application has all the beneficial effects of the readable storage medium defined in the above technical solution, and thus details are not repeated here.

[0218] Embodiment sixteen

[0219] The fifth aspect embodiment of the present application provides a pumping machine, comprising: the target behavior determination device in any of the above technical solutions; and / or the readable storage medium in any of the above technical solutions; and / or the electronic device in the above technical solution.

[0220] In the above embodiment, the pumping machine can be a pump truck, a vehicle-mounted pump, a trailer pump, a wet sprayer, etc.

[0221] The pumping machine provided by the present application comprises the target behavior determination device in any of the above technical solutions, and thus has all the beneficial effects of the target behavior determination device in any of the above technical solutions, and thus details are not repeated here.

[0222] Further, the pumping machine provided by the present application can further comprise the readable storage medium defined in any of the above technical solutions. Therefore, the pumping machine provided by the present application has all the beneficial effects of the readable storage medium defined in the above technical solution, and thus details are not repeated here.

[0223] Further, the pumping machine provided by the present application further comprises the electronic device defined in the above technical solution, and thus the pumping machine provided by the present application has all the beneficial effects of the electronic device defined in the above technical solution, and thus details are not repeated here.

[0224] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "certain embodiments", and the like is intended to indicate that the described implementation, feature, structure, material or characteristic is included in at least one embodiment or example of the application. The illustrative representations of the above terms in the specification are not necessarily referring to the same embodiment or example. Moreover, the described implementation, feature, structure, material or characteristic can be combined in any one or more embodiments or examples in a suitable manner.

[0225] The above only is the preferred embodiment of the present application, and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of determining a target behavior, characterized by, The method for determining the target behavior comprises: identifying a target object and a reference object in a target image, and determining position information of the target object and the reference object; determining whether the target behavior exists in the target image according to the position information of the target object and the reference object; the reference object comprises a first reference object and a second reference object, and the determination of whether the target behavior exists in the target image according to the position information of the target object and the reference object specifically comprises: determining a first calibration region of the target object according to the position information of the target object, and determining a second calibration region of the first reference object and a third calibration region of the second reference object according to the position information of the first reference object and the second reference object; in the case where a first coincidence degree of the first calibration region and the second calibration region is greater than or equal to a first preset threshold, determining a second coincidence degree of the first calibration region and the third calibration region; in the case where the second coincidence degree of the third calibration region and the first calibration region is greater than or equal to a second preset threshold, determining that the target behavior exists in the target image; the target behavior specifically comprises: an action of using the target object to add water to a hopper of a pumping machine in violation of regulations; wherein the tubular object, the hopper screen surface and the person correspond to the target object, the first reference object and the second reference object respectively.

2. The method of claim 1, wherein, The identification of the target object and the reference object in the target image and the determination of the position information of the target object and the reference object specifically comprise: establishing a target detection model; inputting the target image into the target detection model to identify the target object in the target image; in the case where the target object is identified, identifying the reference object and determining the position information of the target object and the reference object.

3. The method of determining a target behavior according to claim 1 or 2, characterized in that, After determining that the target behavior exists in the target image, the method for determining the target behavior further comprises: sending a first signal to a controller, the first signal being used to indicate that the target behavior currently exists, and the controller being used to send the target image in which the target behavior exists to a user; and / or sending an alarm prompt information, the prompt information being used to prompt the processing of the target behavior.

4. The method of claim 1 or 2, wherein The method for determining the target behavior further comprises: storing multiple target images in which the target behavior exists into a target database.

5. The method of claim 1 or 2, wherein Before the identification of the target object and the reference object in the target image and the determination of the position information of the target object and the reference object, the method for determining the target behavior further comprises: obtaining the target image; wherein the target image comprises the target object and / or the reference object.

6. A determination device of a target behavior, characterized by, The device for determining the target behavior comprises: an identification unit configured to identify a target object and a reference object in a target image, and determine position information of the target object and the reference object; a processing unit configured to determine whether the target behavior exists in the target image according to the position information of the target object and the reference object. The reference objects include a first reference object and a second reference object, and the processing unit is specifically configured to determine a first calibration region of the target object according to position information of the target object, and determine a second calibration region of the first reference object and a third calibration region of the second reference object according to position information of the first reference object and the second reference object; in a case where a first coincidence degree of the first calibration region and the second calibration region is greater than or equal to a first preset threshold, determine a second coincidence degree of the first calibration region and the third calibration region; in a case where the second coincidence degree of the third calibration region and the first calibration region is greater than or equal to a second preset threshold, determine that the target image contains the target behavior. The target behavior specifically includes an action of illegally adding water to a hopper of a pumping machine by using the target object. The tubular object, the hopper screen surface and the person correspond to the target object, the first reference object and the second reference object respectively.

7. A readable storage medium, on which a program or instructions are stored, characterized in that, The program or the instruction is executed by the processor to implement the steps of the target behavior determination method according to any one of claims 1 to 5.

8. An electronic device, comprising: The program or the instruction is executed by the processor to implement the steps of the target behavior determination method according to any one of claims 1 to 5. The target behavior determination apparatus according to claim 6; And / or The readable storage medium according to claim 7.

9. A pumping machine characterized by, The target behavior determination apparatus according to claim 6; And / or The readable storage medium according to claim 7; and / or The electronic device according to claim 8. ​

Images