Behavior detection method and device, readable storage medium and program product

Abstract

The invention discloses a behavior detection method and device, a readable storage medium and a program product. The method comprises the following steps: acquiring a detection image of a detection area; inputting the detection image into a target detection model to obtain an initial handrail pose detection result output by the target detection model, the position information of the operator rectangular frame and the position information of the escalator rectangular frame; judging whether the initial handrail pose detection result represents that the operator does not hold the handrail of the escalator or not; under the condition that the initial handrail pose detection result represents that the operator does not hold the handrail of the escalator, the overlapping area of the operator rectangular frame and the escalator rectangular frame is determined based on the position information of the operator rectangular frame and the position information of the escalator rectangular frame; according to the method, the target ratio is obtained based on the ratio of the area of the overlapping area to the area of the rectangular frame of the operator, the target handrail pose detection result of the operator is determined based on the target ratio, and whether the operator holds the handrail of the escalator or not when the operator is on the escalator can be accurately and efficiently detected.

Description

Technical Field

[0001] The present application belongs to the technical field of intelligent production safety monitoring, and in particular, relates to a behavior detection method, device, readable storage medium and program product. Background Art

[0002] In industrial scenarios, if workers do not hold the handrails of the escalator while on it, it will not only cause safety problems, but may also affect the work efficiency and safety management of the entire work site. Therefore, ensuring that workers hold the handrails of the escalator is a key measure to ensure work safety, avoid accidents, and improve safety management efficiency.

[0003] At present, based on the deep learning model, the detection of whether the workers are holding the handrails on the escalator has problems such as low accuracy, poor real-time performance and high consumption of hardware resources. Summary of the invention

[0004] The embodiments of the present application provide a behavior detection method, device, readable storage medium, and program product, which can accurately and efficiently detect whether a worker is holding the handrail of an escalator while on the escalator.

[0005] In a first aspect, an embodiment of the present application provides a behavior detection method, the method comprising: acquiring a detection image of a detection area, wherein there are workers and escalators in the detection area; inputting the detection image into a target detection model to obtain an initial handrail posture detection result, position information of a worker rectangular frame, and position information of an escalator rectangular frame output by the target detection model; judging whether the initial handrail posture detection result represents that the worker does not hold the handrail of the escalator; in the case where the initial handrail posture detection result represents that the worker does not hold the handrail of the escalator, based on the position information of the worker rectangular frame and the position information of the escalator rectangular frame, determining an overlapping area between the worker rectangular frame and the escalator rectangular frame; obtaining a target ratio based on the ratio of the area of ​​the overlapping area to the area of ​​the worker rectangular frame, and determining a target handrail posture detection result of the worker based on the target ratio.

[0006] In an optional embodiment of the first aspect, the process of training a target detection model includes: performing local exposure processing on multiple original images to obtain multiple original images after local exposure processing; performing gamma correction processing on the multiple original images after local exposure processing to obtain multiple enhanced processed images; based on the multiple original images and the multiple enhanced processed images, training the target detection model to obtain a trained target detection model.

[0007] In an optional embodiment of the first aspect, local exposure processing is performed on multiple original images to obtain multiple original images after local exposure processing, including: generating a target circle in each original image; determining the distance between each pixel point in each original image and the center of the target circle to obtain multiple target distances; determining the exposure pixel value of each pixel point based on the size relationship between the target distance of each pixel point and the radius of the target circle; adjusting the pixel value of each pixel point to the exposure pixel value to obtain each original image after local exposure processing.

[0008] In an optional embodiment of the first aspect, based on the size relationship between the target distance of each pixel and the radius of the target circle, determining the exposure pixel value of each pixel, including: when the target distance of the pixel is less than the radius of the target circle, determining the exposure pixel value of the pixel as the pixel upper limit value; when the target distance of the pixel is greater than or equal to the radius of the target circle, determining the exposure pixel value of the pixel based on a predetermined adjustment strategy, wherein the predetermined adjustment strategy is: as the target distance increases, the exposure pixel value gradually decreases.

[0009] In an optional embodiment of the first aspect, an exposure pixel value of a pixel point is determined based on a predetermined adjustment strategy, including: determining an initial exposure correction parameter based on a target distance and a radius of a target circle; determining a maximum value between a preset correction parameter value and the initial exposure correction parameter as a target exposure correction parameter; determining an incremental intensity value based on a product of the target exposure correction parameter and a preset incremental intensity parameter; determining an initial exposure pixel value based on a sum of a pixel value of the pixel point and the incremental intensity value; and determining a minimum value between a pixel upper limit value and the initial exposure pixel value as the exposure pixel value of the pixel point.

[0010] In an optional embodiment of the first aspect, based on the position information of the operator rectangular frame and the position information of the escalator rectangular frame, the overlapping area between the operator rectangular frame and the escalator rectangular frame is determined, including: determining a cropping parameter based on the height value of the escalator rectangular frame; based on the cropping parameter, cropping the bottom area and the top area of ​​the escalator rectangular frame respectively to obtain a cropped escalator rectangular frame; and determining the overlapping area based on the position information of the cropped escalator rectangular frame and the position information of the operator rectangular frame.

[0011] In an optional embodiment of the first aspect, based on the target ratio, the target handrail posture detection result of the operator is determined, including: when the target ratio is greater than the preset ratio, the target handrail posture detection result of the operator is determined to be: the operator is not holding the handrail of the escalator.

[0012] In the second aspect, an embodiment of the present application provides a behavior detection device, which includes: an acquisition module, which is used to acquire a detection image of a detection area, wherein there are workers and escalators in the detection area; an acquisition module, which is used to input the detection image into a target detection model to obtain an initial handrail posture detection result, position information of the worker's rectangular frame, and position information of the escalator rectangular frame output by the target detection model; a judgment module, which is used to judge whether the initial handrail posture detection result represents that the worker does not hold the handrail of the escalator; a first determination module, which is used to determine the overlapping area between the worker's rectangular frame and the escalator rectangular frame based on the position information of the worker's rectangular frame and the position information of the escalator rectangular frame when the initial handrail posture detection result represents that the worker does not hold the handrail of the escalator; a second determination module, which is used to obtain a target ratio based on the ratio of the area of ​​the overlapping area to the area of ​​the worker's rectangular frame, and determine the target handrail posture detection result of the worker based on the target ratio.

[0013] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the behavior detection method of the first aspect is implemented.

[0014] In a fourth aspect, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the behavior detection method of the first aspect.

[0015] In the embodiment of the present application, since there are workers and escalators in the detection area, the image of the detection area is collected to obtain a detection image containing the workers and the escalator. The collected detection image is input into the target detection model, and the target detection model can detect whether the workers in the detection image are holding the handrail of the escalator, so as to obtain the initial handrail posture detection result, the position information of the worker's rectangular frame and the position of the escalator rectangular frame. Using the target detection model, the input detection image is detected, so that the initial handrail posture detection result can be obtained quickly and efficiently. When the initial handrail posture detection result indicates that the worker did not hold the handrail of the escalator, based on the position information of the worker's rectangular frame and the position information of the escalator's rectangular frame, the overlapping area between the worker's rectangular frame and the escalator's rectangular frame can be determined, and the target ratio can be obtained based on the area of ​​the overlapping area and the area of ​​the worker's rectangular frame. Based on the target ratio, it can be determined again whether the worker held the handrail of the escalator. This makes the target handrail posture detection result more accurate. In addition, when the initial handrail posture detection result indicates that the worker did not hold the handrail of the escalator, re-inspection can be performed again, which can not only reduce the amount of calculation and the consumption of hardware resources, but also ensure the real-time performance of this solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 It is a flowchart of a behavior detection system provided by an embodiment of the present application; Figure 2 is a flow chart of a behavior detection method provided by an embodiment of the present application; Figure 3 is a schematic diagram of the structure of an IAT network provided by an embodiment of the present application; Figure 4 It is a schematic diagram of a worker holding an escalator handrail provided by an embodiment of the present application; Figure 5 is a schematic diagram of a worker holding an escalator handrail provided by another embodiment of the present application; Figure 6 It is a schematic diagram of an operator not holding the handrail of an escalator provided by an embodiment of the present application; Figure 7 It is a flowchart of a behavior detection method in a well logging operation scenario provided by an embodiment of the present application; Figure 8 is a structural schematic diagram of a behavior detection device provided by another embodiment of the present application; Fig. 9 It is a structural schematic diagram of an electronic device provided in yet another embodiment of the present application.

[0018] The above drawings include the following reference numerals: The above drawings include the following reference numerals: 100, image acquisition device; 200, electronic device; 800, behavior detection device; 801, acquisition module; 802, acquisition module; 803, judgment module; 804, first determination module; 805, second determination module; 901, processor; 902, memory; 903, communication interface; 910, bus. DETAILED DESCRIPTION

[0019] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0020] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0021] In industrial scenarios, if workers do not hold the handrails of the escalator while on it, it will not only cause safety problems, but may also affect the efficiency and safety management of the entire work site. Therefore, ensuring that workers hold the handrails of the escalator while on it is a key measure to ensure work safety, avoid accidents, and improve safety management efficiency.

[0022] For example, in a well logging operation scenario, if the operator does not hold the handrail of the escalator when going up and down the well platform, the probability of a safety accident will increase.

[0023] Currently, the following methods can be used to ensure that workers hold the handrails of the escalator when on the escalator.

[0024] Based on the manual method, whether the operator holds the handrail of the escalator when on the escalator is detected. This method not only consumes a lot of human resources, but is also prone to missed detection or false detection due to human negligence or fatigue.

[0025] Based on the deep learning model, it detects whether the operator holds the handrail of the escalator. Due to the harsh environment and changeable weather at the work site, the accuracy is low, the real-time performance is poor, and the hardware resource consumption is large.

[0026] In order to solve the above problems, the present application provides a behavior detection method, device, readable storage medium and program product. Since there are workers and escalators in the detection area, the image of the detection area is collected to obtain a detection image containing the workers and the escalator. The collected detection image is input into the target detection model. The target detection model can detect whether the worker in the detection image is holding the handrail of the escalator, so as to obtain the initial handrail posture detection result, the position information of the worker's rectangular frame and the position of the escalator rectangular frame. Using the target detection model, the input detection image is detected, so that the initial handrail posture detection result can be obtained quickly and efficiently. When the initial handrail posture detection result indicates that the worker did not hold the handrail of the escalator, based on the position information of the worker's rectangular frame and the position information of the escalator's rectangular frame, the overlapping area between the worker's rectangular frame and the escalator's rectangular frame can be determined, and the target ratio can be obtained based on the area of ​​the overlapping area and the area of ​​the worker's rectangular frame. Based on the target ratio, it can be determined again whether the worker held the handrail of the escalator. This makes the target handrail posture detection result more accurate. In addition, when the initial handrail posture detection result indicates that the worker did not hold the handrail of the escalator, re-inspection can be performed again, which can not only reduce the amount of calculation and the consumption of hardware resources, but also ensure the real-time performance of this solution.

[0027] Below, taking the well logging operation scenario as an example, the application architecture of the behavior detection method, device, readable storage medium and program product provided by the present application is briefly introduced.

[0028] Figure 1 A schematic diagram of the application architecture of a behavior detection system provided in an embodiment of the present application. The behavior detection system includes an image acquisition device 100 and an electronic device 200. The image acquisition device 100 takes multiple detection images of the detection area. After receiving the multiple detection images taken by the image acquisition device 100, the electronic device 200 determines whether the operator holds the handrail of the escalator according to the behavior detection method provided in the present application.

[0029] In practical applications, the behavior detection method provided in the embodiment of the present application can be implemented by the processor of the electronic device when executing a program or instruction. In some embodiments, the image acquisition device or other device can also have similar functions, for example: the behavior detection method provided in the embodiment of the present application is implemented when the image acquisition device installed in the detection area executes a program or instruction, and the embodiment of the present application is not limited to this.

[0030] It should be noted that the application scenarios described in the above embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. A person of ordinary skill in the art will know that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems. The behavior detection method provided by the embodiments of the present application can be applied to various application scenarios where it is necessary to determine whether the operator is holding the handrail of the escalator.

[0031] The following first introduces the behavior detection method provided in the embodiment of the present application.

[0032] Figure 2 FIG. 1 is a flow chart of a behavior detection method provided by an embodiment of the present application. Figure 2 As shown, the behavior detection method of the embodiment of the present application includes steps S201 to S205.

[0033] In step S201, a detection image is collected for a detection area, and there are workers and escalators in the detection area.

[0034] In the well logging operation scenario, the detection area may be a well site area, where there are operators and escalators. Of course, there may also be other equipment related to the well logging task in the well site area.

[0035] Among them, well logging is the process of using various instruments and equipment to measure the wellbore (usually an oil well, gas well or water well) to obtain downhole formation information.

[0036] Among them, the detection image may include one operator and one escalator. Of course, in some scenarios, the detection image may include multiple operators and multiple escalators, or the detection image may include multiple operators and one escalator, or the detection image may include one operator and multiple escalators.

[0037] In step S202, the detection image is input into the target detection model to obtain the initial handrail posture detection result, the position information of the operator's rectangular frame, and the position information of the escalator rectangular frame output by the target detection model.

[0038] The initial handrail posture detection result is used to indicate whether the operator holds the handrail of the escalator. One operator in the detection image may correspond to one operator rectangular frame and one initial handrail posture detection result.

[0039] As an optional embodiment, the detection area may have multiple workers and multiple escalators at the same time, so a detection image may include multiple workers and multiple escalators. Using the target detection model, the detection image is detected to obtain the worker rectangular frame and initial handrail posture detection results corresponding to each worker, as well as the escalator rectangular frame corresponding to each escalator. In this scenario, the detection image is input into the target detection model, and the target detection model can output multiple worker rectangular frames, multiple escalator rectangular frames and multiple initial handrail posture detection results.

[0040] At this time, each initial handrail posture detection result can be traversed. When the initial handrail posture detection result indicates that the corresponding worker did not hold the handrail of the escalator, it is determined again whether the worker held the handrail of the escalator based on the worker rectangular box and the escalator rectangular box corresponding to the worker.

[0041] Next, the network structure of the target detection model in this application is introduced.

[0042] For example, a deep learning model capable of performing target detection is lightweighted to construct a target detection model of an embodiment of the present application. For example, the deep learning model capable of performing target detection may be YOLOv8. In low light or scenes with severe exposure problems, Backbone in YOLOv8 may not be able to extract sufficiently clear features, thereby affecting detection accuracy.

[0043] Illumination Adaptive Transformer (IAT) can be used to perform low-light enhancement and exposure correction on images. Through the attention mechanism and image processing capabilities, IAT can effectively restore the illumination information of the image and repair the brightness and contrast of the image.

[0044] Using IAT as the backbone of YOLOv8, IAT can perform low-light enhancement or exposure correction on the input low-light or overexposed images. The processed feature map is then passed to other networks in YOLOv8 for target detection. After the lighting conditions are improved, YOLOv8 can better extract the features of the target, improve the detection effect under extreme lighting conditions, and improve the robustness of the target detection model.

[0045] like Figure 3 As shown, IAT includes two network branches, namely, Local Branch and Global Branch.

[0046] Among them, the local branch includes two independent Local branches, which are used to predict the pixel-level multiplication graph and addition graph, and achieve the purpose of image denoising by combining them with the input original image.

[0047] For the original image input to IAT (RGB), after entering the local branch, a convolution kernel of 3 3, and convolution processing with a channel number of 16. Then the original image after convolution processing (RGB) enters two independent Local branches respectively. Each independent Local branch includes three pixel-wise enhancement modules (PEM). In each independent Local branch, the original image after convolution processing (RGB) and original image output by PEM (RGB) is added pixel-wise to obtain an addition map. A convolution kernel of 3 is performed on the addition map. 3, and convolution processing with the number of channels (Channel) of 3, to obtain the addition image after convolution processing. In an independent Local branch, the addition image after convolution processing is combined with the original image (RGB) performs pixel-wise multiplication (element-wise product) to obtain a multiplication graph. This multiplication graph is element-wise added with the addition graph output by another independent Local branch.

[0048] Global branch, for the original image input to IAT (RGB), after entering the global branch, the original image (RGB) performs a convolution with a kernel of 3 3, and convolution with 32 channels, and then another convolution with 3 channels. 3, and convolution processing with 64 channels to obtain the original image after convolution processing (RGB). After that, the original image after convolution (RGB) is input into the cross attention module. The cross attention module determines the degree of attention to different V (values) by calculating the similarity between K (key) and Q (query), thereby achieving weighted fusion of different information to better capture long-distance dependency information and global information in the data. Afterwards, based on the original image output from the cross attention module (RGB), respectively generate the color matrix and gamma correction parameters. Then, through two convolution kernels of 1 1, and convolution processing with a channel number of 1, for the original image (RGB) Perform color correction and gamma correction to adjust the original image (RGB) brightness and contrast.

[0049] Through the local branch and the global branch, the white balance (White Balance, WB), color correction matrix (Color Correction Matrix, CCM) and gamma correction parameters are obtained to correct the original image. (RGB) for dark light enhancement and exposure correction, thus converting the original image into a (RGB) Restore to sRGB image with normal lighting.

[0050] Next, taking the well logging operation scenario as an example, obtaining multiple original images and labeling data labels for the multiple original images are introduced.

[0051] First, a plurality of original images are obtained. Specifically, a monitoring video of the well site area is obtained using a well site camera, and a plurality of original images containing escalators and operators are screened out from the monitoring video.

[0052] Secondly, multiple original images are labeled with data labels. In practical applications, the well site camera in the logging operation is facing the well platform to shoot, and the postures and movements of the operators when going up and down the well platform are relatively fixed in the image. Figure 4 As shown in the figure, the operator on the well platform has his back to the camera, with his hands open and above his body to hold the handrail of the escalator. Figure 5 As shown in the figure, the operator on the platform always faces the camera, with both hands open and diagonally below the body to hold the handrail of the escalator. Figure 6 As shown in the figure, the hands of the workers who go up and down the well platform without holding the handrail are not open and are at the sides of the body. Based on this, the handrail posture of the workers can be divided into the handrail of the escalator in front, the handrail of the escalator in the back, and the handrail of the escalator. Based on the results of the above analysis, the workers and escalators in multiple original images can be annotated in the form of rectangular frames, so that the data labels corresponding to the multiple original images can be obtained.

[0053] Finally, through the above process, multiple original images and data labels corresponding to the multiple original images can be obtained.

[0054] Next, the process of determining multiple enhanced processed images based on multiple original images is introduced.

[0055] In one embodiment of the present application, local exposure processing is performed on multiple original images to obtain multiple original images after local exposure processing, including: generating a target circle in each original image; determining the distance between each pixel point in each original image and the center of the target circle to obtain multiple target distances; determining the exposure pixel value of each pixel point based on the size relationship between the target distance of each pixel point and the radius of the target circle; adjusting the pixel value of each pixel point to the exposure pixel value to obtain each original image after local exposure processing.

[0056] The light sources in real scenes are mostly point light sources, such as the sun, light bulbs, etc. For images in exposure scenes, a circular exposure area centered on the light source is generally formed. In this solution, a target circle is generated in each original image, and the target distance between each pixel in the original image and the center of the target circle is calculated. Based on the size relationship between the target distance of each pixel and the radius of the target circle, the exposure pixel value of the pixel is determined, so that local exposure processing of the original image can be achieved. In addition, by using the target circle, this solution can better simulate the local exposure scenario in the real scene.

[0057] Among them, a target circle can be randomly generated in each original image. Of course, when simulating exposure under a multi-light source situation, multiple target circles can be randomly generated in each original image.

[0058] When randomly generating a target circle in each original image, a position in the original image can be randomly selected as the center of the target circle, and a radius of the target circle can be randomly generated, so as to realize the random generation of the target circle in the original image. Of course, in the actual application process, the position of the center of the randomly selected target circle can be inside the original image, at the edge of the original image, or at the corner point of the original image.

[0059] In one embodiment of the present application, the exposure pixel value of each pixel is determined based on the size relationship between the target distance of each pixel and the radius of the target circle, including: when the target distance of the pixel is less than the radius of the target circle, the exposure pixel value of the pixel is determined as the pixel upper limit value; when the target distance of the pixel is greater than or equal to the radius of the target circle, the exposure pixel value of the pixel is determined based on a predetermined adjustment strategy, wherein the predetermined adjustment strategy is: as the target distance increases, the exposure pixel value gradually decreases.

[0060] Optionally, when the target distance of the pixel point is smaller than the radius of the target circle, the exposure pixel value of the pixel point may be 255, that is, the pixel upper limit value may be 255.

[0061] The pixel values ​​of the pixels inside the target circle are adjusted to the upper pixel limit, so that the pixel values ​​inside the target circle can be used to simulate overexposure. For the pixels outside the target circle, the exposure pixel values ​​of the pixels are determined according to the strategy of "as the distance to the target increases, the exposure pixel value gradually decreases". That is to say, for the pixels outside the target circle, the farther they are from the center of the target circle, the smaller the exposure pixel value, and the closer they are to the center of the target circle, the larger the exposure pixel value. This can more realistically simulate the situation where the acquired image is exposed due to a strong light source, and further improve the robustness of the target detection model.

[0062] In one embodiment of the present application, based on a predetermined adjustment strategy, the exposure pixel value of a pixel point is determined, including: determining an initial exposure correction parameter based on a target distance and a radius of a target circle; determining the maximum value of a preset correction parameter value and the initial exposure correction parameter as a target exposure correction parameter; determining an incremental intensity value based on the product of the target exposure correction parameter and a preset incremental intensity parameter; determining an initial exposure pixel value based on the sum of the pixel value of the pixel point and the incremental intensity value; and determining the minimum value of a pixel upper limit value and the initial exposure pixel value as the exposure pixel value of the pixel point. In this way, the pixel value of a pixel point closer to the center of the target circle is larger, and the pixel value of a pixel point farther from the center of the target circle is smaller, so that the situation in which the acquired image is exposed when there is a strong light source can be further better simulated.

[0063] The predetermined adjustment strategy can be expressed in the form of a mathematical expression:

[0064] Among them, P is used to represent the exposure pixel value, I (x, y) is used to represent the pixel value of each pixel in the original image, distance is used to represent the target distance, r is used to represent the radius of the target circle, and max_increase is used to represent the preset incremental intensity parameter. Used to indicate initial exposure correction parameters.

[0065] In actual application, the preset correction parameter may be 0, and the preset incremental intensity parameter is used to adjust the exposure intensity. Optionally, the preset incremental intensity parameter may be 100.

[0066] In one embodiment of the present application, the following formula can be used to perform gamma correction on multiple original images after local exposure processing to obtain multiple enhanced processed images.

[0067] As an optional embodiment, based on , determine the pixel value in each enhanced image, where, It is used to represent the pixel value of each pixel in the enhanced image, and I(x,y) is used to represent the pixel value of each pixel in the original image after local exposure processing. Used to represent adjustment factors.

[0068] Optionally, It can be 0.75.

[0069] By changing the brightness of multiple original images after local exposure processing based on gamma correction, different lighting conditions can be simulated. For example, reducing the gamma value can make the image darker, thereby simulating the effect in a low-light environment.

[0070] In this application, a plurality of original images are subjected to local exposure processing and gamma correction processing to obtain a plurality of enhanced processed images. The complex environment in the actual well logging operation scene can be simulated through the plurality of enhanced processed images.

[0071] There are many ways to implement the training of target detection models.

[0072] In one embodiment, the target detection model may be trained based on multiple original images to obtain a trained target detection model.

[0073] In another embodiment, a plurality of original images may be subjected to local exposure processing to obtain a plurality of original images after local exposure processing; a target detection model may be trained using the plurality of original images after local exposure to obtain a trained target detection model.

[0074] In another embodiment, gamma correction processing may be performed on multiple original images to obtain multiple original images after gamma correction processing; and the target detection model may be trained using the multiple original images after gamma correction processing to obtain a trained target detection model.

[0075] In another embodiment, the process of training the target detection model includes: performing local exposure processing on multiple original images to obtain multiple original images after local exposure processing; performing gamma correction processing on the multiple original images after local exposure processing to obtain multiple enhanced processed images; based on the multiple original images and the multiple enhanced processed images, training the target detection model to obtain a trained target detection model.

[0076] In this embodiment, a plurality of original images are subjected to local exposure processing and gamma correction processing to obtain a plurality of enhanced processed images, which can not only increase the diversity of data, but also simulate the exposure conditions of images in real scenes through a plurality of enhanced processed images. The target detection model is then trained using a plurality of original images and a plurality of enhanced processed images to improve the robustness of the target detection model.

[0077] In the specific training process, the target detection model can also be trained based on multiple original images and data labels corresponding to the multiple original images, multiple enhanced processed images and data labels corresponding to the multiple enhanced processed images to obtain a trained target detection model.

[0078] In step S203, it is determined whether the initial handrail posture detection result indicates that the operator is not holding the handrail of the escalator.

[0079] In step S204, when the initial handrail posture detection result indicates that the worker is not holding the handrail of the escalator, the overlapping area of ​​the worker rectangular frame and the escalator rectangular frame is determined based on the position information of the worker rectangular frame and the position information of the escalator rectangular frame.

[0080] It should be noted that the position information of the operator rectangular frame and the position information of the escalator rectangular frame may be coordinate information in a coordinate system corresponding to the original image.

[0081] In one embodiment of the present application, based on the position information of the operator rectangular frame and the position information of the escalator rectangular frame, the overlapping area between the operator rectangular frame and the escalator rectangular frame is determined, including: determining the cropping parameters based on the height value of the escalator rectangular frame; based on the cropping parameters, cropping the bottom area and the top area of ​​the escalator rectangular frame respectively to obtain a cropped escalator rectangular frame; based on the position information of the cropped escalator rectangular frame and the position information of the operator rectangular frame, determining the overlapping area.

[0082] In actual application, workers on the ground or workers on the well platform in logging scenarios may be at the edge of the escalator. In this case, the target detection model will mistakenly determine that the workers in this case are not holding the handrail of the escalator. Therefore, in order to reduce the probability of misjudgment, this solution crops the bottom area and the top area of ​​the escalator rectangular frame based on the cropping parameters, that is, the escalator area that is connected to the ground or the well platform is cropped off. In this way, based on the position information of the cropped escalator rectangular frame and the position information of the worker rectangular frame, the overlapping area is determined, so that the determined overlapping area is more accurate, and further the target ratio determined subsequently is more accurate, further achieving a more accurate determination of whether the worker is holding the handrail of the escalator.

[0083] In step S205, a target ratio is obtained based on the ratio of the area of ​​the overlapping region to the area of ​​the rectangular frame of the operator, and a target handrail posture detection result of the operator is determined based on the target ratio.

[0084] In one embodiment of the present application, when the target ratio is greater than the preset ratio, the target handrail posture detection result of the operator is determined as: the operator is not holding the handrail of the escalator. In this way, the target handrail posture detection result of the operator is determined quickly and efficiently.

[0085] The target handrail posture detection result can be used to characterize whether the operator holds the escalator handrail.

[0086] Optionally, the preset ratio may be 0.5.

[0087] In an optional embodiment, when it is determined that the target handrail posture detection result of the operator is that the operator is not holding the handrail of the escalator, a reminder device at the work site can issue a prompt message.

[0088] In an embodiment of the present application, the input detection image is detected using a target detection model, so that the initial handrail posture detection result can be obtained quickly and efficiently. In the case where the initial handrail posture detection result indicates that the operator did not hold the handrail of the escalator, based on the position information of the operator's rectangular frame and the position information of the escalator rectangular frame, it can be determined again whether the operator held the handrail of the escalator, so that the target handrail posture detection result is more accurate, and in the case where the initial handrail posture detection result indicates that the operator did not hold the handrail of the escalator, re-inspection is performed again, which can not only reduce the amount of calculation and reduce the consumption of hardware resources, but also ensure the real-time performance of this solution.

[0089] For ease of understanding, take the logging operation scenario as an example and combine Figure 7 , the behavior detection method in the embodiment of the present application is further described. Figure 7 As shown, the behavior detection method includes steps S701 to S709.

[0090] In step S701, the monitoring video of the well logging operation site is obtained using the well site camera .

[0091] In step S702, from the surveillance video Multiple original images with escalators and workers going up and down escalators are screened out.

[0092] In step S703, a well site data set is constructed.

[0093] based on Figure 4 , Figure 5 as well as Figure 6 For example, the handrail postures of the workers in multiple original images are classified and labeled, and the escalators in multiple original images are labeled to obtain data labels corresponding to the multiple original training images.

[0094] Local exposure processing and gamma correction processing are performed on multiple original images to obtain multiple enhanced processed images.

[0095] A well site data set is composed of a plurality of original images and data labels corresponding to the plurality of original images, and a plurality of enhanced processed images and data labels corresponding to the plurality of enhanced processed images.

[0096] The well site data set is divided according to the ratio of 8:2 to obtain training samples and validation samples .

[0097] In step S704, IAT is used as the Backbone network of YOLOv8 to obtain the target detection model in the embodiment of the present application. and validation samples , train the target detection model to obtain the trained target detection model.

[0098] In step S705, the well site camera at the logging operation site is connected. A tool with video encoding, decoding, post-processing and other functions can be used to extract frames from the video stream of the well site camera at a predetermined frame interval to obtain a detection image. .

[0099] For example, a tool with functions such as video encoding, decoding, and post-processing may be FFmpeg. Based on FFmpeg, the video stream sent by the well site camera is subjected to frame extraction processing to obtain multiple detection images. Specifically, one frame of image may be extracted from the video stream every N frames. N may be determined according to the inference speed of the target detection model and the task requirements.

[0100] In step S706, the detected image The input is input into the trained target detection model to obtain the escalator rectangular frame, operator rectangular frame and initial handrail posture detection results output by the target detection model.

[0101] In step S707, the cropping parameters are determined based on the height value of the escalator rectangular frame; based on the cropping parameters, the bottom area and the top area of ​​the escalator rectangular frame are cropped respectively to obtain a cropped escalator rectangular frame, which can avoid the problem of misjudgment caused by the workers on the ground and the workers on the well platform when going up and down the escalator.

[0102] In step S708, each initial handrail posture detection result is traversed in turn to determine whether the initial handrail posture detection result indicates that the operator does not hold the handrail of the escalator.

[0103] In step S709, when the initial handrail posture detection result indicates that the corresponding worker does not hold the handrail of the escalator, a target ratio is determined.

[0104] based on , calculate the target ratio, where Used to indicate the target ratio, It is used to represent the operator rectangular frame corresponding to the initial handrail posture detection result. Used to represent the cropped escalator rectangle. Rectangular frame used to represent workers area.

[0105] like When the value is >0.5, it indicates that the operator did not hold the handrail of the escalator when going up and down the well platform. Therefore, the reminder device at the logging operation site can send a reminder message.

[0106] Based on the same inventive concept, the present application embodiment also provides a behavior detection device. Figure 8 , the behavior detection device provided in the embodiment of the present application is described in detail.

[0107] It should be noted that the behavior detection device is a device corresponding to the above-mentioned behavior detection method. All implementation methods in the above-mentioned method embodiment are applicable to the embodiment of the device and can achieve the same technical effect, which will not be repeated here.

[0108] Figure 8 Schematic diagram of a behavior detection device provided in an embodiment of the present application. Figure 8 As shown, the behavior detection device 800 includes a collection module 801 , an acquisition module 802 , a judgment module 803 , a first determination module 804 and a second determination module 805 .

[0109] The acquisition module 801 can be used to acquire detection images of the detection area, where there are workers and escalators.

[0110] The acquisition module 802 can be used to input the detection image into the target detection model to obtain the initial handrail posture detection result, the position information of the operator's rectangular frame and the position information of the escalator rectangular frame output by the target detection model.

[0111] The judgment module 803 can be used to judge whether the initial handrail posture detection result indicates that the operator is not holding the handrail of the escalator.

[0112] The first determination module 804 can be used to determine the overlapping area of ​​the operator rectangular frame and the escalator rectangular frame based on the position information of the operator rectangular frame and the position information of the escalator rectangular frame when the initial handrail posture detection result indicates that the operator is not holding the escalator handrail.

[0113] The second determination module 805 can be used to obtain a target ratio based on the ratio of the area of ​​the overlapping region to the area of ​​the operator's rectangular frame, and determine the target handrail posture detection result of the operator based on the target ratio.

[0114] In one embodiment, the acquisition module can also be used to perform local exposure processing on multiple original images to obtain multiple original images after local exposure processing; perform gamma correction processing on the multiple original images after local exposure processing to obtain multiple enhanced processed images; based on the multiple original images and the multiple enhanced processed images, train the target detection model to obtain the trained target detection model.

[0115] In one embodiment, the acquisition module can also be used to generate a target circle in each original image; determine the distance between each pixel point in each original image and the center of the target circle to obtain multiple target distances; determine the exposure pixel value of each pixel point based on the size relationship between the target distance of each pixel point and the radius of the target circle; adjust the pixel value of each pixel point to the exposure pixel value to obtain each original image after local exposure processing.

[0116] In one embodiment, the acquisition module can also be used to determine the exposure pixel value of the pixel point as the pixel upper limit value when the target distance of the pixel point is less than the radius of the target circle; when the target distance of the pixel point is greater than or equal to the radius of the target circle, determine the exposure pixel value of the pixel point based on a predetermined adjustment strategy, wherein the predetermined adjustment strategy is: as the target distance increases, the exposure pixel value gradually decreases.

[0117] In one embodiment, the acquisition module can also be used to determine an initial exposure correction parameter based on the target distance and the radius of the target circle; determine the maximum value of a preset correction parameter value and the initial exposure correction parameter as the target exposure correction parameter; determine the incremental intensity value based on the product of the target exposure correction parameter and the preset incremental intensity parameter; determine the initial exposure pixel value based on the sum of the pixel value of the pixel point and the incremental intensity value; and determine the minimum value of a pixel upper limit value and the initial exposure pixel value as the exposure pixel value of the pixel point.

[0118] In one embodiment, the first determination module can also be used to determine cropping parameters based on the height value of the escalator rectangular frame; based on the cropping parameters, the bottom area and the top area of ​​the escalator rectangular frame are cropped respectively to obtain a cropped escalator rectangular frame; based on the position information of the cropped escalator rectangular frame and the position information of the operator rectangular frame, the overlapping area is determined.

[0119] In one embodiment, the second determination module may also be used to determine, when the target ratio is greater than a preset ratio, that the target handrail posture detection result of the operator is: the operator is not holding the handrail of the escalator.

[0120] Fig. 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0121] The electronic device may include a processor 901 and a memory 902 storing computer program instructions.

[0122] Specifically, the processor 901 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0123] The memory 902 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 902 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 902 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 902 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 902 is a non-volatile solid-state memory.

[0124] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present application.

[0125] The processor 901 implements any one of the behavior detection methods in the above embodiments by reading and executing computer program instructions stored in the memory 902 .

[0126] In one example, the electronic device may further include a communication interface 903 and a bus 910. Fig. 9 As shown, the processor 901, the memory 902, and the communication interface 903 are connected via a bus 910 and communicate with each other.

[0127] The communication interface 903 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0128] Bus 910 includes hardware, software or both components coupled to each other. For example and not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 910 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0129] The electronic device can execute the behavior detection method in the embodiment of the present application, thereby realizing the combination Figures 2 to 7 Describe the behavioral detection method.

[0130] In addition, in combination with the behavior detection method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the behavior detection methods in the above embodiment is implemented.

[0131] An embodiment of the present application also provides a computer program product, including a computer program, which implements any one of the behavior detection methods in the above embodiments when the computer program is processed and executed.

[0132] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0133] The functional blocks shown in the structural block diagram described above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" may include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0134] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0135] The above describes various aspects of the present application with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0136] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A behavior detection method, characterized in that: include: Collecting a detection image of a detection area, where there are workers and escalators in the detection area; Inputting the detection image into the target detection model, obtaining the initial handrail posture detection result, the position information of the operator rectangular frame and the position information of the escalator rectangular frame output by the target detection model; Determining whether the initial handrail posture detection result indicates that the operator does not hold the handrail of the escalator; In a case where the initial handrail posture detection result indicates that the operator is not holding the handrail of the escalator, determining an overlapping area between the operator rectangular frame and the escalator rectangular frame based on position information of the operator rectangular frame and position information of the escalator rectangular frame; Based on the ratio of the area of ​​the overlapping region to the area of ​​the operator rectangular frame, a target ratio is obtained, and based on the target ratio, a target handrail posture detection result of the operator is determined.

2. The method according to claim 1, characterized in that The process of training the object detection model includes: Performing local exposure processing on a plurality of original images to obtain the plurality of original images after the local exposure processing; Performing gamma correction processing on the multiple original images after local exposure processing to obtain multiple enhanced processed images; Based on the multiple original images and the multiple enhanced processed images, the target detection model is trained to obtain the trained target detection model.

3. The method according to claim 2, characterized in that Performing local exposure processing on a plurality of original images to obtain the plurality of original images after the local exposure processing includes: Generating a target circle in each of the original images; Determine the distance between each pixel point in each of the original images and the center of the target circle to obtain multiple target distances; Determining an exposure pixel value of each pixel point based on a size relationship between the target distance of each pixel point and the radius of the target circle; The pixel value of each pixel point is adjusted to the exposure pixel value to obtain each original image after local exposure processing.

4. The method according to claim 3, characterized in that Determining the exposure pixel value of each pixel point based on the size relationship between the target distance of each pixel point and the radius of the target circle includes: When the target distance of the pixel point is less than the radius of the target circle, determining the exposure pixel value of the pixel point as a pixel upper limit value; When the target distance of the pixel point is greater than or equal to the radius of the target circle, the exposure pixel value of the pixel point is determined based on a predetermined adjustment strategy, wherein the predetermined adjustment strategy is: as the target distance increases, the exposure pixel value gradually decreases.

5. The method according to claim 4, characterized in that Determining the exposure pixel value of the pixel point based on a predetermined adjustment strategy includes: determining an initial exposure correction parameter based on the target distance and the radius of the target circle; Determine the maximum value of the preset correction parameter value and the initial exposure correction parameter as the target exposure correction parameter; Determining an incremental intensity value based on a product of the target exposure correction parameter and a preset incremental intensity parameter; Determining an initial exposure pixel value based on a sum of a pixel value of the pixel point and the incremental intensity value; The minimum value between the pixel upper limit value and the initial exposure pixel value is determined as the exposure pixel value of the pixel point.

6. The method according to any one of claims 1 to 5, characterized in that Based on the position information of the operator rectangular frame and the position information of the escalator rectangular frame, determining the overlapping area between the operator rectangular frame and the escalator rectangular frame includes: Determining a cropping parameter based on a height value of the escalator rectangular frame; Based on the cropping parameters, cropping the bottom area and the top area of ​​the escalator rectangular frame respectively to obtain the cropped escalator rectangular frame; The overlapping area is determined based on the position information of the cropped escalator rectangular frame and the position information of the operator.

7. The method according to any one of claims 1 to 5, characterized in that Based on the target ratio, determining the target handrail posture detection result of the operator includes: When the target ratio is greater than the preset ratio, the target handrail posture detection result of the operator is determined as: the operator is not holding the handrail of the escalator.

8. A behavior detection device, characterized in that: include: A collection module, used for collecting a detection image of a detection area, wherein there are operating personnel and escalators in the detection area; An acquisition module, used for inputting the detection image into a target detection model to obtain the initial handrail posture detection result, the position information of the operator rectangular frame and the position information of the escalator rectangular frame output by the target detection model; A judgment module, used to judge whether the initial handrail posture detection result indicates that the operator does not hold the handrail of the escalator; A first determination module is used to determine an overlapping area between the operator rectangular frame and the escalator rectangular frame based on the position information of the operator rectangular frame and the position information of the escalator rectangular frame when the initial handrail posture detection result indicates that the operator does not hold the handrail of the escalator; The second determination module is used to obtain a target ratio based on the ratio of the area of ​​the overlapping area to the area of ​​the operator's rectangular frame, and determine a target handrail posture detection result of the operator based on the target ratio.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the behavior detection method according to any one of claims 1 to 7 is implemented.

10. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the behavior detection method as described in any one of claims 1 to 7.

Images