An open-pit mine-oriented agent remote monitoring method and system

Abstract

The present application relates to a kind of open-pit mine-oriented agent remote monitoring method and system, belong to unmanned control technical field, solve the problem that the open-pit mine operating environment in prior art is prone to miss detection, false detection barrier and the problem that key monitoring information is not easy to be perceived.The agent remote monitoring method and system of the present application are provided with image collector, image processing unit and remote monitoring platform, by enhancing the image collected to process and based on the deflection angle and the overhead angle of image collector, the calibration before image collector and agent is carried out, reduce the miss detection and false detection of barrier, improve the accuracy, stability and robustness of barrier ranging, can improve the accuracy of agent to collision risk assessment.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of unmanned control, and relates to an intelligent agent remote monitoring method and system for open-pit mines. BACKGROUND

[0002] With the popularization of unmanned technology, unmanned transport vehicles are increasingly used in mine areas, logistics parks, university campuses and other scenarios. As the current mainstream research direction, unmanned transport vehicles can fundamentally solve the safety problem of drivers, reduce labor costs, and greatly improve production and transportation efficiency. In open-pit mines, unmanned technology is also widely used. However, in open-pit mines, in order to ensure the safety and reliability of unmanned mine trucks, it is extremely important to timely and accurately determine the information of obstacles. Therefore, in order to enable monitoring personnel to better monitor the vehicle's surrounding environment, unmanned mine trucks are equipped with multiple cameras for remote monitoring. The video stream data of these cameras is transmitted through the network and displayed to the remote monitoring platform terminal. Remote supervisors will watch the video, and when the mine truck encounters a risk, the supervisor will manually intervene in the driving state of the unmanned mine truck. However, during monitoring, the supervisor needs to consider multiple video screens at the same time, which can easily lead to a dispersion of personnel's energy and is not conducive to real-time and accurate guarantee of vehicle safety. Therefore, it is crucial to build a dynamic remote monitoring system to remind the monitoring personnel. SUMMARY

[0003] In view of the above problems, the present application provides an intelligent agent remote monitoring method and system for open-pit mines, which is used for unmanned vehicles, unmanned aerial vehicles and intelligent robots operating in open-pit mine environments. The problems of easy missed detection and false detection of obstacles in the existing technology in open-pit mine operating environments and the problem of key monitoring information not being easily detected are solved.

[0004] The present application provides an intelligent agent remote monitoring method for open-pit mines, and the specific steps are as follows:

[0005] Obtain the current frame image data of the intelligent agent end;

[0006] Perform image enhancement processing on the current frame image to obtain an enhanced image, and the specific steps are as follows:

[0007] Filter out the expression of the value of the first pixel point in the image after filtering out noise during the acquisition of each pixel point in the image: i

[0008] pi ( xi , yi )=( pi ( xi -1, yi -1)+ pi ​( xi , yi -1)+ pi ( xi +1, yi -1)+ pi ( xi -1, yi )+ pi ( xi , yi )

[0009] + pi ( xi +1, yi )+ pi ( xi -1, yi )+ pi ( xi , yi +1)+ pi ( xi +1, yi +1))*(1 / 9);

[0010] in, xi For the first in the image i The x-coordinate of each pixel yi For the first in the image i The ordinate of each pixel; For the first in the image i The pixel value after noise removal from each pixel;

[0011] Each noise-filtered pixel is converted to a grayscale pixel to obtain a grayscale image. i The expression for the grayscale pixel value of a pixel is:

[0012] ;

[0013] ;

[0014] in, Indicates the first in the image i The pixel value of each pixel in the red channel of the original image; Indicates the first in the image i The pixel value of each pixel in the green channel of the original image; Indicates the first in the image i The pixel value of each pixel in the blue channel of the original image; Indicates the first in the image i The pixel value of each pixel in the grayscale image; , and These are the weighting coefficients;

[0015] The edge detail value of each pixel point in the gray image is obtained by using the second-order differential, and the expression of the edge detail value of the i-th pixel point in the image is: i

[0016] The expression of the edge detail value is:

[0017]

[0018]

[0019] wherein, di is the second-order derivative of the gray pixel value of the i-th pixel point in the image, di is the second-order derivative of the gray pixel value of the i-th pixel point in the image along the x-axis, i di is the second-order derivative of the gray pixel value of the i-th pixel point in the image along the y-axis; i x i y

[0020] The enhanced pixel point is obtained by superimposing the pixel point value after filtering out the noise of each pixel point and the edge detail value of the pixel point, and the expression of the enhanced pixel point value of the i-th pixel point in the image is: i

[0021]

[0022] wherein, a is the enhanced coefficient;

[0023] All the corresponding enhanced pixel points in the current frame image are obtained by traversing all the pixel points in the current frame image, and the current frame enhanced image is obtained by all the enhanced pixel points;

[0024] The pixel coordinate information of the rectangular bounding box of the obstacle in the current frame image is obtained based on the current frame enhanced image;

[0025] The distance between the agent and the obstacle is obtained based on the pixel coordinate information of the rectangular bounding box of the obstacle in the current frame image, and the collision risk is evaluated;

[0026] Based on the collision risk evaluation result, the image code rate parameter is adjusted using the image code rate dynamic decision method;

[0027] The current frame image is encoded using the adjusted image code rate parameter to obtain the encoded image data, and the encoded image data is transmitted to the remote monitoring platform end.

[0028] Optionally, the specific steps of obtaining the pixel coordinate information of the rectangular bounding box of the obstacle in the current frame enhanced image are as follows: the coordinate information of the four corner points of the rectangular bounding box: the left upper corner pixel point coordinate​​​​​​​​​​​x 1, y 1), the upper right pixel point coordinate x 2, y 2), the lower left pixel point coordinate x 3, y 3) and the lower right pixel point coordinate x 4, y 4).

[0029] Optionally, based on the pixel point coordinate information of the rectangular bounding box of the obstacle in the current frame image, the distance between the agent and the obstacle is obtained, and the specific steps of evaluating the collision risk are as follows:

[0030] The coordinate of the midpoint pixel point of the line connecting the lower left pixel point and the lower right pixel point in the rectangular bounding box of the obstacle is obtained u , v ):

[0031] ;

[0032] ;

[0033] The horizontal distance and the vertical distance between the midpoint pixel point and the obstacle are obtained;

[0034] ;

[0035] ;

[0036] wherein, h is the height of the image collector from the ground; h is the overhead angle of the image collector, which can be obtained by manual measurement; , are the internal parameters of the image collector; Based on the deflection angle

[0037] of the image collector and the coordinate of the midpoint pixel point θ , u , v ), the horizontal distance x ’’ and the vertical distance y ’’ between the agent and the obstacle at this time are obtained, and the expressions of the horizontal distance x ’’ and the vertical distance y ’’ are as follows:

[0038] If , then:

[0039] ;

[0040] ;

[0041] If , then:

[0042] ;

[0043] ;

[0044] If , then:

[0045] ;

[0046] ;

[0047] wherein width is the horizontal pixel resolution of the current frame image;

[0048] The collision risk of the agent is evaluated based on the horizontal distance and the vertical distance between the obstacle and the agent.

[0049] Optionally, the method for evaluating the collision risk of the agent is: setting a safety distance as d 1, and a braking distance of the agent as d 2; when the vertical distance between the obstacle and the agent is y ’’> d 1, it is determined that the collision risk level at this time is low; when the vertical distance between the obstacle and the agent is d 2 y < d 1, it is determined that the collision risk level at this time is medium; when the vertical distance between the obstacle and the agent is y ’ d 2, it is determined that the collision risk level at this time is high.

[0050] Optionally, based on the collision risk evaluation result, the specific steps for adjusting the current frame image code rate parameter using the image code rate dynamic decision method are:

[0051] determining whether the collision risk evaluation result is high, and if it is high, setting the code rate to a high code rate threshold M3;

[0052] if it is not high, further determining whether the collision risk evaluation result is medium; if the collision risk evaluation result is medium, setting the code rate to a medium code rate threshold M2;

[0053] if it is not medium, obtaining the risk evaluation result of the previous frame image; determining whether the risk evaluation result of the previous frame image is low; if the risk evaluation result of the previous frame image is low, setting the code rate to a low code rate threshold M1; if the previous frame is not low, setting the code rate to a medium code rate threshold M2.

[0054] Optionally, based on the collision risk assessment result, if the collision risk assessment result is not medium, if the current frame image is the first frame image, the risk assessment result of the previous frame image is set to be the same as the risk assessment result of the current frame image when adjusting the current frame image code rate parameter using the image code rate dynamic decision method.

[0055] Optionally, the specific step of encoding the current frame image using the adjusted image code rate parameter is:

[0056] Converting the image format with the collision risk assessment result into a color-coded format;

[0057] Encoding the converted format image using the adjusted image code rate parameter to obtain the encoded current frame image;

[0058] Pushing the encoded current frame image to the remote monitoring platform end.

[0059] Optionally, the category information of the obstacle in the current frame image is also obtained when the enhanced image is detected for obstacles.

[0060] Optionally, after the remote monitoring platform end obtains the encoded current frame image, the encoded image data and / or the category information of the obstacle in the current frame image are displayed on the display device.

[0061] On the other hand, the present application also discloses an intelligent agent remote monitoring system for open-pit mines, comprising an image collector, an image processing unit and a remote monitoring platform arranged on the intelligent agent; characterized in that,

[0062] The image collector is used to collect the environment image or video when the intelligent agent is running in real time, and transmit the current frame image of the environment image or video to the image processing unit;

[0063] The image processing unit is used to perform image enhancement processing on the current frame image, detect obstacles based on the enhanced image after the enhancement processing, obtain the pixel coordinate information of the rectangular bounding box of the detected obstacle in the image, perform obstacle distance calculation and collision risk assessment judgment using the pixel coordinate information as input, adjust the image stream code rate parameter using the image code rate dynamic decision method according to the collision risk assessment result, encode the image according to the image stream code rate parameter, and transmit the encoded image data to the remote monitoring platform;

[0064] The remote monitoring platform is used to receive the encoded image data, and display the encoded image data on the terminal display device.

[0065] Compared with the prior art, the present application can at least achieve the following beneficial effects:

[0066] The remote monitoring method of the present application adopts an image enhancement method on the image before target detection, enhances the contour information of the obstacle, can improve the contrast of the obstacle and the background in the image, and is thus beneficial to more accurately and quickly detecting the obstacle. The remote monitoring method of the present application improves the accuracy, stability and robustness of the obstacle ranging based on the deflection angle and the overhead angle of the image collector for the calibration between the image collector and the intelligent agent, and can provide the accuracy of the collision risk assessment of the intelligent agent. The remote monitoring method of the present application adopts an image code rate dynamic decision method, uses the collision risk assessment result as prior information, dynamically adjusts the video code rate parameter, makes the video with dangerous situation transmitted to the monitoring personnel as key video and / or issues an alarm to automatically remind the safety officer, achieves the purpose of reminding the monitoring personnel to focus more attention on the key video, is beneficial to the monitoring personnel to more quickly find the dangerous situation, and also reduces the working strength of the monitoring personnel. The remote monitoring method of the present application improves the accuracy of distance perception according to the detection target category information and the rectangular bounding box information to obtain the distance information between the obstacle and the intelligent agent. The remote monitoring system of the present application is provided with multiple image collectors, so that the unmanned mine truck can collect environmental image data from multiple angles during driving. BRIEF DESCRIPTION OF DRAWINGS

[0067] The accompanying drawings are included to provide a further understanding of the present application, and are incorporated herein and constitute a part of this application.

[0068] Figure 1 is a schematic diagram of the remote monitoring system of the present application;

[0069] Figure 2 is a flowchart of the remote monitoring method of the present application;

[0070] Figure 3 is a flowchart of the image code rate dynamic decision method of the remote monitoring method of the present application. DETAILED DESCRIPTION

[0071] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict. In addition, the present application can also be implemented in other ways different from those described herein, and thus the protection scope of the present application is not limited by the specific embodiments disclosed below.

[0072] One specific embodiment of the present application, as Figures 1-3 , discloses an intelligent agent remote monitoring method and system for open-pit mine, which is used for intelligent agents such as unmanned vehicles, unmanned aerial vehicles and intelligent robots working in open-pit mine environment, and the specific steps are as follows:

[0073] S1, obtaining current frame image data of an agent end;

[0074] Optionally, the agent is an unmanned mining truck, an unmanned vehicle, a drone, an intelligent robot, or the like, and the image data is collected by an image collector arranged at the agent end; the image collector is a camera, a camera, or the like image or video collection device; the image collector is arranged in multiple paths, and the multiple-path image collector is arranged at different positions of the agent, such as front, rear, left, or right positions of the agent.

[0075] Optionally, the current frame image data is obtained from real-time video stream data or multi-frame image data.

[0076] S2, performing image enhancement processing on the current frame image to obtain a current frame enhanced image;

[0077] S3, performing obstacle detection based on the current frame enhanced image to obtain pixel point coordinate information of a rectangular bounding box of the obstacle in the current frame image; the rectangular bounding box is a rectangular bounding box surrounding the obstacle; further, the category information of the obstacle in the current frame image is also obtained; further, when there are multiple obstacles in the current frame image, the rectangular bounding box is a rectangular bounding box formed after surrounding all the obstacles;

[0078] S4, based on the pixel point coordinate information of the rectangular bounding box of the obstacle in the current frame image, obtaining the distance between the agent and the obstacle, and evaluating the collision risk;

[0079] S5, based on the collision risk evaluation result, using an image code rate dynamic decision method or a video stream code rate dynamic decision method to adjust the current frame image code rate parameter or the code rate parameter of the video stream data in which the current frame image is located;

[0080] S6, using the adjusted image code rate parameter or the code rate parameter of the video stream data in which the current frame image is located to encode the current frame image to obtain encoded image data; and transmitting the encoded image data to a remote monitoring platform end;

[0081] S7, displaying the encoded image data on a display device; further, simultaneously reminding the monitoring personnel automatically according to the risk evaluation result.

[0082] Further, when the category information of the obstacle in the current frame image is also obtained, the category information is transmitted to the monitoring platform end, and the category information of the obstacle is displayed on the display device at the same time, so as to facilitate the monitoring personnel to obtain the multi-aspect information of the obstacle in time and accurately.

[0083] Optionally, the display device is a display screen; when a multi-channel image acquisition device is set, the display device simultaneously or in time-division displays multi-channel encoded image data and / or obstacle category information; after processing multiple frames of images, single-frame image data, multi-frame image data, or video stream data are displayed on the display device.

[0084] Optionally, the specific steps for performing image enhancement processing on the current frame image in step S2 to obtain the enhanced image are as follows:

[0085] S21. Filter out noise (such as noise from the sensor) from each pixel during image acquisition to obtain the filtered image. The image after noise removal... i The expression for the value of a pixel is:

[0086] pi ( xi , yi )=( pi ( xi -1, yi -1)+ pi ( xi , yi -1)+ pi ( xi +1, yi -1)+ pi ( xi -1, yi )+ pi ( xi , yi )

[0087] + pi ( xi +1, yi )+ pi ( xi -1, yi )+ pi ( xi , yi +1)+ pi ( xi +1, yi +1))*(1 / 9);

[0088] in, xi For the first in the image i The x-coordinate of each pixel yi For the first in the image i The ordinate of each pixel; For the first in the image i The pixel value after filtering out noise.

[0089] S22. Convert each noise-filtered pixel to a grayscale pixel to obtain the grayscale image of the filtered image.i The expression of the gray pixel point value of the i-th pixel point is:

[0090] ;

[0091] ;

[0092] The original image is divided into three color channels of red, green and blue, and the gray image is one color channel. The pixel point value of the i-th pixel point in the red channel of the original image is represented as: i The pixel point value of the i-th pixel point in the green channel of the original image is represented as: The pixel point value of the i-th pixel point in the blue channel of the original image is represented as: i The pixel point value of the i-th pixel point in the gray image is represented as: , i , , i , , , are weight coefficients.

[0093] S23, obtain the edge detail value of each pixel point in the gray image by second-order differentiation, and the expression of the edge detail value of the i-th pixel point in the image is: i

[0094]

[0095]

[0096] ;

[0097] wherein, is the second-order derivative of the gray pixel point value of the i-th pixel point in the image, i is the second-order derivative of the gray pixel point value of the i-th pixel point in the image along the x-axis, is the second-order derivative of the gray pixel point value of the i-th pixel point in the image along the y-axis. i x i y

[0098] S24, superimpose the filtered pixel point value of each pixel point and the edge detail value of the pixel point to obtain an enhanced pixel point, and the expression of the enhanced pixel point value of the i-th pixel point in the image is:

[0099] i

[0100] ; ​​​​​​​​​

[0101] wherein, is an enhancement coefficient.

[0102] All pixel points in the image in step S1 are traversed to obtain corresponding enhanced pixel points, and an enhanced image is obtained from all the enhanced pixel points.

[0103] Since the mine card is large in size, the image collector is installed at a high position on the mine card, and small obstacles on the ground in the image account for a very low proportion of pixel points in the image, which is not conducive to the accurate detection of the agent on these small obstacles. The image enhancement method of the present application enhances the contour information of the image by extracting the gradient image of the image and superimposing it on the original image, thereby facilitating the agent to more accurately detect obstacle information.

[0104] It can be understood that the obstacle is a person, a signboard, a fallen stone block or other agents, etc.

[0105] Optionally, the specific steps of obtaining the pixel point coordinate information of the rectangular bounding box of the obstacle in the enhanced image in step S3 are: the coordinate information of the four corner points of the rectangular bounding box: the left upper corner pixel point coordinate (x1, y1), the right upper corner pixel point coordinate (x2, y2), the left lower corner pixel point coordinate (x3, y3) and the right lower corner pixel point coordinate (x4, y4). x 1, y 1), the right upper corner pixel point coordinate (x2, y2), the left lower corner pixel point coordinate (x3, y3) and the right lower corner pixel point coordinate (x4, y4). x 2, y 2), the left lower corner pixel point coordinate (x3, y3) and the right lower corner pixel point coordinate (x4, y4). x 3, y 3), and the right lower corner pixel point coordinate (x4, y4). x 4, y 4).

[0106] Optionally, the specific steps of obtaining the distance between the agent and the obstacle and evaluating the collision risk based on the pixel point coordinate information of the rectangular bounding box of the obstacle in the current frame image in step S4 are:

[0107] S41: obtaining the coordinate (x, y) of the midpoint pixel point of the line connecting the left lower corner pixel point and the right lower corner pixel point in the rectangular bounding box of the obstacle: u , v

[0108] ;

[0109] ;

[0110] S42: obtaining the horizontal distance and the vertical distance between the midpoint pixel point and the obstacle.

[0111] ;

[0112] ;

[0113] ​wherein, h is the height of the image collector from the ground; is the tilt angle of the image collector, which can be measured manually; , is the intrinsic parameter of the image collector;

[0114] Optionally, are the positions of the origin of the image physical coordinate system in the image pixel coordinate system, respectively, are the length units of one pixel point in the image physical coordinate system along the lateral and longitudinal directions, respectively, which can be calibrated by a calibration board.

[0115] S43: based on the tilt angle of the image collector θ and the coordinates of the midpoint pixel point u , v ), the lateral distance x ’’ and the longitudinal distance y ’’ between the obstacle and the agent are obtained.

[0116] If , then:

[0117] ;

[0118] ;

[0119] If , then:

[0120] ;

[0121] ;

[0122] If , then:

[0123] ;

[0124] ;

[0125] wherein width is the lateral pixel resolution of the current frame image.

[0126] It can be understood that the tilt angle of the image collector θThe angle between the normal direction of the agent and the image acquisition direction of the image collector is the angle between the normal direction of the agent and the image acquisition direction of the image collector. The image acquisition direction of the image collector is negative on the left of the normal direction of the agent, and the image acquisition direction of the image collector is positive on the right of the normal direction of the agent. Wherein, the normal of the agent is the perpendicular line perpendicular to the installation surface of the image collector installed on the agent, such as the image collector is installed in front of the agent, the normal is perpendicular to the plane perpendicular to the position of the image collector installed on the right of the agent.

[0127] S44: based on the lateral distance and the longitudinal distance between the obstacle and the agent, the collision risk of the agent is evaluated: the safety distance is set as d 1, and the braking distance of the agent is d 2. When the longitudinal distance between the obstacle and the agent is y ’’> d 1, it is determined that the collision risk level at this time is low; when the longitudinal distance of the obstacle is d 2 y ’’ d 1, it is determined that the collision risk level at this time is medium; when the longitudinal distance of the obstacle is y ’’ d 2, it is determined that the collision risk level at this time is high.

[0128] Optionally, based on the collision risk evaluation result in step S5, the specific steps of adjusting the current frame image code rate parameter using image code rate dynamic decision method are as follows:

[0129] S51, the image code rate parameter includes low code rate threshold M1, medium code rate threshold M2 and high code rate threshold M3;

[0130] S52, adjusting the image code rate parameter based on the current frame image collision risk evaluation result:

[0131] determine whether the collision risk evaluation result is high, if it is high, set the code rate to the high code rate threshold M3; if it is not high, further determine whether the collision risk evaluation result is medium;

[0132] if the collision risk evaluation result is medium, set the code rate to the medium code rate threshold M2; if it is not medium, get the risk evaluation result of the previous frame image;

[0133] determine whether the risk evaluation result of the previous frame image is low; if the risk evaluation result of the previous frame image is low, set the code rate to the low code rate threshold M1; if the previous frame is not low, set the code rate to the medium code rate threshold M2.

[0134] Optionally, if the current frame image is the first frame image, the risk assessment result of the previous frame image is set to be the same as the risk assessment result of the current frame image.

[0135] Optionally, the specific step of encoding the current frame image using the adjusted image code rate parameter in step S6 is:

[0136] Step S61, convert the image format with the ranging result into a color coding format;

[0137] Optionally, the image format with both the obstacle detection category information and the ranging result is converted into a color coding format.

[0138] Optionally, the color coding format is a YUV format.

[0139] Step S62, obtain the image converted in step S61 and the adjusted image code rate parameter in step S5, and dynamically encode the color coding format image to obtain an encoded current frame image;

[0140] Optionally, if the adjusted image code rate parameter or the video stream data is a medium code rate threshold M2 or a high code rate threshold

[0141] Value M3, when dynamically encoding the color coding format image, a medium code rate or a high code rate is used to make the image quality higher than that of a low code rate.

[0142] Step S63, push the encoded current frame image in step S62 to the remote monitoring platform end.

[0143] Optionally, if the encoded current frame image data is encoded as a medium code rate or a high code rate, the image quality will be higher and the picture will be clearer when the display device displays the encoded current frame image data, so as to remind the monitoring personnel to pay attention; further, an alarm is also sent to remind the monitoring personnel.

[0144] Further, when the obstacle detection category information is obtained, the display device also displays the obstacle detection category information.

[0145] Optionally, when the current frame image data is obtained from real-time video stream data or multi-frame image data, each frame image in the real-time video stream data or the multi-frame image data is processed according to the aforementioned current frame image processing method, and then the encoded image data is transmitted to the remote monitoring platform end; the encoded image data or the real-time video composed of multi-frame encoded image data is displayed on the display device.

[0146] Another embodiment of the present application discloses an open-pit mine-oriented intelligent agent remote monitoring system, which is used for unmanned vehicles, unmanned aerial vehicles and intelligent robots and other intelligent agents operating in open-pit mine environment, adopts the aforementioned remote control method to process collected images, and includes an image collector, an image processing unit and a remote monitoring platform arranged on the intelligent agent.

[0147] The image collector is used for collecting environment images or videos when the intelligent agent is running in real time, and transmitting current frame images of the environment images or videos to the image processing unit.

[0148] The image processing unit is used for performing image enhancement processing on the current frame images, performing obstacle detection based on the enhanced images after the enhancement processing, obtaining the detected obstacle category information and the pixel coordinate information of the rectangular bounding box in the images, performing obstacle distance calculation and collision risk assessment and judgment by taking the pixel coordinate information as input, adjusting the image stream code rate parameter by using the image code rate dynamic decision method according to the collision risk assessment result, encoding the images according to the image stream code rate parameter, and transmitting the encoded image data to the remote monitoring platform end.

[0149] The remote monitoring platform is used for receiving the encoded image data and displaying the encoded image data on a terminal display device, and further automatically reminding the monitoring personnel according to the risk assessment result.

[0150] The above is only the preferred specific implementation of the present application, but the protection scope of the present application is not limited to this, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application.

Claims

1. An open-pit mine-oriented agent remote monitoring method, characterized in that, The specific steps are as follows: Obtain the current frame image data of the agent end; Perform image enhancement processing on the current frame image to obtain an enhanced image, and the specific steps are as follows: Filtering out each pixel during image acquisition, the image after noise removal... i The expression for the value of a pixel is: pi ( xi , yi )=( pi ( xi -1, yi -1)+ pi ( xi , yi -1)+ pi ( xi +1, yi -1)+ pi ( xi -1, yi )+ pi ( xi , yi )+ pi ( xi +1, yi )+ pi ( xi -1, yi )+ pi ( xi , yi +1)+ pi ( xi +1, yi +1))*(1 / 9)； in, xi For the first in the image i The x-coordinate of each pixel yi For the first in the image i The ordinate of each pixel; For the first in the image i The pixel value after noise removal from each pixel; Each noise-removed pixel point is converted into a gray pixel point to obtain a gray image, and an expression of a gray pixel point value of a pixel point in the image is as follows: i ​ ； ； wherein, represents the pixel point value of the red color channel of the original image for the i-th pixel point in the image; i represents the pixel point value of the green color channel of the original image for the i-th pixel point in the image; i represents the pixel point value of the blue color channel of the original image for the i-th pixel point in the image; i represents the pixel point value of the gray scale image for the i-th pixel point in the image; i , and are weight coefficients, respectively.​​​​ The edge detail value of each pixel in the grayscale image is obtained using the second-order derivative. i The edge of a pixel The expression of the detail value is: ； wherein, is the second derivative of the grey scale pixel value of the pixel in the image, i is the second derivative of the grey scale pixel value of the pixel in the image, is the second derivative of the grey scale pixel value of the pixel in the image, i is the second derivative of the grey scale pixel value of the pixel in the image, x is the second derivative of the grey scale pixel value of the pixel in the image, is the second derivative of the grey scale pixel value of the pixel in the image, i is the second derivative of the grey scale pixel value of the pixel in the image, y is the second derivative of the grey scale pixel value of the pixel in the image. Superimpose the filtered pixel point value and the edge detail value of the pixel point to obtain an enhanced pixel point. The expression of the enhanced pixel point value of the pixel point in the image is: i ​ ； wherein are enhancement coefficients; Obtain the corresponding enhanced pixel points by traversing all the pixel points in the current frame image, and obtain the current frame enhanced image from all the enhanced pixel points; The obstacle detection is performed based on the current frame enhanced image, and pixel point coordinate information of a rectangular surrounding frame of the obstacle in the current frame image is obtained. The specific steps are as follows: coordinate information of four corner points of the rectangular surrounding frame: left upper corner pixel point coordinate (x x 1, y 1), right upper corner pixel point coordinate (x x 2, y 2), left lower corner pixel point coordinate (x x 3, y 3) and right lower corner pixel point coordinate (x x 4, y 4). Based on the pixel point coordinate information of the rectangular bounding box of the obstacle in the current frame image, the distance between the agent and the obstacle is obtained, and the collision risk is evaluated, and the specific steps are as follows: Obtain the coordinates of the midpoint pixel of the line connecting the left-bottom pixel and the right-bottom pixel in the rectangular bounding box of the obstacle u , v ) : ； ； Obtaining lateral distance of midpoint pixel from obstacle and longitudinal distance ; ； ； wherein, h is the height of the image collector from the ground; is the vertical angle of the image collector, which can be measured by a person; , is the internal parameter of the image collector; Deflection angle based on image collector θ and the coordinates of the midpoint pixel point u , v , the lateral distance x ’’ and longitudinal distance y ’’ of the obstacle and the agent at this time are obtained, and the expressions of the lateral distance x ’’ and longitudinal distance y ’’ are: If then: ； ； If then: ； ； If then: ； ； Wherein, width is the horizontal pixel resolution of the current frame image; Based on the horizontal distance and the vertical distance between the obstacle and the agent, the collision risk of the agent is evaluated; Based on the collision risk evaluation result, the image code rate parameter of the current frame image is adjusted using the image code rate dynamic decision method; The current frame image is encoded using the adjusted image code rate parameter to obtain encoded image data; the encoded image data is transmitted to the remote monitoring platform end.

2. The intelligent agent remote monitoring method of claim 1, wherein, The method for evaluating the collision risk of the intelligent agent is: setting a safety distance as d 1, the braking distance of the intelligent agent as d 2; when the longitudinal distance between the obstacle and the intelligent agent is y ’’> d 1, determining that the collision risk level at this time is low; when the longitudinal distance between the obstacle and the intelligent agent is d 2 y ’’ d 1, determining that the collision risk level at this time is medium; when the longitudinal distance between the obstacle and the intelligent agent is y ’’ d 2, determining that the collision risk level at this time is high.

3. The intelligent agent remote monitoring method of claim 2, wherein, The specific steps of adjusting the current frame image code rate parameter using the image code rate dynamic decision method based on the collision risk evaluation result are as follows: If the collision risk evaluation result is high, set the code rate to a high code rate threshold M3; If it is not high, further judge whether the collision risk evaluation result is medium; if the collision risk evaluation result is medium, set the code rate to a medium code rate threshold M2; If it is not medium, obtain the risk evaluation result of the previous frame image; Judge whether the risk evaluation result of the previous frame image is low; If the risk evaluation result of the previous frame image is low, set the code rate to a low code rate threshold M1; If the previous frame is not low, set the code rate to a medium code rate threshold M2.

4. The intelligent agent remote monitoring method of claim 3, wherein, When adjusting the current frame image code rate parameter using the image code rate dynamic decision method based on the collision risk evaluation result, if the collision risk evaluation result is not medium, if the current frame image is the first frame image, set the risk evaluation result of the previous frame image to be the same as that of the current frame image.

5. The intelligent entity remote monitoring method of claim 2, wherein, The specific steps of encoding the current frame image using the adjusted image code rate parameter are as follows: Convert the image format with the collision risk evaluation result to a color coding format; Use the adjusted image code rate parameter to dynamically encode the image after format conversion to obtain the encoded current frame image; Push the encoded current frame image to the remote monitoring platform end.

6. The intelligent entity remote monitoring method according to any one of claims 1-5, wherein, When detecting the obstacle based on the enhanced image, the category information of the obstacle in the current frame image is also obtained.

7. The intelligent entity remote monitoring method of claim 6, wherein, After the remote monitoring platform end obtains the encoded current frame image, the encoded image data and / or the category information of the obstacle in the current frame image are displayed on the display device.

8. An open-pit mine-oriented agent remote monitoring system, which adopts the remote monitoring method of any one of claims 1-7 to process the collected images, comprising an image collector, an image processing unit and a remote monitoring platform arranged on the agent; characterized in that, The image collector is configured to collect an environment image or video of the agent in real time and transmit a current frame image of the environment image or video to the image processing unit; The image processing unit is configured to perform image enhancement processing on the current frame image, perform obstacle detection based on an enhanced image after the enhancement processing, and acquire pixel coordinate information of a rectangular bounding box of the detected obstacle in the image; The pixel coordinate information is used as input to perform obstacle distance calculation and collision risk assessment and judgment, adjust image stream code rate parameters using an image code rate dynamic decision method according to a collision risk assessment result, encode the image according to the image stream code rate parameters, and transmit encoded image data to a remote monitoring platform; and the remote monitoring platform is configured to receive the encoded image data and display the encoded image data on a terminal display device.

Images