Robot inspection image searching method and device, electronic equipment and storage medium

Abstract

The invention provides an inspection image searching method and device of a robot, electronic equipment and a storage medium, and relates to the technical field of robot inspection. The inspection image searching method of the robot comprises the following steps: acquiring a preliminary inspection image according to a preset inspection task; wherein the inspection task comprises an initial shooting parameter of the holder shooting equipment and an inspection picture configuration requirement; based on a target detection algorithm, determining pixel coordinate information of an inspection target in the preliminary inspection image; determining a target centering offset degree and a target proportion of the inspection target according to the pixel coordinate information; and determining a target shooting parameter based on the routing inspection picture configuration requirement, the initial shooting parameter, the target centering offset degree and the target proportion, so as to obtain a target routing inspection image corresponding to the routing inspection picture configuration requirement shot according to the target shooting parameter. According to the invention, the target shooting parameter is set through the preliminary inspection image, and then the target inspection image is determined, so that the calibration process before inspection is simplified, and the image quality of the target inspection image is improved.

Description

Technical Field

[0001] The present application relates to the field of robot inspection technology, for example, to a robot inspection and image search method, device, electronic equipment and storage medium. Background Art

[0002] Inspection robots include drones, wheeled robots, quadruped robots, wheeled robots, rail-mounted robots, etc. They are generally equipped with pan-tilt cameras and can inspect various target equipment at preset points, such as primary and secondary equipment in the power field. Before inspection, the inspection points need to be calibrated in advance, that is, the shooting target is adjusted to the center of the picture to facilitate subsequent image recognition or manual inspection. Calibration refers to the parameter setting and calibration of the shooting equipment to ensure that it can accurately capture and analyze the target information.

[0003] In the related art, calibration is usually set up through multiple manual corrections before inspection, and the size of the object photographed in the image during calibration is set to achieve calibration of the inspection points. The calibration process is relatively cumbersome and the efficiency of calibration is low. On the other hand, in the inspection process after the calibration is completed, due to the accuracy of sensors, mechanical structure accuracy, mechanical wear, ground deformation, etc., there are different degrees of deviations between the inspection according to the preset calibration and the actual inspection. Even if the object photographed is well centered during calibration, there may be large deviations during the actual inspection. Therefore, based on the current calibration method, the image quality obtained by the inspection robot during the inspection and image search process is low, which leads to poor inspection results. Summary of the invention

[0004] The present application aims to provide a robot inspection and image searching method, device, electronic equipment and storage medium.

[0005] According to one aspect of the present application, a robot inspection and image searching method is proposed, comprising: obtaining a preliminary inspection image according to a preset inspection task; wherein the inspection task includes initial shooting parameters of a pan-tilt camera and inspection image configuration requirements; determining pixel coordinate information of an inspection target in the preliminary inspection image based on a target detection algorithm; determining a target centering offset and a target proportion of the inspection target based on the pixel coordinate information; determining target shooting parameters based on inspection image configuration requirements, initial shooting parameters, target centering offset and target proportion, so as to obtain a target inspection image corresponding to the inspection image configuration requirements shot according to the target shooting parameters.

[0006] According to some embodiments, based on the inspection picture configuration requirements, initial shooting parameters, target centering offset and target ratio, the target shooting parameters are determined, including: determining the preliminary pan-tilt angle and zoom configuration, and preliminary field of view angle of the pan-tilt shooting device according to the initial shooting parameters; determining the pan-tilt angle adjustment offset of the pan-tilt shooting device based on the target centering offset, the preliminary field of view and a preset adjustment offset determination mode; determining the inspection demand ratio of the inspection task according to the inspection picture configuration requirements; determining the pan-tilt zoom number of the pan-tilt shooting device according to the preliminary pan-tilt angle and zoom configuration, the target ratio and the inspection demand ratio; determining the target shooting parameters according to the initial shooting parameters, the pan-tilt angle adjustment offset and the pan-tilt zoom number.

[0007] According to some embodiments, based on the target centering offset, the preliminary field of view angle and the preset adjustment offset determination mode, the gimbal angle adjustment offset of the gimbal shooting device is determined, including: determining the corresponding directional field of view angle in each direction of the preset direction according to the preliminary field of view angle; determining the corresponding directional pixel offset value in each direction according to the target centering offset; determining the directional angle adjustment offset of the corresponding direction according to the corresponding directional pixel offset value in each direction, the directional field of view angle of the corresponding direction and the adjustment offset determination mode, so as to determine the gimbal angle adjustment offset.

[0008] According to one aspect of the present application, a robot inspection and image searching device is proposed, including: a preliminary inspection image acquisition module, used to acquire a preliminary inspection image according to a preset inspection task; wherein the inspection task includes the initial shooting parameters of the pan-tilt shooting device and the inspection image configuration requirements; a pixel coordinate detection module, used to determine the pixel coordinate information of the inspection target in the preliminary inspection image based on the target detection algorithm; an offset and proportion determination module, used to determine the target centering offset and the target proportion of the inspection target according to the pixel coordinate information; a target inspection image determination module, used to determine the target shooting parameters based on the inspection image configuration requirements, the initial shooting parameters, the target centering offset and the target proportion, so as to determine the target inspection image corresponding to the inspection image configuration requirements shot according to the target shooting parameters.

[0009] According to one aspect of the present application, an electronic device is proposed, which includes: a processor; a memory storing a computer program, and when the computer program is executed by the processor, the processor executes the robot inspection and image searching method as described above.

[0010] According to one aspect of the present application, a non-transitory computer-readable medium is provided, on which readable instructions are stored. When the instructions are executed by a processor, the processor executes the robot inspection and image searching method as described above.

[0011] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application.

[0012] Beneficial effects:

[0013] Through the above-mentioned embodiments provided by the present application, the inspection target can be accurately identified by presetting the inspection task, the initial shooting parameters of the pan-tilt camera, and the coordinated use of the target detection algorithm. The pixel coordinate information of the inspection target in the preliminary inspection image is determined by the target detection algorithm, and then the target centering offset and the target proportion are calculated. This step provides accurate data support for subsequent shooting adjustments. Based on these data, the parameters of the pan-tilt camera can be automatically adjusted to ensure that the target is centered in the final captured image and occupies a suitable proportion, avoiding complex manual operations before the inspection begins, thereby simplifying the calibration process before the inspection. In addition, the image quality can be improved, which is convenient for subsequent analysis and processing. By introducing the inspection picture configuration requirements as the basis for adjusting the shooting settings, it can flexibly adapt to different types of inspection tasks. By intelligently adjusting the shooting settings, the size of the inspection target can also be uniformly and automatically adjusted, further improving the image quality of the target inspection image, and thus improving the inspection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without exceeding the scope of protection required by the present application.

[0015] Figure 1 A flowchart of a robot inspection and image search method provided in an embodiment of the present application;

[0016] Figure 2 A flowchart of an implementation method of step S10 provided in an embodiment of the present application;

[0017] Figure 3 A flowchart of an implementation method of step S12 provided in an embodiment of the present application;

[0018] Figure 4 A flowchart of an implementation method of step S13 provided in an embodiment of the present application;

[0019] Figure 5 A flowchart of an implementation method of step S130 provided in an embodiment of the present application;

[0020] Figure 6 A flowchart of an implementation method of step S131 provided in an embodiment of the present application;

[0021] Figure 7 A flowchart of an implementation method of step S132 provided in an embodiment of the present application;

[0022] Figure 8 A block diagram of a robot inspection and image search device provided in an embodiment of the present application;

[0023] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0025] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0026] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0027] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0028] For specific implementation methods, please refer to the following embodiments.

[0029] Figure 1 This is a flow chart of the inspection and image searching method of the robot provided in the embodiment of the present application. The method of this embodiment can be applied to the inspection control chip of the inspection robot, and the inspection control chip can be applied to the inspection robot. Figure 1 As shown, the method includes: step S10, step S11, step S12 and step S13.

[0030] In step S10, a preliminary inspection image is acquired according to a preset inspection task, wherein the inspection task includes initial shooting parameters of the pan-tilt camera device and inspection image configuration requirements.

[0031] The inspection process may include inspections of multiple inspection points. The inspection tasks preset in this application are used to represent the tasks of a certain inspection point, and the tasks can be sent from the inspection host to the inspection robot. A pan-tilt camera is installed on the inspection robot. The inspection task may include inspection points, initial shooting parameters of the pan-tilt camera, inspection picture configuration requirements, position coordinates of the parking position of the inspection robot, and the posture of the inspection robot. Among them, the initial shooting parameters may include the initial PTZ value and preliminary field of view of the pan-tilt device. The inspection picture configuration requirements may include the expected size ratio and pixel offset of the target corresponding to the inspection point in the image captured by the pan-tilt camera. This requirement can be configured during the engineering configuration phase, or it can be a program configuration fixedly stored in the chip of the inspection robot.

[0032] According to the example embodiment, the inspection robot can stop at the corresponding stop position according to the position coordinates and posture in the inspection task, and then adjust the pan-tilt parameters according to the initial shooting parameters of the pan-tilt shooting device to shoot a preliminary inspection image.

[0033] In step S11, based on the target detection algorithm, the pixel coordinate information of the inspection target in the preliminary inspection image is determined.

[0034] The target monitoring algorithm may include algorithms with image detection and recognition functions such as YOLO, faster-rcnn, rtdetr, ssd, and centernet. The inspection target is the type of target device to be inspected at the current inspection point. The coordinate system corresponding to the preliminary inspection image may be established in advance and the relevant coordinate system information may be stored. The relevant coordinate system information may include the setting method of the coordinate system, the coordinate information of the center point of the preliminary inspection image, and the total number of pixels.

[0035] According to an example embodiment, the preliminary inspection image is processed and analyzed using a target detection algorithm to directly obtain pixel coordinate information of the inspection target, which at least includes corresponding coordinate information of the pixel coordinate point at the upper left corner and the pixel coordinate point at the lower right corner of the inspection target.

[0036] In some implementations, a coordinate system may be established with the pixel point at the upper left corner of the preliminary inspection image as the zero point, and the target detection algorithm may be trained based on the coordinate establishment information so that the target detection algorithm can output the coordinate information of the inspection target.

[0037] In step S12, the target centering offset and the target proportion of the inspection target are determined according to the pixel coordinate information.

[0038] The target center offset can be used to indicate the distance offset between the inspection target and the center point of the preliminary inspection image, and the target proportion can be used to indicate the size ratio of the inspection target captured in the preliminary inspection image.

[0039] According to the example embodiment, the coordinate information of the center point of the preliminary inspection image at the current moment can be determined according to the relevant coordinate system information that is pre-set and stored. By comparing the coordinate information of the center point of the preliminary inspection image with the pixel coordinate information, the target centering offset of the inspection target can be obtained, and the target centering offset can include centering offset pixels in various directions. In some implementations, the pixel coordinate information may include pixel points corresponding to multiple inspection targets. Based on the position of each pixel point, the number of pixels of the inspection target on the image can be roughly determined. By comparing the number of pixels corresponding to the inspection target with the total number of pixels, the target proportion can be obtained.

[0040] In step S13, based on the inspection picture configuration requirements, the target centering offset and the target proportion, the target shooting parameters are determined to obtain the target inspection image corresponding to the inspection picture configuration requirements shot according to the target shooting parameters.

[0041] In this application, the inspection picture configuration requirements include the expected size ratio and the expected offset, while the target center offset and target ratio are the actual offset and actual size ratio of the current inspection task, respectively. Therefore, a comparison can be made to update the shooting settings of the PTZ camera, so that the captured image can meet the inspection picture configuration requirements. The target shooting parameters are the shooting parameters of the PTZ camera when shooting the inspection target this time.

[0042] In some implementations, an algorithm for adjusting the initial shooting parameters of the pan-tilt camera device based on the target centering offset and the target proportion may be pre-set to calculate the target shooting parameters. The target is photographed again based on the target shooting parameters to generate a new inspection image, and the new inspection image is analyzed according to step S11 to determine the pixel coordinate information, thereby obtaining a new target centering offset and a new target proportion, which are then compared with the inspection image configuration requirements until a new target inspection image taken according to a certain new target shooting parameter meets the inspection image configuration requirements, that is, the new target inspection image is the final result of this inspection task.

[0043] This application can accurately identify inspection targets by using preset inspection tasks, initial shooting parameters of the pan-tilt camera, and target detection algorithms. The target detection algorithm is used to determine the pixel coordinate information of the inspection target in the preliminary inspection image, and then the target centering offset and target proportion are calculated. This step provides accurate data support for subsequent shooting adjustments. Based on these data, the parameters of the pan-tilt camera can be automatically adjusted to ensure that the target is centered in the final captured image and occupies an appropriate proportion, avoiding complex manual operations before the inspection begins, thereby simplifying the calibration process before the inspection. In addition, the image quality can be improved to facilitate subsequent analysis and processing. By introducing the inspection picture configuration requirements as the basis for adjusting the shooting settings, it can flexibly adapt to different types of inspection tasks. By intelligently adjusting the shooting settings, the size of the inspection target can also be uniformly and automatically adjusted, further improving the image quality of the target inspection image, and thus improving the inspection effect.

[0044] According to some embodiments, reference Figure 2 In step S10, the process can be specifically implemented through steps S100 and S101.

[0045] In step S100, the corresponding inspection point code is determined according to the inspection task.

[0046] In the present application, the inspection task may include inspection point codes corresponding to different photographed devices and shooting positions corresponding to different photographed devices.

[0047] According to an example embodiment, the corresponding inspection point code may be directly extracted in the inspection task.

[0048] In step S101, a preliminary inspection image is acquired at a shooting position corresponding to the inspection point code according to initial shooting parameters.

[0049] In order for the pan-tilt camera to accurately capture the inspection target corresponding to the inspection task, the inspection robot needs to first move to the corresponding shooting position and then capture the image. The shooting positions for different inspection targets can be pre-set and stored, or the shooting positions corresponding to the inspection point codes can be extracted from the inspection task.

[0050] According to the example embodiment, the inspection point code is extracted from the inspection task, and the shooting position corresponding to the inspection point code is searched. The inspection control chip controls the inspection robot to move to the shooting position, and then controls the pan-tilt camera to shoot the inspection target according to the initial shooting parameters to obtain a preliminary inspection image.

[0051] This application takes pictures at the shooting position corresponding to the inspection point code. After determining the shooting position corresponding to the inspection point code, the image can be collected according to the preset initial shooting parameters, ensuring the accuracy and stability of the shooting position, avoiding problems such as image blur and target offset caused by improper shooting position. At the same time, by accurately locating the shooting position, the details and features of the inspection target can be better displayed, improving the image quality.

[0052] According to some embodiments, reference Figure 3 , step S12 can be specifically implemented through steps S120 and S121.

[0053] In step S120, the pixel offset value of the inspection target relative to the center of the preliminary inspection image in a preset direction and the optional pixel ratio of the preliminary inspection image in the preset direction are determined according to the central pixel coordinates and pixel coordinate information of the center of the preliminary inspection image.

[0054] In the present application, the center of the preliminary inspection image is the central pixel point of the preliminary inspection image, and the preset direction can be the horizontal direction and / or the vertical direction. The pixel offset value can be used to represent the numerical value of each pixel offset in different preset directions. The optional pixel ratio can be used to represent the ratio of each pixel in different preset directions.

[0055] According to an example embodiment, the central pixel coordinates of the center of the preliminary inspection image are compared with the pixel coordinate information to obtain a pixel offset value in a preset direction and an optional pixel ratio of the preliminary inspection image in the preset direction.

[0056] In some implementations, a corresponding rectangular frame may be generated based on the pixel coordinate information of the inspection target, and a pixel offset value between the center of the rectangular frame and the center of the preliminary inspection image may be determined based on the center of the rectangular frame. The rectangular frame needs to completely contain the inspection target.

[0057] In step S121, the pixel offset value is determined as the target center offset degree; and the target ratio is determined in the optional pixel ratios according to the preset ratio selection condition.

[0058] The preset ratio selection condition may be selecting the maximum selectable pixel ratio, or other conditions.

[0059] According to an example embodiment, the pixel offset value is directly determined as the target center offset degree. According to a preset ratio selection condition, an optional pixel ratio that meets the selection condition is determined from the optional pixel ratios as the target ratio.

[0060] This application can accurately evaluate the position deviation of the inspection target in the image by calculating the pixel offset value between the inspection target in the preset direction and the center of the preliminary inspection image, that is, the target centering offset. This step helps the automated inspection system to quickly identify and adjust the camera or imaging device to ensure that the inspection target is always in the center of the image or a preset key position, thereby improving the accuracy and availability of the image data. By calculating the optional pixel ratio of the inspection target in the preliminary inspection image and determining the target proportion based on the preset proportion selection conditions, it can be ensured that the inspection target occupies an appropriate proportion in the image, and the problem of missing information due to the target being too small or ignoring important surrounding environmental information due to the target being too large can be avoided, thereby improving the comprehensiveness and accuracy of image analysis.

[0061] According to some embodiments, reference Figure 4 , step S13 can be implemented through step S130, step S131 and step S132.

[0062] In step S130, the preliminary pan / tilt angle, zoom configuration and preliminary field of view of the pan / tilt shooting device are determined according to the initial shooting parameters; the pan / tilt angle adjustment offset of the pan / tilt shooting device is determined based on the target centering offset, the preliminary field of view angle and the preset adjustment offset determination mode.

[0063] In this application, the preliminary field of view angle is used to indicate the angle range of the received image that is initially set and can be directly read, and is often referred to as the field of view range. The size of the field of view angle determines the field of view of the pan / tilt camera. The larger the field of view angle, the larger the field of view and the smaller the optical magnification. In layman's terms, if the inspection target exceeds this angle, it will not be captured in the lens.

[0064] The initial PTZ angle and zoom configuration is the name of some shooting parameters of the PTZ shooting device, which can be used to indicate the initial setting of the PTZ, representing the PTZ horizontal angle, pitch angle and lens zoom. The preset adjustment offset determination mode can be a preset algorithm or model.

[0065] According to the example embodiment, the preliminary pan-tilt angle and zoom configuration and preliminary field of view angle of the pan-tilt shooting device can be directly extracted from the initial shooting parameters. The target centering offset and the preliminary field of view angle are substituted or input into a preset adjustment offset determination mode to calculate or output the pan-tilt angle adjustment offset of the pan-tilt shooting device.

[0066] In step S131, the inspection demand ratio of the inspection task is determined according to the inspection picture configuration requirements; the pan / tilt zoom factor of the pan / tilt shooting device is determined according to the preliminary pan / tilt angle and zoom configuration, the target ratio and the inspection demand ratio.

[0067] In this application, the inspection demand proportion is used to indicate the expected proportion size, and the target proportion is used to indicate the actual proportion of the inspection target in the preliminary inspection image.

[0068] According to the example embodiment, the inspection demand ratio of the inspection task is directly extracted from the inspection picture configuration requirements. A pan / tilt zoom ratio determination model can be pre-set, and after the preliminary pan / tilt angle and zoom ratio configuration, target ratio and inspection demand ratio are input into the pan / tilt zoom ratio determination model, the pan / tilt zoom ratio of the pan / tilt shooting device is directly output.

[0069] In step S132, target shooting parameters are determined according to the initial shooting parameters, the gimbal angle adjustment offset and the gimbal zoom factor.

[0070] According to an example embodiment, based on the gimbal angle adjustment offset and the gimbal zoom factor, the preliminary field of view angle, preliminary gimbal angle and zoom configuration in the initial shooting parameters are adjusted to obtain target shooting parameters.

[0071] This application calculates the gimbal angle adjustment offset based on the target centering offset, which can ensure that the captured target is centered in the image, effectively avoiding poor image composition or information loss caused by target position offset, thereby improving the accuracy and availability of the captured image. In combination with the inspection image configuration requirements and target proportion, the gimbal zoom of the gimbal shooting equipment is dynamically adjusted to ensure that the captured image meets the specific needs of the inspection task and can fully display the target details, optimize the shooting effect, and improve the efficiency and quality of the inspection work.

[0072] According to some embodiments, reference Figure 5 In step S130, based on the target centering offset, the preliminary field of view angle and the preset adjustment offset determination mode, the gimbal angle adjustment offset of the gimbal shooting device is determined, which can be specifically implemented through steps S1300, S1301 and S1302.

[0073] In step S1300 , the directional viewing angle corresponding to each direction in the preset directions is determined according to the preliminary viewing angle.

[0074] According to an example embodiment, the preliminary field of view may include preliminary horizontal and vertical field of view angles, the preliminary horizontal field of view angle is angH1, and the preliminary vertical field of view angle is angV1. Directly extract the corresponding directional field of view angle in each of the preset directions from the preliminary field of view angle.

[0075] In step S1301, the corresponding directional pixel offset value in each direction is determined according to the target center offset degree.

[0076] In some implementations, the target centering offset may include a directional pixel offset value in the vertical direction and a directional pixel offset value in the horizontal direction. The pixel width of the preliminary inspection image may be w, the height may be h, and the center pixel coordinates of the center of the preliminary inspection image may be (x, y) = (w / 2, h / 2). The coordinates of the center of the rectangular frame corresponding to the inspection target are (x1, y1), and the initial PTZ value in the initial shooting parameters of the pan-tilt camera device is (P1, T1, Z1). Then the directional pixel shift value in the horizontal direction is x1-x, and the directional pixel offset value in the vertical direction is y1-y.

[0077] According to an example embodiment, the directional pixel offset value corresponding to each direction may be directly extracted from the target center offset.

[0078] In step S1302, the direction angle adjustment offset of the corresponding direction is determined according to the corresponding direction pixel offset value in each direction, the direction field angle of the corresponding direction and the adjustment offset determination mode, so as to determine the gimbal angle adjustment offset.

[0079] According to an embodiment, the adjustment offset determination mode may be a preset algorithm, the gimbal angle adjustment offset in the horizontal direction is deltP1, and the corresponding calculation method substituted into the adjustment offset determination mode is:

[0080]

[0081] The vertical gimbal angle adjustment offset is deltT1, and the corresponding calculation method for adjusting the offset determination mode is:

[0082]

[0083] Take deltP1 and deltT1 as a whole and use them as the gimbal angle adjustment offset.

[0084] This application calculates the gimbal angle adjustment offset based on the target centering offset, the preliminary field of view angle, and a preset adjustment offset determination mode, thereby ensuring that the gimbal shooting device is more accurately aligned with the target center during automatic adjustment, thereby improving the accuracy of the captured image.

[0085] According to some embodiments, reference Figure 6 In step S131, the pan / tilt zoom factor of the pan / tilt shooting device is determined according to the preliminary pan / tilt angle and zoom configuration, the target ratio and the inspection demand ratio, which can be specifically achieved through steps S1310 and S1311.

[0086] In step S1310, a preliminary zoom factor is determined according to the preliminary pan / tilt angle and zoom configuration.

[0087] According to an example embodiment, a preliminary zoom factor is extracted from a preliminary pan / tilt angle and zoom configuration.

[0088] In step S1311, the ratio of the inspection demand ratio to the target ratio is multiplied by the initial zoom factor to obtain the pan / tilt zoom factor.

[0089] According to an example embodiment, the pan / tilt zoom ratio may be Z2, the inspection demand ratio is R, and the target ratio is R1. The pan / tilt zoom ratio is calculated by the following formula:

[0090]

[0091] This application determines the pan / tilt magnification of the pan / tilt shooting equipment by combining the preliminary pan / tilt angle and zoom configuration, the target ratio, and the inspection demand ratio, so as to dynamically adjust the shooting magnification, thereby ensuring that the shooting effect can meet the accuracy requirements and efficiently complete the inspection task.

[0092] According to some embodiments, reference Figure 7 Step S132 can be specifically implemented through step S1320, step S1321 and step S1322.

[0093] In step S1320, the directional shooting parameters and directional angle adjustment offset corresponding to each direction in the preset directions are determined according to the initial shooting parameters and the gimbal angle adjustment offset.

[0094] In some implementations, the preset direction may be a horizontal direction and a vertical direction. The initial shooting parameters may include the PTZ value of the last inspection, which may be (P1, T1, Z1), where P1 and T1 are used to represent the horizontal shooting angle and vertical shooting angle of the PTZ shooting device during the last inspection, respectively, the direction shooting parameters are P1 and T1, and the initial zoom factor is Z1.

[0095] The gimbal angle adjustment offset may include the offset that the gimbal shooting device needs to be adjusted in the horizontal direction and the vertical direction compared to P1 and T1.

[0096] According to an exemplary embodiment, a directional shooting parameter corresponding to each direction in the preset directions is extracted from the initial shooting parameter, and a directional angle adjustment offset corresponding to each direction in the preset directions is extracted from the gimbal angle adjustment offset.

[0097] In step S1321, target gimbal angle data of the gimbal shooting device is determined based on the direction shooting parameter and the direction angle adjustment offset.

[0098] According to an example embodiment, the direction shooting parameter is adjusted based on the direction angle adjustment offset to obtain the target gimbal angle data of the gimbal shooting device.

[0099] In some implementations, the preset directions may be horizontal and vertical, and the direction angle adjustment offset may include 10 degrees for horizontal adjustment and 5 degrees for vertical adjustment. Then, the corresponding degrees may be added or subtracted from the horizontal shooting angle and the vertical shooting angle in the direction adjustment parameters to obtain the target gimbal angle data.

[0100] In step S1322, the pan / tilt zoom factor is determined as the target zoom factor of the pan / tilt shooting device; and the target pan / tilt angle data and the target zoom factor are determined as the target shooting parameters.

[0101] According to the embodiment, the pan / tilt zoom ratio is used as the target zoom ratio used by the pan / tilt shooting device to shoot the target inspection image, and the target pan / tilt angle data and the target zoom ratio are used as the target shooting parameters when the pan / tilt shooting device shoots the target inspection image.

[0102] This application accurately determines the target shooting parameters of the gimbal shooting device by combining the initial shooting parameters, the gimbal angle adjustment offset and the gimbal zoom. This includes dual optimization of the gimbal angle and zoom, ensuring that the target can be accurately aligned during the shooting process and the picture can be captured at the best viewing angle, thereby greatly improving the accuracy and effect of the shooting. The shooting parameters of the gimbal shooting device can be dynamically adjusted according to the real-time changes of the shooting environment and the preset shooting requirements. This dynamic adaptability enables the system to flexibly respond to various complex shooting scenes and ensure high-quality shooting effects under different conditions. By automatically calculating and simultaneously adjusting the target gimbal angle data and the target zoom, this technology significantly improves shooting efficiency.

[0103] According to some embodiments, the initial shooting parameters are (P1=85.59, T1=12.10, Z=2.0), the resolution of the gimbal shooting device is 1920*1080, the target detection adopts the YOLO algorithm, and the inspection picture configuration requirements of the meters in the inspection result picture include the inspection demand proportion and the inspection demand offset. The inspection demand offset can be a ±10% pixel tolerance requirement, and the inspection demand proportion can be a [60.3%, 73.%] interval tolerance requirement (that is, the horizontal and vertical pixel offsets do not exceed ±192, ±108), and the proportion size is 67%±10% (that is, [60.3%, 73.%]).

[0104] According to the camera resolution, the center coordinates of the image are (x=960, y=540). The coordinates of the upper left corner of the rectangular box corresponding to the inspection target are (566, 249), the coordinates of the lower right corner are (728, 403), and the center pixel coordinates are (x1=647, y1=326). The inspection image configuration requirements include the inspection demand ratio and the inspection demand offset. The inspection demand offset can be a ±10% pixel tolerance requirement, and the inspection demand ratio can be a [60.3%, 73.%] interval tolerance requirement.

[0105] It can be calculated that the horizontal and vertical pixel offsets of the target centering offset are -313 and -214 respectively, which do not meet the inspection requirement offset. The target proportion R1 = fmax((728-566) / 1920, (403-249) / 1080) = 14.26%, which does not meet the inspection requirement proportion. Therefore, it is necessary to adjust the initial shooting parameters of the PTZ shooting device.

[0106] It can be directly read that the horizontal and vertical field angles of the pan-tilt camera device under the current zoom are angH1=30.31 and angV1=17.32 respectively. Calculate according to the method of this application:

[0107] Horizontal angle adjustment offset of the gimbal:

[0108] deltP1=arctan((x1-x) / (x / tan(angH1 / 2))

[0109] =arctan((647-960) / (960 / tan(30.31 / 2)))=-5.05;

[0110] The vertical angle adjustment offset of the gimbal is T2=15.51, that is, the direction angle adjustment offset is the horizontal adjustment offset -5.05, then the horizontal angle of the gimbal shooting device after adjustment is P2=P1+deltP1=80.54, similarly, the vertical angle 15.51 of the gimbal shooting device after adjustment can be calculated, the gimbal magnification, that is, the target magnification is Z2=67% / R1*Z1=9.4, then the target shooting parameters are (P2=80.20, T2=15.51, Z2=9.4).

[0111] The pan-tilt camera is adjusted to the target shooting parameters to shoot, and a target inspection picture is obtained. Then, the same operations as those for the preliminary inspection image can be performed on the target inspection picture until the obtained target inspection picture meets the inspection picture configuration requirements.

[0112] The following describes an apparatus embodiment of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, reference can be made to the method embodiment of the present application.

[0113] Figure 8 This is a block diagram of a robot inspection and image search device provided in an embodiment of the present application. Figure 8 As shown, the robot inspection and image searching device 800 includes a preliminary inspection image acquisition module 801, a pixel coordinate detection module 802, an offset and proportion determination module 803 and a target inspection image determination module 804.

[0114] The preliminary inspection image acquisition module 801 is used to acquire preliminary inspection images according to a preset inspection task; wherein the inspection task includes initial shooting parameters of the pan-tilt camera device and inspection image configuration requirements;

[0115] The pixel coordinate detection module 802 is used to determine the pixel coordinate information of the inspection target in the preliminary inspection image based on the target detection algorithm;

[0116] The offset and proportion determination module 803 is used to determine the target centering offset and target proportion of the inspection target according to the pixel coordinate information;

[0117] The target inspection image determination module 804 is used to determine the target shooting parameters based on the inspection image configuration requirements, initial shooting parameters, target centering offset and target proportion, so as to obtain the target inspection image corresponding to the inspection image configuration requirements shot according to the target shooting parameters.

[0118] Optionally, when obtaining preliminary inspection images according to a preset inspection task, the preliminary inspection image acquisition module 801 can specifically: determine the corresponding inspection point code according to the inspection task; and obtain the preliminary inspection image according to initial shooting parameters at the shooting position corresponding to the inspection point code.

[0119] Optionally, the offset and proportion determination module 803 can specifically determine the pixel offset value of the inspection target relative to the center of the preliminary inspection image in a preset direction and the optional pixel ratio of the preliminary inspection image in a preset direction based on the central pixel coordinates and pixel coordinate information of the center of the preliminary inspection image; determine the pixel offset value as the target centering offset; and determine the target proportion in the optional pixel ratio according to the preset proportion selection conditions.

[0120] Optionally, when the target inspection image determination module 804 determines the target shooting parameters based on the inspection picture configuration requirements, the target centering offset and the target ratio, it can specifically: determine the preliminary pan-tilt angle, zoom configuration and preliminary field of view angle of the pan-tilt shooting device according to the initial shooting parameters; determine the pan-tilt angle adjustment offset of the pan-tilt shooting device based on the target centering offset, the preliminary field of view and a preset adjustment offset determination mode; determine the inspection demand ratio of the inspection task according to the inspection picture configuration requirements; determine the pan-tilt zoom number of the pan-tilt shooting device according to the preliminary pan-tilt angle and zoom configuration, the target ratio and the inspection demand ratio; determine the target shooting parameters according to the initial shooting parameters, the pan-tilt angle adjustment offset and the pan-tilt zoom number.

[0121] Optionally, when the target inspection image determination module 804 determines the pan-tilt angle adjustment offset of the pan-tilt shooting device based on the target centering offset, the preliminary field of view angle and the preset adjustment offset determination mode, it can specifically: determine the corresponding directional field of view angle in each direction in the preset direction according to the preliminary field of view angle; determine the corresponding directional pixel offset value in each direction according to the target centering offset and the corresponding directional field of view angle in each direction; determine the directional angle adjustment offset of the corresponding direction according to the corresponding directional pixel offset value in each direction, the directional field of view angle of the corresponding direction and the adjustment offset determination mode, and determine the pan-tilt angle adjustment offset.

[0122] Optionally, when the target inspection image determination module 804 determines the pan / tilt magnification of the pan / tilt shooting device based on the preliminary pan / tilt angle and magnification configuration, the target proportion and the inspection demand proportion, it can specifically: determine the preliminary magnification based on the preliminary pan / tilt angle and magnification configuration; multiply the ratio of the inspection demand proportion to the target proportion by the preliminary magnification to obtain the pan / tilt magnification.

[0123] Optionally, when the target inspection image determination module 804 determines the target shooting parameters based on the initial shooting parameters, the pan-tilt angle adjustment offset and the pan-tilt magnification, it can specifically: determine the corresponding directional shooting parameters and directional angle adjustment offset in each direction of the preset directions based on the initial shooting parameters and the pan-tilt angle adjustment offset; determine the target pan-tilt angle data of the pan-tilt shooting device based on the directional shooting parameters and the directional angle adjustment offset; determine the pan-tilt magnification as the target magnification of the pan-tilt shooting device; and determine the target pan-tilt angle data and the target magnification as the target shooting parameters.

[0124] The device performs functions similar to the method provided above. For other functions, please refer to the previous description and will not be repeated here.

[0125] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Fig. 9 As shown, the electronic device 900 of this embodiment may include: a memory 901 and a processor 902 .

[0126] The memory 901 stores a computer program. When the computer program is executed by the processor 902, the processor 902 executes the method in the above embodiment.

[0127] The processor 902 and the memory 901 are connected, for example, via a bus.

[0128] Optionally, the electronic device 900 may further include a transceiver. It should be noted that in actual applications, the number of transceivers is not limited to one, and the structure of the electronic device 900 does not constitute a limitation on the embodiments of the present application.

[0129] The processor 902 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 902 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0130] The bus may include a path to transmit information between the above components. The bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0131] The memory 901 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0132] The memory 901 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 902. The processor 902 is used to execute the application code stored in the memory 901 to implement the contents shown in the above method embodiment.

[0133] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Fig. 9 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0134] The electronic device of this embodiment can be used to execute the method of any of the above embodiments, and its implementation principle and technical effect are similar, which will not be described in detail here.

[0135] The present application also provides a non-transitory computer-readable storage medium having computer-readable instructions stored thereon. When the aforementioned instructions are executed by a processor, the processor executes the method in the above embodiment.

[0136] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a non-transient computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0137] The embodiments of the present application are described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present application. At the same time, changes or deformations made by those skilled in the art based on the ideas of the present application, the specific implementation methods and the scope of application of the present application, all belong to the scope of protection of the present application. In summary, the content of this specification should not be construed as a limitation on the present application.

Claims

1. A robot inspection and image search method, characterized in that: include: According to the preset inspection task, a preliminary inspection image is obtained; wherein the inspection task includes the initial shooting parameters of the pan-tilt camera and the inspection image configuration requirements; Based on the target detection algorithm, determining the pixel coordinate information of the inspection target in the preliminary inspection image; Determining the target centering offset and target proportion of the inspection target according to the pixel coordinate information; Based on the inspection picture configuration requirements, the initial shooting parameters, the target centering offset and the target proportion, target shooting parameters are determined to obtain a target inspection image corresponding to the inspection picture configuration requirements shot according to the target shooting parameters.

2. The method according to claim 1, characterized in that The step of obtaining a preliminary inspection image according to a preset inspection task includes: According to the inspection task, determine the corresponding inspection point code; At the shooting position corresponding to the inspection point code, the preliminary inspection image is acquired according to the initial shooting parameters.

3. The method according to claim 1, characterized in that Determining the target centering offset and target proportion of the inspection target according to the pixel coordinate information includes: Determine, according to the central pixel coordinates of the center of the preliminary inspection image and the pixel coordinate information, a pixel offset value of the inspection target relative to the center of the preliminary inspection image in a preset direction and an optional pixel ratio of the inspection target to the preliminary inspection image in a preset direction; Determining the pixel offset value as the target centering offset; According to a preset ratio selection condition, the target ratio is determined in the optional pixel ratios.

4. The method according to claim 3, characterized in that The determining of the target shooting parameters based on the inspection picture configuration requirements, the initial shooting parameters, the target centering offset and the target proportion includes: Determine the initial pan / tilt angle and zoom configuration and the initial field of view angle of the pan / tilt shooting device according to the initial shooting parameters; Determining a pan / tilt angle adjustment offset of the pan / tilt shooting device based on the target centering offset, the preliminary field of view angle, and a preset adjustment offset determination mode; According to the inspection picture configuration requirements, determine the inspection demand ratio of the inspection task; Determining the pan / tilt zoom ratio of the pan / tilt shooting device according to the preliminary pan / tilt angle and zoom configuration, the target ratio, and the inspection demand ratio; The target shooting parameters are determined according to the initial shooting parameters, the gimbal angle adjustment offset and the gimbal zoom factor.

5. The method according to claim 4, characterized in that The step of determining the gimbal angle adjustment offset of the gimbal shooting device based on the target centering offset, the preliminary field of view angle, and a preset adjustment offset determination mode includes: Determine the directional field of view angle corresponding to each direction in the preset directions according to the preliminary field of view angle; Determine the corresponding directional pixel offset value in each direction according to the target centering offset; According to the corresponding directional pixel offset value in each direction, the directional field of view angle of the corresponding direction and the adjustment offset determination mode, the directional angle adjustment offset of the corresponding direction is determined to determine the pan / tilt angle adjustment offset.

6. The method according to claim 4, characterized in that The determining the pan / tilt zoom factor of the pan / tilt shooting device according to the preliminary pan / tilt angle and zoom configuration, the target ratio and the inspection demand ratio includes: Determining a preliminary zoom factor according to the preliminary pan / tilt angle and zoom configuration; The ratio of the inspection demand ratio to the target ratio is multiplied by the preliminary zoom factor to obtain the pan / tilt zoom factor.

7. The method according to claim 4, characterized in that The step of determining the target shooting parameters according to the initial shooting parameters, the gimbal angle adjustment offset, and the gimbal zoom factor includes: Determine the direction shooting parameters and the direction angle adjustment offset corresponding to each direction in the preset directions according to the initial shooting parameters and the gimbal angle adjustment offset; Determining target gimbal angle data of the gimbal shooting device based on the directional shooting parameter and the directional angle adjustment offset; Determine the pan / tilt zoom factor as the target zoom factor of the pan / tilt shooting device; The target pan / tilt angle data and the target zoom factor are determined as the target shooting parameters.

8. A robot inspection and image search device, characterized in that: include: A preliminary inspection image acquisition module is used to acquire preliminary inspection images according to a preset inspection task; wherein the inspection task includes initial shooting parameters of the pan-tilt camera and inspection image configuration requirements; A pixel coordinate detection module, used to determine the pixel coordinate information of the inspection target in the preliminary inspection image based on the target detection algorithm; An offset and proportion determination module, used to determine the target centering offset and target proportion of the inspection target according to the pixel coordinate information; The target inspection image determination module is used to determine the target shooting parameters based on the inspection picture configuration requirements, the initial shooting parameters, the target centering offset and the target proportion, so as to determine the target inspection image corresponding to the inspection picture configuration requirements according to the target shooting parameters.

9. An electronic device, characterized in that: include: processor; A memory storing a computer program, which, when executed by the processor, enables the processor to execute the robot inspection and image searching method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and when the instructions are executed by a processor, the processor executes the robot inspection and image searching method as described in any one of claims 1-7.

Images