An instrument recognition method for inspection robots with adaptive light source adjustment

Abstract

The present invention provides an instrument recognition method for an inspection robot with adaptive light source adjustment. By intelligently analyzing the content of the captured image, the lighting conditions of the detection target object are determined to ensure that the instrument readings are clear and visible. By identifying the brightness level of the image, it is determined whether there is an excessive dimming or reflection problem. Once these conditions are found, the light source will be automatically triggered or the position of the light source will be adjusted to ensure the clarity of the instrument area. The present invention solves the current situation in which instrument recognition cannot be achieved due to external factors. The light source can be adaptively adjusted to meet the instrument recognition standard without human intervention, which not only increases the recognition accuracy, but also can replace the expensive light-sensing camera by combining the algorithm with the standard binocular camera equipment, effectively reducing the cost of the robot and making the automated inspection robot more economical and efficient.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision technology, and in particular to an inspection robot instrument recognition method with adaptive light source adjustment. Background Art

[0002] In the era of intelligent systems, the adoption of computer vision technology has become a key trend in reducing labor costs and improving operational efficiency. Image recognition technology demonstrates broad application potential in a wide range of life scenarios. Exploring how to integrate computer vision-driven image recognition technology into patrol robot systems to achieve automated, intelligent, and efficient patrol inspections is becoming a cutting-edge research topic. A key area of ​​current research is how to accurately locate instruments within images and analyze their status, such as reading instrument values ​​and determining switch states. Patrol robots equipped with instrument image recognition algorithms are already playing a vital role in unmanned patrol and safety monitoring tasks in locations such as substations and data centers. However, the ability of instruments at the same location to produce dimming and reflections over time, depending on lighting conditions, and other factors presents a current technical challenge. Summary of the Invention

[0003] In view of the shortcomings of the existing technology, the present invention aims to provide an inspection robot instrument recognition method with adaptive light source adjustment.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] A method for recognizing an instrument on an inspection robot with adaptive light source adjustment, characterized by comprising the following steps:

[0006] S1, model training step, collects historical instrument images captured by the inspection robot and forms a data set, and annotates the data set with the annotation information into three categories: reflective, dark light, and normal. The annotated data set is divided into a training set, a validation set, and a test set, and the neural network model is trained using the divided training set, validation set, and test set;

[0007] S2, image acquisition and recognition step, acquiring the inspection image captured by the inspection robot in real time, cropping it to obtain an instrument image, and using the neural network model obtained in step S1 to recognize the instrument image to confirm its image category;

[0008] S3. Performing a preset process based on the identified category of the instrument image and then identifying the content in the instrument image, which specifically includes:

[0009] S31. When the recognized image category is reflection, first identify the light source area in the instrument image, then move the light source out of the instrument image area, and then recognize the content of the instrument image;

[0010] S32: When the image category is dark light, turn on the light source and then perform image category recognition. If the recognition result is reflective light, proceed to step S31; if the recognition result is normal, proceed to step S33;

[0011] S33: If the recognized image category is normal, the content of the instrument image is directly recognized.

[0012] The step S31 of confirming the light source area in the instrument image specifically includes:

[0013] S311, converting the input instrument image into a grayscale image;

[0014] S312, binarizing the grayscale image obtained in step S311 by threshold processing, and dividing the image into reflective areas and non-reflective areas, wherein pixels above the threshold are regarded as potential reflective areas;

[0015] S313, performing an erosion operation on the image binarized in step S312;

[0016] S314, using a contour detection algorithm to determine the contour of the reflective area in the image, and depicting the contour on the original instrument image;

[0017] S315: Using a deep learning model to obtain size information of the cropped instrument image, and based on the size information, calculate the center point of the instrument image;

[0018] S316. Draw two diagonal lines and a horizontal line and a vertical line passing through the center point on the instrument image to evenly divide the instrument image into eight sections. Calculate the area of ​​the reflective region in each section and set an area threshold. When the area of ​​the reflective region in a section is greater than the area threshold, it is considered that there is a light source area in that section.

[0019] Furthermore, the processing of moving the light source out of the instrument image area in step S31 further includes:

[0020] S311', when the light source area is only on one part, the direction of the right-angled side a of the part passing through the center point of the instrument image is used as the moving direction. According to whether the right-angled side a is above or below the center point, the optimal moving route preliminarily obtained is to move the light source upward or downward, and then calculate the distance between the other right-angled side b of the part and the lower boundary of the original inspection image taken by the inspection robot. If the distance is above the set value, the light source is moved according to the obtained optimal moving route, and then continues to shoot and judge until it is judged that the light intensity of the part is less than the threshold; if the distance between the right-angled side b and the lower boundary of the original inspection image taken by the inspection robot is less than the set value, the light source is moved in the opposite direction of the optimal moving route, and then continues to shoot and judge; if the dial leaves the shooting range after the light source moves, the light source is moved to the original position to continue shooting and judging. If the result of judging and moving the light source for three consecutive times is that the dial is out of the shooting range, the instrument is marked as needing manual adjustment;

[0021] S312′: When the light source area is located on two of the parts, find the overlapping edges of the two parts using the expression for the side length of each part, specifically including:

[0022] S3121', if the right-angled sides coincide, and the coincident sides are above or below the center point of the instrument image, the optimal moving route initially obtained is to move the light source upward or downward, and further calculate the distance between the non-coincident right-angled sides a of the two parts and the lower boundary of the original inspection image taken by the inspection robot. If the distance is greater than the set value, the light source is moved according to the obtained optimal moving route, and continues to shoot and judge until the light intensity of the part is less than the threshold; if the distance between the non-coincident right-angled sides a of the two parts and the lower boundary of the original inspection image taken by the inspection robot is less than the set value, the light source is moved in the opposite direction of the optimal moving route and continues to shoot and judge; if the dial leaves the shooting range after moving the light source, the light source is returned to the original position and continues to judge. If the result of moving the light source after five consecutive shots and judgments is that the dial is out of the shooting range, then the point is marked as needing manual adjustment;

[0023] S3122', if the hypotenuse overlaps, when judging the direction of the overlapping edge relative to the center point of the instrument image, the optimal moving route preliminarily obtained is to move the light source in the direction of the overlapping edge relative to the center point of the instrument image, further calculate the angle between the overlapping edge and the x-axis according to the hypotenuse formula, calculate the farthest point of the overlapping edge from the center point of the instrument image, extend the overlapping edge and the original inspection image taken by the inspection robot to intersect at points a and b, if the distance between this point and point b is greater than the set value, move the light source in the direction of the overlapping edge relative to the center point of the instrument image according to the optimal moving route, continue shooting and judging until it is judged that the light intensity of all areas is less than the threshold; if the distance between this point and point b is less than the set value, move the light source downward and continue shooting and judging; if the dial leaves the shooting range after moving the light source, return the light source to the original position and continue shooting and judging. If the result of moving the light source after five consecutive shots and judgments is that the dial leaves the shooting range, mark the point as requiring manual adjustment;

[0024] S313', when three parts are detected at the same time to obtain light source areas, the directions of the three parts are determined by the horizontal line or vertical line through the center point of the instrument image. If the three parts are all above the center point of the instrument image, it is preliminarily concluded that the optimal movement route is to move the light source upward; further calculate the distance between the upper boundary of the instrument image and the lower boundary of the original inspection image taken by the inspection robot. If the distance is greater than the set value, move the camera upward according to the optimal movement route, continue shooting and judging until it is determined that the light intensity of all areas is less than the threshold; if the distance between the upper boundary of the instrument image and the lower boundary of the original inspection image taken by the inspection robot is less than the set value, move the camera downward, and continue shooting and judging. If the dial leaves the shooting range at this time, return the camera to the original position to continue shooting and judging. If the result of moving the camera after five consecutive shots and judgments is that the dial is out of the shooting range, mark the point as requiring manual adjustment;

[0025] S314′, when four parts are detected simultaneously to obtain light source areas, the center point of the instrument image is used as the basis for judgment;

[0026] S3141', through contour analysis, it is determined that the figure enclosed by the four parts is a right triangle, and then it is determined that the image enclosed by the four parts is a right triangle. The optimal movement route is initially derived to move the light source to the upper left; the midline of the enclosed right triangle is calculated according to the hypotenuse formula, and the midline is extended to intersect with the original inspection image taken by the inspection robot at points a and b. If the distance between this point and point b is greater than the set value, the camera is moved to the upper left according to the optimal movement route, and then continues to shoot and judge until it is determined that the light intensity of all areas is less than the threshold; if the distance is less than the set value, the camera is moved downward and continues to judge;

[0027] S3142', if the figure enclosed by the four parts is judged to be a rectangle through contour analysis, then the image enclosed by the four parts is judged to be a rectangle, and is on the left side of the instrument image, then the optimal movement route preliminarily obtained is to move the light source to the right, calculate the distance between the right boundary of the instrument image and the left boundary of the original inspection image taken by the inspection robot, if the distance is above the set value, then move the camera to the right according to the optimal movement route, continue shooting and judging, until it is judged that the light intensity of all areas is less than the threshold; if the distance between the right boundary of the instrument image and the left boundary of the original inspection image taken by the inspection robot is less than the set value, then move the camera to the left and continue shooting and judging; if the dial leaves the shooting range after moving the light source, then return the light source to its original position and continue shooting and judging; if the dial is out of the shooting range after the light source is moved after five consecutive shots and judgments, then mark the point as requiring manual adjustment;

[0028] S315′: If more than five parts have light source areas, calculate the distance between each side boundary of the instrument image and the corresponding side boundary of the original inspection image captured by the inspection robot, and move the light source in the direction with the smallest distance.

[0029] Furthermore, in step S1, the dataset is annotated by Labelme to generate a corresponding txt file, and the annotated dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0030] Furthermore, in step S3.6, the area of ​​the reflective region on each part is calculated using BoundingRect.

[0031] Another object of the present invention is to propose a patrol robot with adaptive light source adjustment, characterized in that the patrol robot and the light source and camera arranged on the patrol robot move synchronously; a memory for storing computer instructions; and a control module for calling and executing the computer instructions stored in the memory to implement any one of the methods in the above technical solutions.

[0032] Another object of the present invention is to provide a computer-readable storage medium, comprising: a memory for storing computer instructions; and a controller for calling and executing the computer instructions stored in the memory to implement any one of the methods in the above technical solutions.

[0033] The beneficial effects of the present invention are as follows: the present invention intelligently analyzes the captured image content to determine the lighting conditions of the target object to ensure that the meter readings are clear and visible. By identifying the brightness level of the image, it is determined whether there is an excessive dimming or reflection problem. Once these conditions are found, the light source will be automatically triggered or the position of the light source will be adjusted to ensure the clarity of the meter area. The present invention solves the current situation in which instrument recognition cannot be recognized due to external factors. The light source can be adaptively adjusted to meet the instrument recognition standard without human intervention, which not only increases the recognition accuracy, but also can replace the expensive light-sensing camera by combining the algorithm with the standard binocular camera equipment, effectively reducing the cost of the robot and making the automated inspection robot more economical and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0035] Figure 1 This is a flow chart of a method for identifying an instrument in an inspection robot according to an embodiment of the present invention;

[0036] Figure 2 is a schematic diagram of dividing the instrument image in an embodiment of the present invention;

[0037] Figure 3 This is a schematic diagram of an embodiment of the present invention in which only one portion has a light source area;

[0038] Figure 4 A schematic diagram of an embodiment of the present invention in which two parts have light source areas and their right-angled sides coincide with each other;

[0039] Figure 5 A schematic diagram of an embodiment of the present invention in which two parts have light source areas and their hypotenuses overlap;

[0040] Figure 6 Schematic diagram of three parts detecting light source areas simultaneously in an embodiment of the present invention;

[0041] Figure 7-10 In the embodiment of the present invention, there are four parts detecting light source areas at the same time, and the light source areas are triangular, and the light source areas are at different positions: upper right, lower left, lower right, and upper left;

[0042] Figure 11-14These are four situations in which, in an embodiment of the present invention, four parts simultaneously detect a light source area, the light source area is rectangular, and the light source area is at different positions, namely, right, left, bottom, and top. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0044] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0045] The present invention will be further described below in conjunction with the accompanying drawings. It should be noted that this embodiment is based on the technical solution and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to this embodiment.

[0046] like Figure 1 As shown, this embodiment provides an inspection robot instrument recognition method with adaptive light source adjustment, comprising the following steps:

[0047] S1. Model training step: collect historical instrument images taken by the inspection robot and form a data set, and annotate the data set with annotation information into three categories: reflective, dark light, and normal. The annotated data set is divided into training set, validation set, and test set, and the neural network model is trained using the divided training set, validation set, and test set.

[0048] In one embodiment, a corresponding txt file can be generated through Labelme annotation, with the annotation information being three categories: reflective, dark light, and normal. The annotated data set is then divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0049] It should be noted that the recognition of whether an image is reflective, dark, or normal by constructing a neural network model can employ known techniques in the field of imaging, and the present invention is not limited thereto. For example, the image category can be recognized by preprocessing the input training, validation, and test data sets, extracting features through convolutional and pooling layers, integrating features through fully connected layers, and classifying the model, and optimizing parameters through labeled data training.

[0050] S2, image acquisition and recognition step, obtains the inspection image captured by the inspection robot in real time, and crops it to obtain an instrument image, and uses the neural network model obtained in step S1 to recognize the instrument image to confirm its image category.

[0051] In one embodiment, the trained neural network model obtained in step S1 is used to identify the inspection images taken by the inspection robot. Preferably, the confidence level is set to 0.8, and only the detection frames with a confidence level greater than or equal to 0.8 are retained. The coordinate information of the detection frame is used to crop the detection frame to obtain the instrument image.

[0052] S3. Performing a preset process based on the identified category of the instrument image and then identifying the content in the instrument image, which specifically includes:

[0053] S31. When the recognized image category is reflection, first identify the light source area in the instrument image, then move the light source out of the instrument image area, and then recognize the content of the instrument image;

[0054] S32: When the image category is dark light, turn on the light source and then perform image category recognition. If the recognition result is reflective light, proceed to step S31; if the recognition result is normal, proceed to step S33;

[0055] S33: If the recognized image category is normal, the content of the instrument image is directly recognized.

[0056] It should be noted that how to identify the content of an image is not the technical content of the present invention, and those skilled in the art can implement it through various possible technical methods, which is not limited here.

[0057] In one embodiment, the step S31 of confirming the light source area in the instrument image specifically includes:

[0058] S311, converting the input instrument image into a grayscale image;

[0059] S312, binarizing the grayscale image obtained in step S311 by threshold processing, and dividing the image into reflective areas and non-reflective areas, wherein pixels above the threshold are regarded as potential reflective areas;

[0060] S313. In order to remove noise from the image and clearly identify the reflective area, an erosion operation is performed on the image binarized in step S312. This morphological operation eliminates unnecessary small bright spots through the defined kernel, thereby purifying the image.

[0061] S314, using a contour detection algorithm to determine the contour of the reflective area in the image, and depicting the contour on the original instrument image, thereby clarifying the boundary of the reflective area;

[0062] S315: In order to further analyze the position and distribution of the reflective area, the neural network model obtained in step S1 is used to obtain the size information of the cropped instrument image, and the center point of the instrument image is calculated based on the size information;

[0063] S316: Draw two diagonal lines and a horizontal line and a vertical line passing through the center point on the instrument image to evenly divide the instrument image into eight parts, providing a clear spatial reference frame for the reflective area. Figure 2 As shown, 1-8 represent eight equally divided parts. BoundingRect is used to calculate the area of ​​the reflective region on each part, and an area threshold is set. When the area of ​​the reflective region on a part is greater than the area threshold, it is considered that there is a light source area on the part.

[0064] In one embodiment, when the area of ​​the reflective region on a certain portion is greater than 20% of the area of ​​the region, it is considered that there is a light source region on the portion.

[0065] In addition, since the light source and the camera usually move synchronously, it is necessary to move the light source out of the instrument image area through an optimal route.

[0066] In a specific embodiment, the processing of moving the light source out of the instrument image area in step S31 further includes:

[0067] S311', when the light source area is only on one part, such as Figure 3 When the part "1" is shown, take the shape of part "1" as an example. Figure 3As shown, the expressions of its two right-angled sides a and b are calculated, and the direction of the right-angled side a passing through the center point of the instrument image is the moving direction. The right-angled side a is above the center point, so the optimal moving route is initially obtained to move the light source upward, and calculate the distance between the right-angled side b and the lower boundary of the original inspection image taken by the inspection robot. If the distance is more than 1.5 times the height of the cropped instrument image, the light source is moved upward according to the obtained optimal moving route, and then the shooting and judgment are continued until it is determined that the light intensity of the part "1" is less than the threshold; if the distance between the right-angled side b and the lower boundary of the original inspection image taken by the inspection robot is less than 1.5 times the height of the cropped instrument image, the light source is moved downward, and then the shooting and judgment are continued; if it is determined that the dial has left the shooting range after moving the light source, the light source is moved to the original position and the judgment is continued. If the result of three consecutive judgments and camera movements is that the dial is out of the shooting range, the point is marked as needing manual adjustment.

[0068] S312′: When the light source area is located on two of the parts, find the overlapping edges of the two parts using the expression for the side length of each part, specifically including:

[0069] S3121', if the right angles coincide (e.g. Figure 4 As shown, the light source area is in parts "1" and "2," and their right-angled sides overlap (and the overlapping sides are above the center point of the meter image). The optimal movement path initially obtained is to move the light source upward. The distance between the non-overlapping right-angled side a of the two parts and the lower boundary of the original inspection image captured by the inspection robot is further calculated. If this distance is at least 1.5 times the height of the meter image, the light source is moved upward according to the obtained optimal movement path. Continue shooting and judging until the illumination intensity of areas 1 and 2 falls below the threshold. If the distance between the non-overlapping right-angled side a of the two parts and the lower boundary of the original inspection image captured by the inspection robot is less than 1.5 times the height of the meter image, the light source is moved downward and the judgment is continued. If the dial leaves the shooting range after moving the light source, the light source is moved back to its original position and the judgment is continued. If the dial is out of the shooting range after five consecutive shots and judgments, the result of moving the camera is that the point needs manual adjustment.

[0070] S3122', if the hypotenuses coincide (e.g. Figure 5As shown, the light source area is in sections "2" and "3" and their hypotenuses overlap. When the overlapping edges are above and to the left of the center point of the instrument image, the optimal movement path is initially determined to be to move the light source to the upper left. The angle between the overlapping edges and the x-axis is further calculated using the hypotenuse formula. The farthest point of the overlapping edges from the center point of the instrument image is calculated. The overlapping edges are then extended to intersect with the original image at points a and b. If the distance between the center point and point b is at least 1.5 times the length of the hypotenuse, the light source is moved to the upper left according to the optimal movement path. Continue shooting and judging until the light intensity in areas 2 and 3 is determined to be less than the threshold. If the distance between this point and point b is less than 1.5 times the hypotenuse, move the light source downward and continue shooting and judging. If the dial leaves the shooting range after moving the light source, move the light source back to its original position and continue shooting and judging. If the dial leaves the shooting range after five consecutive shots and judgments, the point is marked as requiring manual adjustment.

[0071] S313', when three parts are detected to obtain the light source area at the same time (such as Figure 6 As shown, parts "1," "2," and "3" all have light source areas. The directions of these three parts are determined using a horizontal or vertical line through the center point of the meter image. If all three parts are above the center point of the meter image, the optimal movement path is to move the light source upward. The distance between the upper boundary of the meter image and the lower boundary of the original inspection image captured by the inspection robot is further calculated. If this distance is greater than 1.5 times the height of the meter image, the light source is moved upward according to the optimal movement path. Continue capturing and evaluating until the illumination intensity of all areas is determined to be less than a threshold. If the distance between the upper boundary of the meter image and the lower boundary of the original inspection image captured by the inspection robot is less than 1.5 times the height of the meter image, the light source is moved downward, and capturing and evaluating are continued. If the dial leaves the capturing range after moving the light source, the light source is returned to its original position and capturing and evaluating are continued. If the dial remains outside the capturing range after five consecutive captures and evaluations, the point is marked as requiring manual adjustment.

[0072] S314′: When four parts are detected simultaneously to obtain a light source area, the center point of the instrument image is used as a basis for judgment, specifically including:

[0073] S3141', through contour analysis, it is determined that the figure enclosed by the four parts is a right triangle, and the optimal movement route initially obtained is to move the light source to the upper left; the midline of the enclosed right triangle is calculated according to the hypotenuse formula, and the midline is extended to intersect with the original inspection image taken by the inspection robot at points a and b. If the distance between this point and point b is above the set value, the camera is moved to the upper left according to the optimal movement route, and then shooting and judging are continued until it is determined that the light intensity in all areas is less than the threshold; if the distance is less than the set value, the camera is moved downward to continue judging.

[0074] exist Figure 7 In the embodiment shown, parts "1", "2", "3" and "4" are all areas with light sources. At this time, contour analysis is used to determine that the image enclosed by the four parts is a right triangle. The midline of the enclosed right triangle is calculated according to the hypotenuse formula, and the midline is extended to intersect with the original inspection image taken by the inspection robot at points a and b. If the distance between the center point of the disk and the farthest point b is more than 1.5 times the length of the midline of the triangle, the light source is moved to the upper left according to the optimal moving route, and then shooting and judgment are continued until it is determined that the light intensity of all areas is less than the threshold; if the distance is less than 1.5 times the length of the midline of the triangle, the light source is moved to the farthest side of the original image with the dial as the target to continue judgment.

[0075] exist Figure 8 In the embodiment shown, parts "5", "6", "7" and "8" are all areas with light sources. At this time, contour analysis is used to determine that the image enclosed by the four parts is a right triangle. The midline of the enclosed right triangle is calculated according to the hypotenuse formula, and the midline is extended to intersect with the original inspection image taken by the inspection robot at points a and b. If the distance between the center point of the disk and the farthest point b is more than 1.5 times the length of the midline of the triangle, the light source is moved to the upper left according to the optimal moving route, and then shooting and judgment are continued until it is determined that the light intensity of all areas is less than the threshold; if the distance is less than 1.5 times the length of the midline of the triangle, the light source is moved to the farthest side of the original image with the dial as the target to continue judgment.

[0076] exist Figure 9In the embodiment shown, parts "3", "4", "5" and "6" are all areas with light sources. Through contour analysis, it is determined that the image enclosed by the four parts is a right triangle. The midline of the enclosed right triangle is calculated according to the hypotenuse formula, and the midline is extended to intersect with the original inspection image taken by the inspection robot at points a and b. If the distance between the center point of the disk and the farthest point b is more than 1.5 times the length of the midline of the triangle, the light source is moved to the upper left according to the optimal moving route, and then shooting and judgment are continued until it is determined that the light intensity of all areas is less than the threshold; if the distance is less than 1.5 times the length of the midline of the triangle, the light source is moved to the farthest side of the original image with the dial as the target to continue judgment, as shown by the arrow in the figure.

[0077] exist Figure 10 In the embodiment shown, parts "1", "2", "7" and "8" are all areas with light sources. Through contour analysis, it is determined that the image enclosed by the four parts is a right triangle. The midline of the enclosed right triangle is calculated according to the hypotenuse formula, and the midline is extended to intersect with the original inspection image taken by the inspection robot at points a and b. If the distance between the center point of the disk and the farthest point b is more than 1.5 times the length of the midline of the triangle, the light source is moved to the upper left according to the optimal moving route, and then shooting and judgment are continued until it is determined that the light intensity of all areas is less than the threshold; if the distance is less than 1.5 times the length of the midline of the triangle, the light source is moved to the farthest side of the original image with the dial as the target to continue judgment, as shown by the arrow in the figure.

[0078] S3142'. If the figure enclosed by the four parts is judged to be a rectangle through contour analysis, then the image enclosed by the four parts is judged to be a rectangle, and on the left side of the instrument image, then the optimal moving route preliminarily obtained is to move the light source to the right, calculate the distance between the right boundary of the instrument image and the left boundary of the original inspection image taken by the inspection robot, and if the distance is above the set value, move the camera to the right according to the optimal moving route, continue shooting and judging, until it is judged that the light intensity of all areas is less than the threshold; if the distance between the right boundary of the instrument image and the left boundary of the original inspection image taken by the inspection robot is less than the set value, move the camera to the left and continue shooting and judging; if the dial leaves the shooting range after moving the light source, return the light source to its original position and continue shooting and judging. If the dial is out of the shooting range after the light source is moved after five consecutive shots and judgments, then mark the point as requiring manual adjustment.

[0079] exist Figure 11In the illustrated embodiment, sections "2," "3," "4," and "5" are all illuminated areas. Contour analysis determines that the image enclosed by these four sections is a rectangle. The median of the rectangle is calculated and extended to intersect with points a and b, the original inspection image captured by the inspection robot. The furthest point (point b in this image) is calculated based on the center of the instrument panel and points a and b. The light source is then moved toward point a, and imaging and assessment are continued until the illumination intensity in all areas is determined to be below the threshold. If the center of the instrument panel is equidistant from points a and b, the light source is prioritized for rightward movement. If the dial leaves the imaging range, the light source is returned to its original position and imaging and assessment continue. If the dial remains outside the imaging range after five consecutive imaging and assessments, the point is marked as requiring manual adjustment.

[0080] exist Figure 12 In the illustrated embodiment, sections "1," "6," "7," and "8" are all illuminated areas. Contour analysis determines that the image enclosed by these four sections is a rectangle. The median of the rectangle is calculated and extended to intersect with points a and b, the original inspection image captured by the inspection robot. The furthest point (point b in this image) is calculated based on the center of the instrument panel and points a and b. The light source is then moved toward point a, and imaging and assessment are continued until the illumination intensity in all areas is determined to be below the threshold. If the center of the instrument panel is equidistant from points a and b, the light source is preferentially moved to the right. If the dial leaves the imaging range, the light source is returned to its original position and imaging and assessment continue. If the dial remains outside the imaging range after five consecutive imaging and assessments, the point is marked as requiring manual adjustment.

[0081] exist Figure 13 In the illustrated embodiment, sections "4," "5," "6," and "7" are illuminated areas. Contour analysis determines that the image enclosed by these four sections is a rectangle. The median of the rectangle is calculated and extended to intersect with points a and b, the original inspection image captured by the inspection robot. The furthest point (point b in this image) is calculated based on the center of the instrument panel and points a and b. The light source is then moved toward point a, and imaging and assessment are continued until the illumination intensity in all areas is determined to be below the threshold. If the center of the instrument panel is equidistant from points a and b, downward movement is prioritized. If the dial leaves the imaging range, the light source is returned to its original position and imaging and assessment continue. If the dial remains outside the imaging range after five consecutive imaging and assessments, the point is marked as requiring manual adjustment.

[0082] exist Figure 14In the illustrated embodiment, sections "1," "2," "3," and "8" represent areas with light sources. Contour analysis determines that the image enclosed by these four sections is a rectangle. The median of the rectangle is calculated and extended to intersect with points a and b, the original inspection image captured by the inspection robot. The furthest point (point b in this image) is calculated based on the center of the instrument panel and points a and b. The light source is then moved toward point a, and imaging and assessment are continued until the illumination intensity in all areas is determined to be below the threshold. If the center of the instrument panel is equidistant from points a and b, downward movement is prioritized. If the dial leaves the imaging range, the light source is returned to its original position and imaging and assessment continue. If the dial remains outside the imaging range after five consecutive imaging and assessments, the point is marked as requiring manual adjustment.

[0083] S315′: If more than five parts have light source areas, calculate the distance between each side boundary of the instrument image and the corresponding side boundary of the original inspection image captured by the inspection robot, and move the light source in the direction with the smallest distance.

[0084] In addition, in another embodiment of the present disclosure, a patrol robot with adaptive light source adjustment is proposed, which is characterized by the patrol robot and the light source and camera arranged on the patrol robot, and the light source and camera move synchronously; a memory for storing computer instructions; and a control module for calling and executing the computer instructions stored in the memory to implement any one of the methods in the above technical solutions.

[0085] In another embodiment of the present disclosure, a computer-readable storage medium is proposed, including: a memory for storing computer instructions; and a controller for calling and executing the computer instructions stored in the memory to implement any method in the above technical solutions.

[0086] Those skilled in the art can make various corresponding changes and modifications based on the above technical solutions and concepts, and all of these changes and modifications should be included in the scope of protection of the claims of the present invention.

Claims

1. A patrol robot instrument recognition method with adaptive light source adjustment, characterized in that: The following steps are involved: S1, model training step, forming a data set of historical instrument images captured by the inspection robot, annotating the data set into three categories: reflective, dark, and normal, dividing the annotated data set into a training set, a validation set, and a test set for training the neural network model; S2, an image acquisition and recognition step, acquiring the inspection image taken by the inspection robot in real time, cropping it to obtain an instrument image, and using the neural network model obtained in step S1 to recognize the instrument image to confirm its image category; S3. Performing preset processing according to the image category and then identifying the content of the instrument image, specifically including: S31. When the image category is reflective, identify the light source area in the instrument image, move the light source out of the instrument image area, and then recognize the content of the instrument image; S32: When the image category is dark light, turn on the light source and perform image category recognition again. If the recognition result is reflective light, proceed to step S31; if the recognition result is normal, proceed to step S33; S33, when the image category is normal, directly recognize the content of the instrument image; The step S31 of determining the light source area specifically includes: S311, converting the input instrument image into a grayscale image; S312, binarizing the grayscale image using threshold processing to segment the image into a reflective area and a non-reflective area, wherein pixels above the threshold are regarded as potential reflective areas; S313, performing an erosion operation on the binarized image; S314, using a contour detection algorithm to determine the contour of the reflective area in the image, and depicting the contour on the original instrument image; S315. Using a deep learning model to obtain size information of the cropped instrument image, and based on the size information, calculate the center point of the instrument image; S316. Draw two diagonal lines and a horizontal line and a vertical line passing through the center point on the instrument image to evenly divide the instrument image into eight sections; calculate the area of ​​the reflective region in each section, and set an area threshold. When the area of ​​the reflective region in a section is greater than the area threshold, it is considered that the section has a light source area; The movement process in step S31 includes: S311', when the light source area is only on one part, the direction of the right-angled side a of the part passing through the center point of the instrument image is the moving direction. According to whether the right-angled side a is above or below the center point, the optimal moving route is initially obtained to move the light source upward or downward, and then calculate the distance between the other right-angled side b of the part and the lower boundary of the original inspection image taken by the inspection robot. If the distance is above the set value, the light source is moved according to the obtained optimal moving route, and then continues to shoot and judge until it is determined that the light intensity of the part is less than the threshold; if the distance between the right-angled side b and the lower boundary of the original inspection image taken by the inspection robot is less than the set value, the light source is moved in the opposite direction of the optimal moving route, and then continues to shoot and judge; if the dial leaves the shooting range after the light source moves, the light source is moved to the original position and then shoots and judges. If the result of judging and moving the light source for three consecutive times is that the dial is out of the shooting range, the instrument is marked as needing manual adjustment.

2. The method according to claim 1, characterized in that The movement process in step S31 further includes: S312′: When the light source area is located on two of the parts, find the overlapping edges of the two parts using the expression for the side length of each part, specifically including: S3121', if the right-angled sides coincide, and the coincident sides are above or below the center point of the instrument image, the optimal moving route initially obtained is to move the light source upward or downward, and further calculate the distance between the non-coincident right-angled sides a of the two parts and the lower boundary of the original inspection image taken by the inspection robot. If the distance is greater than the set value, the light source is moved according to the obtained optimal moving route, and continues to shoot and judge until the illumination intensity of the two parts is less than the threshold; if the distance between the non-coincident right-angled sides a of the two parts and the lower boundary of the original inspection image taken by the inspection robot is less than the set value, the light source is moved in the opposite direction of the optimal moving route and continues to shoot and judge; if the dial leaves the shooting range after moving the light source, the light source is returned to the original position and continues to judge. If the result of moving the light source after five consecutive shots and judgments is that the dial is out of the shooting range, the instrument is marked as requiring manual adjustment; S3122', if the hypotenuse overlaps, when judging the direction of the overlapping edge relative to the center point of the instrument image, the optimal moving route preliminarily obtained is to move the light source in the direction of the overlapping edge relative to the center point of the instrument image, further calculate the angle between the overlapping edge and the x-axis according to the hypotenuse formula, calculate the farthest point of the overlapping edge from the center point of the instrument image, extend the overlapping edge and the original inspection image taken by the inspection robot to intersect at points a and b, if the distance between point a and point b is greater than the set value, move the light source in the direction of the overlapping edge relative to the center point of the instrument image according to the optimal moving route, continue shooting and judging until it is judged that the illumination intensity of the two parts is less than the threshold; if the distance between point a and point b is less than the set value, move the light source downward and continue shooting and judging; if the dial leaves the shooting range after moving the light source, return the light source to the original position and continue shooting and judging. If the result of moving the light source after five consecutive shots and judgments is that the dial leaves the shooting range, mark the instrument as requiring manual adjustment; S313', when three parts are detected at the same time to obtain light source areas, the directions of the three parts are determined by the horizontal line or vertical line through the center point of the instrument image. If the three parts are all above the center point of the instrument image, it is preliminarily concluded that the optimal movement route is to move the light source upward; further calculate the distance between the upper boundary of the instrument image and the lower boundary of the original inspection image taken by the inspection robot. If the distance is greater than the set value, move the camera upward according to the optimal movement route, continue shooting and judging until it is determined that the light intensity of all areas is less than the threshold; if the distance between the upper boundary of the instrument image and the lower boundary of the original inspection image taken by the inspection robot is less than the set value, move the camera downward, and continue shooting and judging. If the dial leaves the shooting range at this time, return the camera to the original position to continue shooting and judging. If the result of moving the camera after five consecutive shots and judgments is that the dial is out of the shooting range, mark the instrument as requiring manual adjustment; S314′, when four parts are detected simultaneously to obtain light source areas, the center point of the instrument image is used as the basis for judgment; S3141'. If the figure formed by the four parts is determined to be a right triangle through contour analysis, the optimal movement route initially determined is to move the light source to the upper left. The midline of the right triangle is calculated according to the hypotenuse formula. The midline is extended to intersect with the original inspection image taken by the inspection robot at points a and b. If the distance between point a and point b is greater than the set value, the camera is moved to the upper left according to the optimal movement route. Then, the camera continues to shoot and judge until it is determined that the light intensity in all areas is less than the threshold value. If the distance is less than the set value, the camera is moved downward and the judgment is continued. S3142', if it is determined through contour analysis that the figure enclosed by the four parts is a rectangle, then the image enclosed by the four parts is determined to be a rectangle, and is on the left side of the instrument image, then the optimal movement route preliminarily obtained is to move the light source to the right, calculate the distance between the right boundary of the instrument image and the left boundary of the original inspection image taken by the inspection robot, and if the distance is greater than the set value, move the camera to the right according to the optimal movement route, continue shooting and judging, until it is determined that the light intensity of all areas is less than the threshold; if the distance between the right boundary of the instrument image and the left boundary of the original inspection image taken by the inspection robot is less than the set value, move the camera to the left and continue shooting and judging; if the dial leaves the shooting range after moving the light source, return the light source to its original position and continue shooting and judging; if the dial is out of the shooting range after the light source is moved after five consecutive shots and judgments, then mark the instrument as requiring manual adjustment; S315′: If more than five parts have light source areas, calculate the distance between each side boundary of the instrument image and the corresponding side boundary of the original inspection image captured by the inspection robot, and move the light source in the direction with the smallest distance.

3. The method according to claim 2, characterized in that In step S1, Labelme is used to annotate the dataset and generate the corresponding txt file. The annotated dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:

1.

4. The method according to claim 3, characterized in that In step S316, the area of ​​the reflective region on each part is calculated using BoundingRect.

5. An inspection robot with adaptive light source adjustment, characterized in that: An inspection robot and a light source and a camera arranged on the inspection robot, the light source and the camera moving synchronously; a memory for storing computer instructions; and a control module for calling and executing the computer instructions stored in the memory to implement the method according to any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that include: Memory, for storing computer instructions; A controller, configured to call and execute computer instructions stored in the memory to implement the method according to any one of claims 1 to 4.

Images