A marine narrow cabin automatic meter reading robot and a control method thereof

Abstract

The application provides a marine narrow cabin automatic meter reading robot and a control method thereof, and relates to the technical field of robot control. The robot comprises: a ship-mounted robot control terminal installed in the cabin; a control terminal cable installed on the wall in the cabin; a relay station installed on the top wall of the cabin; an automatic meter reading robot identification terminal; a five-axis mechanical arm used to adjust the spatial position of the automatic meter reading robot identification terminal; a transmission cable used to power the five-axis mechanical arm and transmit data; a V-shaped guide rail installed on the top of the cabin and used to move the five-axis mechanical arm; and an electric wheel module used to fix the five-axis mechanical arm on the guide rail and change the position of the five-axis mechanical arm. The marine narrow cabin automatic meter reading robot improves the efficiency and accuracy of meter reading, avoids the occupation of space caused by arranging too many devices in the narrow cabin, and makes the entire meter reading system highly integrated with the cabin environment without interference.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot control, in particular to a marine narrow cabin automatic meter reading robot and a control method thereof. BACKGROUND

[0002] The existing marine meter reading method has many limitations in the narrow cabin environment. The current conventional marine meter reading method mainly includes manual meter reading, semi-automatic meter reading and simple mechanical device meter reading in some areas. The disadvantages of the traditional meter reading method are becoming more and more prominent. The current meter reading activities mostly rely on special personnel to enter the narrow cabin regularly for manual meter reading. This method not only consumes a lot of manpower and time cost, but also has low meter reading efficiency, and the accuracy and timeliness of the data are difficult to guarantee, which is difficult to meet the demand of modern ship efficient operation. In addition, due to the special environment of the narrow cabin and the high value of the equipment, equipment damage and data loss often occur, which poses a safety hazard to the ship operation. Therefore, there is an urgent need for an intelligent automatic meter reading robot that can accurately collect narrow cabin equipment data and protect the safety of equipment and data. SUMMARY

[0003] In view of the problem in the prior art that the traditional meter reading method of the marine narrow cabin mostly relies on special personnel to enter the narrow cabin regularly for manual meter reading, which consumes a lot of manpower and time cost, has low meter reading efficiency, and the accuracy and timeliness of the data are difficult to guarantee, which is difficult to meet the demand of modern ship efficient operation, the present application provides a marine narrow cabin automatic meter reading robot and a control method thereof.

[0004] To achieve the above purpose, the technical scheme adopted by the present application is as follows: a marine narrow cabin automatic meter reading robot, the robot comprises:

[0005] A control terminal, the control terminal comprises: a ship-mounted robot control terminal, a control terminal cable and a relay station; wherein the ship-mounted robot control terminal is installed inside the cabin body; the control terminal cable is installed on the wall inside the cabin body; the relay station is installed on the top of the cabin body near the wall;

[0006] An automatic meter reading robot identification terminal;

[0007] A five-axis mechanical arm for adjusting the spatial position of the automatic meter reading robot identification terminal;

[0008] A transmission cable for power supply and data transmission of the five-axis mechanical arm;

[0009] A V-shaped guide rail installed on the top of the cabin body for moving the five-axis mechanical arm;

[0010] An electric wheel module for fixing the five-axis mechanical arm on the guide rail and moving to change the position of the five-axis mechanical arm;

[0011] The automatic meter reading robot recognition terminal is installed at the end of the five-axis mechanical arm.

[0012] Further, the automatic meter reading robot recognition terminal comprises an inertial sensor, a high-precision recognition camera, a laser radar, a torque sensor, and a 4G module. The inertial sensor is used to calculate the current speed and position of the object by detecting the acceleration and rotational speed of the object, combining the initial speed and position, and using integral calculation. The high-precision recognition camera is used to identify the data of the screen and instrument panel in real time, and to accurately identify and convert the read data using image processing technology. The laser radar is used to achieve high-precision ranging and imaging of objects inside the cabin, as well as to determine the movement of personnel within a certain range. The torque sensor is used to sense the interaction force between the automatic meter reading robot and the external environment in real time. The 4G module is used for remote communication and transmission of information from the automatic meter reading robot and the inertial sensor to the remote integrated control center.

[0013] Further, the control terminal is used to transmit device initialization signals, control signals, and electric energy to the relay station through the control terminal cable. The relay station undertakes voltage adjustment and transmits electric energy and signals to the automatic meter reading robot recognition terminal, the transmission cable, and the five-axis mechanical arm.

[0014] After the automatic meter reading robot recognition terminal starts working, it models the cabin space and objects through the laser radar, and calculates the spatial position of the five-axis mechanical arm through the inertial sensor.

[0015] The five-axis mechanical arm is used to drive the electric wheel module to displace according to the spatial position and instruction information, while adjusting its own posture and the orientation position of the end automatic meter reading robot recognition terminal.

[0016] Further, the torque sensor is in working state throughout the initialization stage, and it monitors the interaction force between the automatic meter reading robot and the external environment in real time to prevent collision.

[0017] After the shipboard robot control terminal confirms that it has reached the specified position according to the spatial and position information of the five-axis mechanical arm, the high-precision recognition camera identifies the data of the screen and instrument panel in real time and uploads it to the terminal.

[0018] Further, the control mode of the five-axis mechanical arm includes teaching control mode, remote control mode, and autonomous scanning mode.

[0019] The teaching control mode and the autonomous scanning mode perform bidirectional transmission of instruction information, spatial information, position information, and scanning data through the control terminal cable, the relay station, and the transmission cable.

[0020] The remote control mode transmits instruction information, spatial information, position information and scanning data between the 4G module and the remote control center in both directions; when the 4G network signal is lost, the above information will be transmitted to the shipborne robot control terminal; the remote control mode and the teaching control mode can control the posture of the electric wheel module and the five-axis mechanical arm by manual control;

[0021] In the autonomous scanning mode, the shipborne robot control terminal controls the posture of the electric wheel module and the five-axis mechanical arm according to sensor information and task requirements.

[0022] On the other hand, the application also provides a control method for the above-mentioned automatic meter reading robot for narrow cabin of ship, which comprises:

[0023] Step 1, terminal system initialization, electric wheel module initialization, five-axis mechanical arm posture initialization, after completion, enter step 2;

[0024] Step 2, the five-axis mechanical arm carries out scanning in the cabin and builds a model map with laser radar, restores the initial position after modeling is completed, and activates the inertial sensor, marks the origin, and after completion, enters step 3;

[0025] Step 3, control mode selection is carried out, and after running, enters step 4;

[0026] Step 4, the teaching control mode is selected, and the torque sensor is activated, and after completion, enters step 51;

[0027] Step 51, the system judges whether a control instruction is received, if yes, enters step 61, if not, returns to step 4;

[0028] Step 61, the five-axis mechanical arm starts the electric wheel module according to the instruction to adjust the position, adjusts the posture of the five-axis mechanical arm, and records the spatial position, and after completion, enters step 71;

[0029] Step 71, the torque sensor judges whether an abnormal interaction force appears, if yes, obstacle avoidance is carried out and returns to step 61, if not, enters step 81;

[0030] Step 81, judges whether the control is ended, if not, enters step 61, if yes, enters step 91;

[0031] Step 91, the flow is ended.

[0032] Further, the step 4 selects the autonomous scanning mode, and then enters step 52;

[0033] Step 52, the torque sensor is activated, and after completion, enters step 62;

[0034] Step 62, according to the laser radar modeling map and object position, search for the target, and enter step 72 after completion;

[0035] Step 72, determine whether the target is found, if yes, enter step 82, if no, return to step 62;

[0036] Step 82, compare the target position with the self position, start the electric wheel module to adjust the position, adjust the five-axis mechanical arm posture, and record the space position, and enter step 92 after completion;

[0037] Step 92, the torque sensor judges whether an abnormal interaction force appears, if yes, avoid obstacles and return to step 82, if no, enter step 102;

[0038] Step 102, the shipborne robot control terminal positions according to the data of the laser radar and the inertial sensor, and calls the automatic focusing algorithm to find the best focus point, and enter step 112 after completion;

[0039] Step 112, the shipborne robot control terminal starts the electric wheel module to adjust the position according to the best focus point position, adjusts the five-axis mechanical arm posture, and stops after reaching the best focus point position, and enter step 122 after completion;

[0040] Step 122, the shipborne robot control terminal calls the visual recognition algorithm, and enter step 132 after completion;

[0041] Step 132, the high-precision recognition camera identifies the screen and instrument panel by using the visual recognition algorithm, reads data and uploads, and enter step 142 after completion;

[0042] Step 142, the terminal system judges whether the device data is read and uploaded, if yes, enter step 152, if no, return to step 132;

[0043] Step 152, end running.

[0044] Further, the step 4 selects the remote control mode, and enter step 53;

[0045] Step 53, activate the torque sensor, and enter step 63 after completion;

[0046] Step 63, activate the 4G module, and communicate with the remote integrated control center to shake hands, and enter step 73 after completion;

[0047] Step 73, the remote integrated control center judges whether the automatic meter reading robot device state information and instruction signal are received, if yes, enter step 83, if no, return to step 63;

[0048] Step 83, the remote integrated control center obtains the laser radar and inertial sensor information to obtain the spatial position information of the automatic meter reading robot, and enters step 93 after completion;

[0049] Step 93, the remote integrated control center remotely starts the electric wheel module to adjust the position and adjust the posture of the five-axis mechanical arm, and enters step 103 after completion;

[0050] Step 103, the torque sensor judges whether an abnormal interaction force appears, if yes, obstacle avoidance is performed and the process returns to step 93, and if no, the process enters step 113;

[0051] Step 113, the remote integrated control center positions according to the data of the laser radar and the inertial sensor transmitted by the 4G module, and calls the automatic focusing algorithm to find the best focus point, and enters step 123 after completion;

[0052] Step 123, the remote integrated control center remotely adjusts the position by starting the electric wheel module according to the best focus point position, adjusts the posture of the five-axis mechanical arm, and stops after reaching the best focus point position, and enters step 133 after completion;

[0053] Step 133, the remote integrated control center calls the visual recognition algorithm, and enters step 143 after completion;

[0054] Step 143, the high-precision recognition camera identifies the screen and the instrument panel by using the visual recognition algorithm, reads the data and uploads to the remote integrated control center through the 4G network, and enters step 153 after completion;

[0055] Step 153, the remote integrated control center receives the data transmitted by the 4G network and stores it, and enters step 163 after completion;

[0056] Step 163, judges whether the device data is read and uploaded, if yes, enters step 173, and if no, returns to step 153;

[0057] Step 173, ends the running.

[0058] Further, the automatic focusing algorithm includes the following steps:

[0059] S1, initialize the deep learning model, the sensor and the focusing parameter;

[0060] S2, obtain the current image frame, the IMU data and the laser radar data;

[0061] S3, fuse the sensor data to estimate the relative position and posture;

[0062] S4, determine the focusing search range according to the estimated information;

[0063] S5, input the image into the deep learning model to predict the focus point quality;

[0064] S6, Scan the focus position within the search range according to the current step size;

[0065] S7, acquire the new image and calculate the actual focus quality;

[0066] S8 compares the actual and predicted focus quality and adjusts the step size;

[0067] S9 records the optimal focus position;

[0068] S10, determine whether the stopping condition is met;

[0069] S11 outputs the final focus position and controls the lens focus;

[0070] Specifically, S5 includes:

[0071] The input image is , where H, W, and C are the height, width, and number of channels of the image, respectively;

[0072] After feature extraction using a lightweight CNN model, feature maps are obtained. ;

[0073] in, Let be the eigenvalue at coordinates (k, l) in the feature map; h, w, and c are the height, width, and number of channels of the feature map, respectively. This indicates that the feature map is a dimensional map. A real matrix;

[0074] The feature map is transformed into a one-dimensional vector using a global average pooling layer. , represents a one-dimensional vector obtained by transforming the feature map through a global average pooling layer. The dimension is c. This indicates that the vector is a real vector of length c;

[0075] The formula for global average pooling is shown below:

[0076]

[0077] in, A one-dimensional vector The One element; This represents the total number of pixels in the feature map. Indicates the first feature map All pixel values ​​in each channel Summing and then dividing by the total number of pixels yields the average value for that channel, which is then used as a vector. The One element;

[0078] Finally, the focus quality prediction value is output through the full connection layer and the Sigmoid activation function , which is expressed as:

[0079]

[0080] wherein, represents the focus quality prediction value, ranging from [0, 1], and the value closer to 1 indicates that the current image is clearer; is the Sigmoid activation function, which maps the output of the full connection layer to the interval [0, 1], and the calculation formula is:

[0081]

[0082] W is the weight parameter of the full connection layer, with a dimension of , which is used for linear transformation of the feature vector ; v is a one-dimensional feature vector obtained by global average pooling; b is the bias parameter of the full connection layer, which is used to adjust the baseline offset of the linear transformation.

[0083] Further, the visual recognition algorithm comprises the following steps:

[0084] S1, image acquisition: obtaining the original frame through a high-precision recognition camera , and the frame rate is set to 30 FPS;

[0085] S2, image preprocessing: performing Gaussian filtering on the original frame , histogram equalization H to obtain an enhanced frame :

[0086]

[0087] S3, target detection: taking the enhanced frame as input, and obtaining a candidate frame set :

[0088] ;

[0089] wherein, each candidate frame contains the candidate target region information identified in the target detection;

[0090] represents the center coordinates of in the image coordinate system, which is used to determine the position of the candidate frame in the image; represents the width and height of the candidate frame , which is used to describe the size of the candidate frame; The logistic Sigmoid function used to calculate the bounding box confidence, maps the raw score output by the network to the interval [0, 1], resulting in the probability that the bounding box belongs to the real target. The activation function of Sigmoid. The raw score output by the target detection network for the i-th bounding box, which has not been mapped by the Sigmoid function, has a value range in the real number field, and the confidence is obtained after processing by the Sigmoid function .

[0091] Non-maximum suppression (NMS) is used:

[0092]

[0093] wherein, is the set of bounding boxes that remain after the non-maximum suppression operation, and these bounding boxes are the regions that can accurately correspond to the real target after screening. The function of the non-maximum suppression operation, for multiple bounding boxes that exist for the same target, calculates the intersection-over-union between them, and if the intersection-over-union is greater than a set threshold , the bounding box with the higher confidence is retained and the bounding box with the lower confidence is deleted.

[0094] S4, ROI perspective transformation: if the target box is detected , the region of interest (ROI) containing the target is extracted and represented by . For an ROI with an inclination, perspective transformation is needed for correction, and a homography matrix is obtained through four-point mapping:

[0095]

[0096] wherein, is the homography matrix used to realize the perspective transformation of the two-dimensional image plane, which can convert the inclined ROI to an orthographic view according to the set mapping rule, and make the text region straight. and are matrices constructed from the original four-point coordinates in the ROI and the expected corrected four-point coordinates. is the orthographic view obtained after perspective transformation correction, and the target region in the image is horizontal or straight. represents the perspective transformation function, and the input is the original inclined ROI image and the homography matrix , and the output is the orthographic view corrected after perspective transformation.

[0097] S5, OCR recognition: input the corrected image into the OCR engine​ Obtain the character sequence:

[0098]

[0099] in This represents an optical character recognition (OCR) engine, which receives the corrected image. Output the identified characters and related confidence information;

[0100] S6, Result Analysis and Output: (For...) Perform regularization parsing Output JSON structure:

[0101]

[0102] in, This indicates a regularization parsing operation; This represents the confidence threshold, which is used to filter the identification results during the regularization parsing process.

[0103] Compared with the prior art, the beneficial effects of this application are:

[0104] 1. This application proposes an automated meter reading robot specifically designed for the confined environments of marine cabins. A guide rail, running from bow to stern within the narrow cabin, serves as the robot's sole path of movement. This guide rail is expandable or reconfigurable to adapt to different cabin layouts, enhancing system flexibility. This design makes full use of the limited cabin space, avoiding the space-consuming problems associated with arranging complex tracks or supports in confined environments. The robot moves along the guide rail, flexibly reaching various meter reading locations without interfering with other equipment and personnel within the cabin.

[0105] 2. The automatic meter reading robot proposed in this application is equipped with a five-degree-of-freedom robotic arm. With its flexible multi-axis motion capabilities, it can cover the entire narrow cabin space, easily reaching various instruments and equipment to achieve comprehensive data collection. This design not only improves the efficiency and accuracy of meter reading but also avoids the space encroachment caused by placing too much equipment in the narrow cabin, allowing the entire meter reading system to be highly integrated with the cabin environment without interference.

[0106] 3. This application employs a combination of LiDAR, an inertial measurement unit (IMU), and an autofocus algorithm. The LiDAR provides high-precision 3D point clouds, while the IMU compensates for motion jitter, enabling accurate pose estimation (position + attitude) of the robot within the cabin. The autofocus algorithm adjusts the LiDAR or camera parameters based on the target distance, improving the detection accuracy of distant or small targets.

[0107] 4. The technical solution of the present application combines an automatic focusing algorithm and a visual recognition algorithm, realizes accurate focusing, target detection and data reading of the robot in a narrow cabin environment through multi-sensor fusion (laser radar, IMU, camera) and deep learning technology, and further realizes efficient and accurate operation of the robot in the narrow cabin environment, significantly improves the automation level of narrow cabin inspection, reduces labor cost, and is suitable for narrow spaces or high-precision tasks in the fields of industry, military, ship, etc.

[0108] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific embodiments part. BRIEF DESCRIPTION OF DRAWINGS

[0109] Figure 1 is a schematic structural diagram of a marine narrow cabin automatic meter reading robot according to the present application;

[0110] Figure 2 is a working structural diagram of a marine narrow cabin automatic meter reading robot according to the present application;

[0111] Figure 3 is a working flowchart of a marine narrow cabin automatic meter reading robot according to the present application. DETAILED DESCRIPTION

[0112] In order for those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application, so as to better understand the purpose, characteristics and advantages of the present application. It should be understood that the embodiments shown in the drawings are not a limitation on the scope of the present application, but only to illustrate the essential spirit of the technical solutions of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts should belong to the scope of protection of the present application.

[0113] Unless the context requires otherwise, the words "comprise", "comprising", and variations such as "contain", "containing", and "have" throughout the specification and claims, are to be construed in an open, inclusive sense, i.e., as "including, but not limited to".

[0114] Throughout the specification and claims, reference to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of the phrase "in one embodiment" or "in an embodiment" in various places throughout the specification are not necessarily all referring to the same embodiment. In addition, particular features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0115] As used in this specification and the appended claims, the singular forms“a,”“an,” and“the” include plural referents unless the content clearly dictates otherwise. It should be noted that the term“or” is generally employed in its sense including“and / or,” unless the content clearly dictates otherwise.

[0116] In the following description, for purposes of clarity, directional terms are used to describe the structural and working relationship of the present application, but the terms“front”,“back”,“left”,“right”,“outer”,“inner”,“outward”,“inward”,“up”,“down” and the like should be understood as convenient language and should not be construed as limiting terms.

[0117] The implementation details of the embodiments of the present application are specifically described below with reference to the accompanying drawings. The following details are provided for the convenience of understanding and are not essential for the implementation of the technical solutions.

[0118] In combination Figure 1 The structural schematic diagram of the automatic meter reading robot for narrow cabin of ship, A2 is a ship-mounted robot control terminal, which is installed inside the cabin body A1, and is used for power distribution, control, and processing and storage of sensor data of the automatic meter reading robot mechanical arm; A3 is a control terminal cable, which is installed on the wall inside the cabin body A1; A4 is a relay station, which is installed on the top wall of the cabin body A1, and is used for voltage conversion and data transmission.

[0119] Continuing to refer to Figure 1 , B is the automatic meter reading robot mechanical arm related equipment, B1 is an automatic meter reading robot identification terminal, B1 includes: an identification terminal carrying inertia sensor B1-1, a high-precision identification camera B1-2, a laser radar B1-3, a torque sensor B1-4, and a 4G module B1-5. Among them, the inertia sensor B1-1 is used to calculate the current speed and position of the object by detecting the acceleration and rotational speed of the object, combined with the initial speed and position. The high-precision identification camera B1-2 is used to identify the data of the screen and instrument panel in real time, and the read data is accurately identified and converted and uploaded by using advanced image processing technology. The laser radar B1-3 is used to realize high-precision ranging and imaging of objects in the cabin by emitting laser beams and receiving reflected signals, and can also realize the determination of personnel walking within a certain range. The torque sensor B1-4 is used to realize accurate control and obstacle avoidance function by sensing the interaction force between the automatic meter reading robot and the external environment in real time. The 4G module B1-5 is used for remote communication and transmission of information of the automatic meter reading robot and the sensor to the remote integrated control center.

[0120] B2 is a transmission cable for powering and data transmission of the automatic meter reading robot arm. B3 is a V-shaped guide rail installed on the inside top of the cabin A1 for moving the automatic meter reading robot arm inside the cabin A1. B4 is an electric wheel module for fixing the automatic meter reading robot arm on the guide rail and moving to change the position of the automatic meter reading robot arm. B5 is a five-axis robot arm for adjusting the spatial position of the identification terminal. The automatic meter reading robot identification terminal B1 is installed at the end of the five-axis robot arm B5.

[0121] The above-mentioned automatic meter reading robot realizes efficient movement, accurate positioning and autonomous operation in a narrow cabin through the cooperation of the top rail layout of the cabin, integrated cable, multi-axis robot arm and electric wheel, while considering safety, flexibility and maintainability. It is significantly better than the traditional manual meter reading or fixed sensor scheme, and is especially suitable for power cabin, ship cabin and other space-limited and complex environment scenarios.

[0122] In some embodiments, in combination Figure 2 The working structure diagram of the automatic meter reading robot for narrow cabin of ship, it is need to explain, for the different position of cabin equipment and its function demand, the sensor, data transmission module etc. carried on the automatic meter reading robot of the present application can exist certain adjustment in actual carrying process. Various electrical components of the present application are carried in the empty space in the cabin, which will not affect the task development and normal walking of the operating personnel.

[0123] As shown in Figure 2 The working structure diagram of the automatic meter reading robot for narrow cabin of ship, it is need to explain, for the different position of cabin equipment and its function demand, the sensor, data transmission module etc. carried on the automatic meter reading robot of the present application can exist certain adjustment in actual carrying process. Various electrical components of the present application are carried in the empty space in the cabin, which will not affect the task development and normal walking of the operating personnel.

[0124] P1 is a control terminal placed against the cabin wall, and the control terminal P1 includes a shipboard robot control terminal A2, a control terminal cable A3 and a relay station A4.

[0125] P2 is a moving part installed at the top end of the cabin, including a V-shaped guide rail B3 and an electric wheel module B4.

[0126] P3 is the main part of the automatic meter reading robot, which is installed on the V-shaped guide rail B3. The main part P3 includes an automatic meter reading robot identification terminal B1, a transmission cable B2 and a five-axis robot arm B5.

[0127] The control terminal P1 transmits the device initialization signal, the control signal and the electric energy into the relay station A4 through the control terminal cable A3. The relay station A4 undertakes the work of voltage adjustment, and transmits the electric energy and the electric signal into the automatic meter reading robot main body part P3 through the transmission cable B2. The automatic meter reading robot identification terminal B1 starts to work, and the cabin body map space and the object are modeled through the laser radar B1-3. The inertial sensor B1-1 calculates the spatial position of the automatic meter reading robot mechanical arm. The five-axis mechanical arm B5 drives the electric wheel module B4 to displace according to the spatial position and the instruction information, and adjusts the posture of the five-axis mechanical arm B5 and the orientation position of the automatic meter reading robot identification terminal B1.

[0128] It should be noted that the torque sensor B1-4 is in a working state throughout the whole process after the initialization stage, and monitors the interaction force between the automatic meter reading robot and the external environment in real time to prevent collision. The shipborne robot control terminal A2 confirms the arrival at the specified position according to the spatial and position information of the robot mechanical arm, and the high-precision identification camera B1-2 identifies the data of the screen and the instrument panel in real time and uploads them to the terminal.

[0129] In some embodiments, the application also provides three automatic meter reading robot mechanical arm control modes: teaching control, remote control and autonomous scanning. The three control modes are different in transmission of signals and data.

[0130] The teaching control mode and the autonomous scanning mode perform bidirectional transmission of instruction information, spatial information, position information and scanning data through the control terminal cable A3, the relay station A4 and the transmission cable B2.

[0131] The remote control mode performs bidirectional transmission of instruction information, spatial information, position information and scanning data through the 4G module and the remote integrated control center. When the 4G network signal is missing, the above information is transmitted to the shipborne robot control terminal A2. The posture of the electric wheel module B4 and the five-axis mechanical arm B5 can be manually controlled in the remote control mode and the teaching control mode.

[0132] In the autonomous scanning mode, the posture of the electric wheel module B4 and the five-axis mechanical arm B5 is autonomously controlled by the shipborne robot control terminal A2 according to the sensor information and the task requirements.

[0133] Next, based on the above-mentioned automatic meter reading robot for narrow cabin of ship, as shown in the following figure, Figure 3 the application also provides a work flow chart of the automatic meter reading robot for narrow cabin of ship. In order to embody its functionality, the working process is introduced in the setting of overall module, which includes the above-mentioned teaching control, remote control and autonomous scanning mode. The steps are as follows:

[0134] Step 1, terminal system initialization, electric wheel module initialization, robot arm posture initialization, after completion, enter step 2;

[0135] Step 2, the mechanical arm carries the laser radar to scan the cabin and models the map, after modeling, it returns to the initial position, and activates the inertial sensor, marks the origin, after completion, it enters step 3;

[0136] Step 3, control mode selection is performed, after running, it enters step 4 (steps after step 4 are sibling steps, independent of each other without interference);

[0137] Step 4, if the teaching control mode is selected, the torque sensor will be activated, after completion, it enters step 51; if the autonomous scanning mode is selected, it enters step 52; if the remote control mode is selected, it enters step 53;

[0138] Step 51, the system judges whether a control instruction is received, if yes, it enters step 61, if not, it returns to step 4;

[0139] Step 61, the robot arm starts the electric wheel module to adjust the position according to the instruction, adjusts the robot arm posture, and records the spatial position. After completion, it enters step 71;

[0140] Step 71, the torque sensor judges whether an abnormal interaction force appears, if yes, it avoids obstacles and returns to step 61, if not, it enters step 81;

[0141] Step 81, it judges whether the control is ended, if not, it enters step 61, if yes, it enters step 91;

[0142] Step 91, the flow is ended.

[0143] Step 52, the torque sensor is activated, after completion, it enters step 62;

[0144] Step 62, according to the map modeled by the laser radar and the object position, the task target is searched, after completion, it enters step 72;

[0145] Step 72, it judges whether the target is searched, if yes, it enters step 82, if not, it returns to step 62;

[0146] Step 82, the target position is compared with the own position, the electric wheel module is started to adjust the position, the robot arm posture is adjusted, and the spatial position is recorded, after completion, it enters step 92;

[0147] Step 92, the torque sensor judges whether an abnormal interaction force appears, if yes, it avoids obstacles and returns to step 82, if not, it enters step 102;

[0148] Step 102, the shipborne robot control terminal locates according to the data of the laser radar and the inertial sensor, and calls an automatic focusing algorithm to find the best focus point. After completion, it enters step 112;

[0149] Step 112, the shipborne robot control terminal starts the electric wheel module to automatically adjust the position according to the position of the best focus point, adjusts the posture of the robot mechanical arm, and stops after reaching the position of the best focus point. After completion, it enters step 122;

[0150] Step 122, the shipborne robot control terminal calls a visual recognition algorithm, and after completion, it enters step 132;

[0151] Step 132, the high-precision recognition camera uses a visual recognition algorithm to identify the screen and the instrument panel, reads data, and uploads. After completion, it enters step 142;

[0152] Step 142, the terminal system judges whether the device data has been read and uploaded, and if so, it enters step 152, and if not, it returns to step 132.

[0153] Step 152, the running is ended.

[0154] Step 53, the torque sensor is activated, and after completion, it enters step 63;

[0155] Step 63, the 4G module is activated, and a handshake is performed with the remote integrated control center for communication. After completion, it enters step 73;

[0156] Step 73, the remote integrated control center judges whether the robot device state information and the instruction signal have been received, and if so, it enters step 83, and if not, it returns to step 63;

[0157] Step 83, the remote integrated control center obtains the laser radar and inertial sensor information to obtain the spatial position information of the robot. After completion, it enters step 93;

[0158] Step 93, the remote integrated control center remotely controls the electric wheel module to adjust the position and adjust the posture of the robot mechanical arm. After completion, it enters step 103;

[0159] Step 103, the torque sensor judges whether an abnormal interaction force appears, and if so, it performs obstacle avoidance and returns to step 93, and if not, it enters step 113;

[0160] Step 113, the remote integrated control center locates according to the data of the laser radar and the inertial sensor transmitted by the 4G module, and calls an automatic focusing algorithm to find the best focus point. After completion, it enters step 123;

[0161] Step 123, the remote integrated control center activates the electric wheel module remote adjustment position based on the best focus position, adjusts the robot mechanical arm posture, and stops after reaching the best focus position. After completion, enter step 133;

[0162] Step 133, the remote integrated control center calls the visual recognition algorithm, and after completion, enter step 143;

[0163] Step 143, the high-precision recognition camera uses the visual recognition algorithm to identify the screen and instrument panel, reads the data and uploads it to the remote integrated control center through the 4G network, and after completion, enter step 153;

[0164] Step 153, the remote integrated control center receives the data transmitted by the 4G network and stores it, and after completion, enter step 163;

[0165] Step 163, determine whether the device data has been read and uploaded, if yes, enter step 173, if no, return to step 153;

[0166] Step 173, end running.

[0167] The control flow of the above automatic meter reading robot, after system initialization, constructs a cabin map based on laser radar and inertial sensor and marks the origin, providing a spatial reference for subsequent tasks; supports three independent modes of teaching control, autonomous scanning, and remote control, respectively through torque sensor interaction teaching control, guiding autonomous scanning, and 4G module to realize remote control, covering the full scene demand from manual guidance to full autonomous operation; fusion of laser radar positioning, inertial sensor posture compensation and automatic focusing algorithm, ensures the mechanical arm to adjust to the best observation position accurately, combined with the visual recognition algorithm to realize high-precision reading and uploading of instrument panel data; at the same time, the torque sensor monitors the abnormal interaction force in real time, triggers the obstacle avoidance mechanism to ensure safety, ultimately significantly improves the efficiency and accuracy of cabin inspection, equipment monitoring and other tasks, and reduces the risk of human intervention.

[0168] In the working process of the above automatic meter reading robot, a high-performance visual recognition algorithm is used to identify the data of the screen and instrument panel using a high-precision recognition camera B1-2, and upload after reading. The present application also has a high-performance automatic focusing algorithm, which relies on an inertial sensor B1-1, a high-precision recognition camera B1-2, and a torque sensor B1-4 to automatically calculate the best focus position and drive the electric wheel module B4 to change position and adjust the five-axis mechanical arm B5 posture to reach the best focus position.

[0169] In some embodiments, the automatic focusing algorithm includes the following steps:

[0170] Step 1, initialize the deep learning model, sensor and focusing parameters;

[0171] Step 2, obtain the current image frame, IMU data and lidar data;

[0172] Step 3, fuse sensor data to estimate relative position and pose: fuse the IMU data and lidar data using Kalman filtering to estimate the relative position and pose of the automatic meter reading robot and the target device;

[0173] Step 4, determine the focused search range according to the estimated information: determine the starting position range of the focused search according to the estimated relative position and pose;

[0174] Step 5, input the image into the deep learning model to predict the focus quality: input the current image frame into the lightweight CNN model to obtain the focus quality prediction value;

[0175] Step 6, scan the focused position within the search range by the current step;

[0176] Step 7, obtain a new image and calculate the actual focus quality;

[0177] Step 8, compare the actual and predicted focus quality and adjust the step size;

[0178] Step 9, record the best focused position;

[0179] Step 10, judge whether the stop condition is met;

[0180] Step 11, output the final focused position and control the lens focus.

[0181] Further, the above step 1, initializing the deep learning model, sensors and focusing parameters, includes: loading a pre-trained lightweight CNN model; initializing the IMU and lidar data receivers; setting the initial focus position, maximum search range, initial step size and threshold.

[0182] Further, the above step 5, inputting the image into the deep learning model to predict the focus quality: inputting the current image frame into the lightweight CNN model to obtain the focus quality prediction value, specifically including:

[0183] Let the input image be , where H, W and C are the height, width and channel number of the image, respectively.

[0184] After feature extraction by the lightweight CNN model, a feature map is obtained. Wherein, is the feature value at coordinate (k, l) in the feature map; h, w and c are the height, width and channel number of the feature map (usually h < H and w < W are the dimension reduction results after convolution operation, indicates that the feature map is a real matrix with dimension .

[0185] The feature map is converted into a one-dimensional vector by a global average pooling layer , which represents the one-dimensional vector converted by the global average pooling layer. is the number of channels of the feature map, so the dimension of the vector v is c, , which represents that the vector is a real number vector of length c.

[0186] The calculation formula of global average pooling is as follows:

[0187]

[0188] wherein, is the kth element of the one-dimensional vector v; is the total number of pixels of the feature map; , which represents the sum of all pixel values in the kth channel of the feature map (coordinates (i, j)), and then divided by the total number of pixels to obtain the average value of the channel as the kth element of the vector v.

[0189] Finally, the focus quality prediction value is output by a fully connected layer and a Sigmoid activation function , which is represented as:

[0190]

[0191] wherein, represents the focus quality prediction value, ranging from 0 to 1, and the value closer to 1 indicates that the current image is clearer. is the Sigmoid activation function, which maps the output of the fully connected layer to the interval [0, 1], and the calculation formula is:

[0192]

[0193] W is the weight parameter of the fully connected layer, with a dimension of (c is the dimension of the feature vector v), which is used for linear transformation of the feature vector v; b is the bias parameter of the fully connected layer, which is used to adjust the baseline offset of the linear transformation.

[0194] In some embodiments, the above visual recognition algorithm includes the following steps:

[0195] S1, image acquisition: obtaining the original frame by a high-precision recognition camera ; the frame rate is set to 30 FPS;

[0196] S2, image preprocessing: performing Gaussian filtering on the original frame ​histogram equalization obtain an enhanced frame :

[0197]

[0198] S3, target detection: take the enhanced frame as input, and obtain a candidate frame set through a lightweight YOLO network :

[0199] ;

[0200] wherein each candidate frame contains the region information of a possible target identified in the target detection.

[0201] represents the center coordinates in the image coordinate system, and is used to determine the position of the candidate frame in the image. represents the width and height of the candidate frame , and is used to describe the size of the candidate frame. is a logistic (Sigmoid) function used to calculate the confidence of the candidate frame, which maps the original score output by the network to the interval [0, 1] to obtain the probability that the candidate frame belongs to the real target. is the activation function of Sigmoid. is the original score output by the target detection network for the i-th candidate frame, which has not been mapped through the Sigmoid function, and its numerical range is generally the real number domain. After processing through the Sigmoid function, the confidence .

[0202] Non-maximum suppression (NMS) is adopted:

[0203]

[0204] wherein refers to the candidate frame set finally retained after the non-maximum suppression (NMS) operation, and these candidate frames are the regions most likely to accurately correspond to the real target after screening. is a function representing the non-maximum suppression operation, which is a key operation in the post-processing stage of target detection. The basic idea is: for multiple candidate frames that may exist for the same target, calculate their intersection-over-union, and if the intersection-over-union is greater than a set threshold , retain the candidate frame with high confidence and delete the candidate frame with low confidence.

[0205] S4, ROI perspective transformation: if the target frame is detected ​​, extract the region of interest, ROI, containing the target, denoted as For the ROI with tilt, perspective transformation is needed to correct, the homography matrix is obtained by four-point mapping:

[0206]

[0207] where, is the homography matrix, for example, a 3x3 matrix, which realizes the perspective transformation of the two-dimensional image plane, and can convert the tilted ROI to an orthographic view according to the set mapping rules, so that the text area is aligned. and are the matrices constructed by the "original four-point coordinates" and "expected corrected four-point coordinates" in the ROI. is the orthographic view obtained after perspective transformation correction, the target area (such as text) in the image is horizontal / orthographic, which can improve the OCR recognition effect. W' represents the perspective transformation function, the input is the original tilted ROI image and the homography matrix , and the output is the orthographic view corrected by perspective transformation.

[0208] S5, OCR recognition: input the corrected image into the OCR engine to get the character sequence:

[0209]

[0210] where, represents the OCR (Optical Character Recognition) engine, which is an algorithm or tool that performs character recognition tasks, and it receives the corrected image and outputs the recognized characters and related confidence information.

[0211] S6, result analysis and output: regularize and analyze , output the JSON structure:

[0212]

[0213] where, represents the regularized analysis operation, which is a process of processing the character sequence T obtained by OCR according to certain rules (regular expressions, etc.), the purpose is to extract effective information, standardize the format, etc., to facilitate subsequent use. ​A confidence threshold value is used to filter the recognition results in the regular analysis process. For example, when the confidence of a character is greater than or equal to 0.8, the character recognition result is considered valid and participates in the final analysis output; characters below the threshold value will be filtered or marked to improve the reliability of the output result.

[0214] The application provides a marine narrow cabin automatic meter reading robot and a control method thereof. A guide rail from a cabin head to a cabin tail is installed in the narrow cabin as the only path for the robot to move. This design makes full use of the limited space in the cabin and avoids the space occupation problem caused by arranging complex tracks or supports in a narrow environment. The robot moves along the guide rail and can flexibly reach each position where meter reading is required without interfering with other equipment and personnel in the cabin. Meanwhile, the robot is equipped with a five-degree-of-freedom mechanical arm, which can cover the entire cabin space and easily access various instruments and equipment to realize comprehensive data collection by virtue of its flexible multi-axis motion capability. In this process, the application also combines an automatic focusing algorithm and a visual recognition algorithm to realize precise focusing, target detection and data reading of the robot in the narrow cabin environment through multi-sensor fusion and deep learning technology. This design not only improves the efficiency and accuracy of meter reading, but also avoids the space occupation caused by arranging too many devices in the narrow cabin, so that the entire meter reading system and the cabin environment are highly integrated and do not interfere with each other.

[0215] Although the present application has been described in detail with reference to the preferred embodiments, the application is not limited to the preferred embodiments. Any modification or replacement of the embodiments of the present application made by those skilled in the art without departing from the spirit and essence of the present application shall fall within the scope of the present application or any modification or replacement made by those skilled in the art within the technical range disclosed by the present application shall fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A marine narrow compartment automatic meter reading robot, characterized by, The robot comprises: a control terminal comprising a ship-mounted robot control terminal, a control terminal cable and a relay station; wherein the ship-mounted robot control terminal is installed inside the cabin; the control terminal cable is installed on the inside wall of the cabin; and the relay station is installed on the inside top wall of the cabin; an automatic meter reading robot recognition terminal; a five-axis mechanical arm for adjusting the spatial position of the automatic meter reading robot recognition terminal; a transmission cable for supplying power to the five-axis mechanical arm and for data transmission; a V-shaped guide rail installed on the inside top of the cabin for moving the five-axis mechanical arm; an electric wheel module for fixing the five-axis mechanical arm on the guide rail and moving to change the position of the five-axis mechanical arm; wherein the automatic meter reading robot recognition terminal is installed at the end of the five-axis mechanical arm; the automatic meter reading robot recognition terminal comprises an inertial sensor, a high-precision recognition camera, a laser radar, a torque sensor and a 4G module; the inertial sensor is used to calculate the current speed and position of an object by detecting the acceleration and rotational speed of the object, combining the initial speed and position, and using integration; the high-precision recognition camera is used to recognize the data on the screen and instrument panel in real time, and to accurately identify and convert the read data using image processing technology; the laser radar is used to achieve high-precision ranging and imaging of objects inside the cabin by emitting a laser beam and receiving a reflected signal, and to determine the movement of personnel within a certain range; the torque sensor is used to sense the interaction force between the automatic meter reading robot and the external environment in real time; and the 4G module is used for remote communication and transmission of information from the automatic meter reading robot and the inertial sensor to the remote integrated control center; the control terminal is used to transmit device initialization signals, control electrical signals and electrical energy into the relay station through the control terminal cable; the relay station undertakes voltage adjustment work and transmits electrical energy and electrical signals into the automatic meter reading robot recognition terminal, the transmission cable and the five-axis mechanical arm through the transmission cable; after the automatic meter reading robot recognition terminal starts working, the laser radar is used to model the cabin space and objects, and the inertial sensor is used to calculate the spatial position of the five-axis mechanical arm; the five-axis mechanical arm is used to drive the electric wheel module to displace according to the spatial position and instruction information, while adjusting its own attitude and the orientation position of the end automatic meter reading robot recognition terminal.

2. The robot of claim 1, wherein, The torque sensor is in a working state throughout the initialization stage, and is used to monitor the interaction force between the automatic meter reading robot and the external environment in real time to prevent collision; after the ship-mounted robot control terminal confirms that the five-axis mechanical arm has reached the specified position according to the spatial and position information of the five-axis mechanical arm, the high-precision recognition camera recognizes the data on the screen and instrument panel in real time and uploads it to the terminal.

3. The robot of claim 1, wherein, The control mode of the five-axis mechanical arm comprises a teaching control mode, a remote control mode and an autonomous scanning mode; wherein the teaching control mode and the autonomous scanning mode perform bidirectional transmission of instruction information, spatial information, position information and scanning data through the control terminal cable, the relay station and the transmission cable. The remote control mode transmits instruction information, spatial information, position information and scanning data between the remote control center and the 4G module; when the 4G network signal is lost, the above information is transmitted to the shipborne robot control terminal; the remote control mode and the teaching control mode can control the posture of the electric wheel module and the five-axis mechanical arm by manual control; In the autonomous scanning mode, the shipborne robot control terminal controls the posture of the electric wheel module and the five-axis mechanical arm according to sensor information and task requirements.

4. A control method for a narrow-cabin marine automatic meter reading robot according to any one of claims 1 to 3, characterized in that, The method comprises: Step 1, terminal system initialization, electric wheel module initialization, five-axis mechanical arm posture initialization, after completion, enter step 2; Step 2, the five-axis mechanical arm carries the laser radar to scan the cabin and model the map, after modeling, restore the initial position, and activate the inertial sensor, mark the origin, after completion, enter step 3; Step 3, control mode selection, after running, enter step 4; Step 4, select the teaching control mode, activate the torque sensor, after completion, enter step 51; Step 51, the system judges whether a control instruction is received, if yes, enter step 61, if not, return to step 4; Step 61, the five-axis mechanical arm starts the electric wheel module to adjust the position according to the instruction, adjusts the posture of the five-axis mechanical arm, and records the spatial position, after completion, enter step 71; Step 71, the torque sensor judges whether an abnormal interaction force appears, if yes, obstacle avoidance is performed and returns to step 61, if not, enter step 81; Step 81, judge whether the control is ended, if not, enter step 61, if yes, enter step 91; Step 91, end the process.

5. The method of claim 4, wherein, Step 4 selects the autonomous scanning mode, then enter step 52; Step 52, activate the torque sensor, after completion, enter step 62; Step 62, search the task target according to the map modeled by the laser radar and the object position, after completion, enter step 72; Step 72, judge whether the target is found, if yes, enter step 82, if not, return to step 62; Step 82, compare the target position with the own position, start the electric wheel module to adjust the position, adjust the posture of the five-axis mechanical arm, and record the spatial position, after completion, enter step 92; Step 92, the torque sensor judges whether an abnormal interaction force appears, if yes, obstacle avoidance is performed and returns to step 82, if not, enter step 102; Step 102, the shipborne robot control terminal positions according to the data of the laser radar and the inertial sensor, and calls the automatic focusing algorithm to find the best focus point, after completion, enter step 112; Step 112, the shipborne robot control terminal starts the electric wheel module to adjust the position according to the best focus point position, adjusts the posture of the five-axis mechanical arm, stops after reaching the best focus point position, after completion, enter step 122; Step 122, the shipborne robot control terminal calls the visual recognition algorithm, after completion, enter step 132; Step 132, the high-precision recognition camera identifies the screen and instrument panel by the visual recognition algorithm, reads data and uploads, after completion, enter step 142; Step 142, the terminal system judges whether the device data is read and uploaded, if yes, it goes to step 152, if not, it returns to step 132; Step 152, the running is ended.

6. The method of claim 5, wherein, The step 4 selects a remote control mode, and then goes to step 53; Step 53, the torque sensor is activated, and then goes to step 63; Step 63, the 4G module is activated to communicate with the remote control center, and then goes to step 73; Step 73, the remote control center judges whether the automatic meter reading robot device state information and instruction signal are received, if yes, it goes to step 83, if not, it returns to step 63; Step 83, the remote control center obtains the laser radar and inertial sensor information to obtain the automatic meter reading robot space position information, and then goes to step 93; Step 93, the remote control center remotely controls the electric wheel module to adjust the position and the five-axis mechanical arm posture, and then goes to step 103; Step 103, the torque sensor judges whether an abnormal interaction force appears, if yes, it performs obstacle avoidance and returns to step 93, if not, it goes to step 113; Step 113, the remote control center positions according to the laser radar and inertial sensor data transmitted by the 4G module, and calls the automatic focusing algorithm to find the best focus point, and then goes to step 123; Step 123, the remote control center remotely controls the electric wheel module to adjust the position and the five-axis mechanical arm posture according to the best focus point position, and then goes to step 133; Step 133, the remote control center calls the visual recognition algorithm, and then goes to step 143; Step 143, the high-precision recognition camera identifies the screen and instrument panel by using the visual recognition algorithm, reads the data, and uploads the data to the remote control center through the 4G network, and then goes to step 153; Step 153, the remote control center receives the data transmitted by the 4G network and stores the data, and then goes to step 163; Step 163, it is judged whether the device data is read and uploaded, if yes, it goes to step 173, if not, it returns to step 153; Step 173, the running is ended.

7. The method according to claim 5 or 6, characterized in that, The automatic focusing algorithm includes the following steps: S1, initialize the deep learning model, sensor, and focusing parameter; S2, obtain the current image frame, IMU data, and laser radar data; S3, fuse the sensor data to estimate the relative position and posture; S4, determine the focusing search range according to the estimated information; S5, input the image into the deep learning model to predict the focus point quality; S6, scan the focusing position in the search range according to the current step length; S7, obtain a new image and calculate the actual focus point quality; S8, compare the actual and predicted focus point quality, and adjust the step length; S9, record the best focusing position; S10, judge whether the stop condition is met; S11, output the final focusing position and control the lens focusing; The S5 specifically includes: The input image is where H, W, C are the height, width and number of channels of the image, respectively. After the lightweight CNN model extracts the features, a feature map is obtained ; wherein, is the feature value at coordinate (k, l) in the feature map; h, w, c are the height, width and channel number of the feature map, respectively, denotes that the feature map is a real matrix of dimension ​ transforming the feature map into a one-dimensional vector by a global average pooling layer , denotes a one-dimensional vector obtained by transforming the feature map into a one-dimensional vector by a global average pooling layer, the vector has a dimension of c, denotes that the vector is a real number vector with a length of c; The calculation formula of the global average pooling is as follows: ； in, A one-dimensional vector The One element; This represents the total number of pixels in the feature map. Indicates the first feature map All pixel values ​​in each channel Summing and then dividing by the total number of pixels yields the average value for that channel, which is then used as a vector. The One element; Finally, the focus quality prediction value is output through the full connection layer and the Sigmoid activation function is expressed as: ； wherein, represents the focus quality prediction value, ranging from [0, 1], and the value closer to 1 indicates that the current image is clearer in focus; is a Sigmoid activation function, which maps the output of the fully connected layer to the interval [0, 1], and the calculation formula is: ； W is a weight parameter of the fully connected layer, with a dimension of , used for linear transformation of the feature vector ; v is a one-dimensional feature vector obtained by global average pooling; b is a bias parameter of the fully connected layer, used for adjusting the reference offset of the linear transformation.

8. The method according to claim 5 or 6, characterized in that, The visual recognition algorithm includes the following steps: S1, image acquisition: obtain the original frame through a high-precision recognition camera , the frame rate is set to 30 FPS; S2, image pre-processing: Gaussian filtering, histogram equalization on the original frame :​​ ； S3, target detection: enhanced frames are input into a lightweight YOLO network to obtain a candidate box set As input, the lightweight YOLO network obtains a candidate box set : ； wherein each candidate box contains target region information of a candidate recognized in target detection; express The center coordinates in the image coordinate system are used to determine the position of the candidate box in the image; Candidate boxes The width and height are used to describe the size of the candidate box; The logistic sigmoid function used to calculate the confidence score of candidate boxes converts the raw scores output by the network. Mapping to the [0,1] interval yields the probability that the candidate box belongs to the true target; is the activation function for sigmoid; This refers to the raw score output by the object detection network for the i-th candidate box, before it has been mapped by the Sigmoid function. Its numerical range is the real number domain. The confidence score is obtained after processing by the Sigmoid function. ; Non-maximum suppression (NMS) is adopted: ； wherein, means the set of candidate boxes left after the non-maximum suppression operation, which are the regions that can accurately correspond to the real target after screening; is a function of the non-maximum suppression operation. For multiple candidate boxes existing for the same target, the intersection-over-union between them is calculated. If the intersection-over-union is greater than a set threshold , the candidate box with high confidence is retained, and the candidate box with low confidence is deleted.​ S4, ROI perspective transform: if the target frame is detected , the ROI containing the target is extracted, denoted as ; for the ROI with tilt, perspective transform is needed to correct, and the homography matrix is obtained by four-point mapping ； wherein, is a homography matrix, which is used to realize the perspective transformation of a two-dimensional image plane, and can convert a tilted ROI into an orthographic view according to a set mapping rule, so as to make the text region straighten up; and is a matrix constructed by original four-point coordinates in the ROI and expected four-point coordinates after correction; is an orthographic view obtained after perspective transformation correction, and the target region in the image is horizontal or orthographic; represents a perspective transformation function, the input is an original tilted ROI image and a homography matrix , and the output is an orthographic view after perspective transformation correction ; S5, OCR recognition: recognize the corrected image inputting the OCR engine obtaining the character sequence: ； wherein represents an optical character recognition (OCR) engine that receives the corrected image and outputs recognized characters and associated confidence information; S6, result analysis and output: on perform regularized analysis output JSON structure: ； wherein, represents a regularized parsing operation; represents a confidence threshold used for screening the recognition result in the regularized parsing process.

Images