Method and device for acquiring display data of microcomputer screen on locomotive

Abstract

The invention discloses a method and a device for acquiring microcomputer screen display data on a locomotive, and relates to the technical field of railway freight trains. According to the scheme provided by the invention, video data corresponding to a display picture on a microcomputer screen is acquired, the video data comprises a plurality of continuous image frames, each image frame corresponds to one frame of the display picture shot, and at least one parameter display area exists on the display picture; in the plurality of continuous image frames, tracking the corresponding pixel block of the parameter display area on each image frame; and tracking and collecting data information on the pixel blocks in the plurality of continuous image frames to obtain data information corresponding to at least one parameter displayed on the microcomputer screen, and applying the data information to log recording of the parameters.

Description

Technical Field

[0001] The present application relates to the technical field of railway freight trains, and in particular to a method and device for obtaining data displayed on a microcomputer screen on a locomotive. Background Art

[0002] In the modern railway transportation industry, real-time monitoring of the operating parameters of freight locomotives (such as diesel engine speed, time, current, and voltage) is crucial to ensure driving safety and optimize maintenance plans. However, most of the microcomputer monitoring systems used on freight locomotives can only provide regional display of diesel engine speed, speed, current, voltage and other parameter information on the microcomputer screen, and do not provide integrated log recording functions. If this function is needed, additional development costs are required. Therefore, the current data collection operation of parameter information still mainly relies on manual transcription, which is time-consuming and error-prone, and is a problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides a method and device for obtaining data displayed on a microcomputer screen on a locomotive. The main purpose is to realize automatic data collection operation of parameters displayed on the microcomputer screen with the help of video image processing, which is applied to complete the log recording of the parameters. The automated operation improves the efficiency and accuracy of data collection of the parameters, and thereby also improves the efficiency and accuracy of the log recording of the parameters.

[0004] In order to achieve the above objectives, this application mainly provides the following technical solutions:

[0005] The first aspect of the present application provides a method for obtaining data displayed on a microcomputer screen on a locomotive, the method comprising:

[0006] Acquire video data corresponding to a display screen on a captured microcomputer screen, wherein the video data includes a plurality of continuous image frames, each of the image frames corresponds to a captured frame of the display screen, and there is at least one parameter display area on the display screen;

[0007] In the plurality of continuous image frames, tracking the pixel blocks corresponding to the parameter display area on each of the image frames;

[0008] In the plurality of continuous image frames, data information on the pixel blocks is tracked and collected to obtain data information corresponding to at least one parameter displayed on the microcomputer screen, which is used to complete the log recording of the parameter.

[0009] The second aspect of the present application provides a device for obtaining data displayed on a microcomputer screen on a locomotive, the device comprising:

[0010] An acquisition unit is used to acquire video data corresponding to a display screen on a captured microcomputer screen, wherein the video data includes a plurality of continuous image frames, each of which corresponds to a captured frame of the display screen, and there is at least one parameter display area on the display screen;

[0011] A tracking processing unit, used for tracking the pixel blocks corresponding to the parameter display area in each of the image frames in the plurality of continuous image frames;

[0012] The acquisition processing unit is used to track and acquire the data information on the pixel blocks in the multiple continuous image frames to obtain the data information corresponding to at least one parameter displayed on the microcomputer screen, which is used to complete the log recording of the parameter.

[0013] The third aspect of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for obtaining display data on a microcomputer screen on a locomotive is implemented as described above.

[0014] The fourth aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the method for obtaining display data on a microcomputer screen on a locomotive is implemented as described above.

[0015] By means of the above technical solution, the technical solution provided by this application has at least the following advantages:

[0016] The present application provides a method and device for obtaining data displayed on a microcomputer screen on a locomotive. The present application is to install a shooting device on a freight locomotive in advance, and to shoot various parameter display areas existing in the display image on the microcomputer screen during the operation of the freight locomotive. For the shot video data, since it contains multiple continuous image frames, by tracking the corresponding pixel blocks of these parameter display areas on each continuous image frame, it is indirectly realized to track and shoot the display content of each parameter display area on each continuous image frame, thereby tracking and collecting the data information on each pixel block on each image frame, realizing the data collection operation of each parameter displayed on the microcomputer screen, and applying to complete the log recording of the parameters.

[0017] Compared with the prior art that relies on manual transcription to complete data collection operations on parameters, resulting in low efficiency and prone to errors, the present application uses video image processing to realize automatic data collection operations on parameters displayed on a microcomputer screen, which is applied to complete the log recording of parameters without incurring additional development costs for the microcomputer monitoring system of freight trains. The automated operation improves the efficiency and accuracy of data collection on parameters, thereby also improving the efficiency and accuracy of log recording of parameters.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0020] Figure 1 A flow chart of a method for obtaining data displayed on a microcomputer screen on a locomotive provided in an embodiment of the present application;

[0021] Figure 2 Another method for obtaining data displayed on a microcomputer screen on a locomotive provided in an embodiment of the present application is a flow chart

[0022] Figure 3 Provides an effect diagram showing tracking of a pixel block corresponding to a parameter display area for an embodiment of the present application;

[0023] Figure 4 A schematic diagram of the data effect obtained by performing a data collection operation on a parameter according to an embodiment of the present application;

[0024] Figure 5 A block diagram of a device for obtaining display data on a microcomputer screen on a locomotive provided in an embodiment of the present application;

[0025] Figure 6 A block diagram of another device for obtaining display data on a microcomputer screen on a locomotive provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0027] The embodiment of the present application provides a method for obtaining data displayed on a microcomputer screen on a locomotive, such as Figure 1 As shown, the present application embodiment provides the following specific steps:

[0028] 101. Obtain video data corresponding to a display screen on a captured microcomputer screen, wherein the video data includes a plurality of continuous image frames, each image frame corresponds to a captured display frame, and there is at least one parameter display area on the display screen.

[0029] In the operation scenario of freight locomotives, the intelligence level of the freight locomotive cab is not as good as that of the passenger train cab. The microcomputer monitoring system used on freight locomotives can mostly only provide regional display of diesel engine speed, speed, current, voltage and other parameter information on the microcomputer screen, and does not provide integrated log recording function. If this function is required, additional development costs will be required.

[0030] Based on the above considerations, the embodiment of the present application is to monitor and shoot the data information of each parameter in the display screen on the microcomputer screen with the help of a shooting device pre-installed in the cab, with the purpose of realizing data collection of each parameter. Among them, the shooting device can be but is not limited to a high-resolution camera, which is used to shoot the entire display screen on the microcomputer screen of the freight locomotive, especially to shoot different parameter display areas on the same display interface, and each parameter display area corresponds to the data information of a unique display parameter. For example, the parameters that can be displayed in the display screen of the microcomputer screen include but are not limited to the data information of parameters such as diesel engine speed, speed, current, and voltage. In order to distinguish different parameters in the display screen, there are actually multiple parameter display areas, and each parameter display area corresponds to a unique service to display the data information of a parameter.

[0031] During the operation of the freight locomotive, the embodiment of the present application utilizes a pre-installed shooting device to shoot the display screen on the computer screen. The captured video data will contain multiple continuous image frames and each image frame corresponds to a captured display frame and multiple parameter display areas in the display screen.

[0032] 102. In a plurality of continuous image frames, a tracking parameter display area corresponds to a pixel block in each image frame.

[0033] For the video data obtained by shooting the display screen on the microcomputer screen, since each image frame uniquely corresponds to a frame of the display screen on the microcomputer screen, and there will be a parameter display area in each frame of the display screen, then in fact, a parameter display area has a corresponding pixel block in an image frame.

[0034] Accordingly, in the continuous image frames in the captured video data, the embodiment of the present application tracks the pixel blocks corresponding to the same parameter display area in different image frames, thereby realizing the monitoring of the same parameter display area based on the video data. The purpose of the monitoring is to obtain the constantly changing data information on the parameter display area.

[0035] It should be noted that, since freight trains will produce significant shaking during operation, this shaking will cause the video image collected by the camera to also shake accordingly. This shaking not only affects the stability of the image, but may also have a negative impact on the accuracy of data acquisition and processing. In the embodiment of the present application, the tracking operation of the pixel blocks between different image frames is used. Since the pixel blocks obtained on different image frames are used to characterize the same parameter display area, the position comparison of the pixel blocks between different image frames can correct the offset caused by the shaking, thereby obtaining more stable and high-quality data information on the image frame, thereby improving the quality of the data information collected from the parameters.

[0036] 103. In a plurality of continuous image frames, data information on pixel blocks is tracked and collected to obtain data information corresponding to at least one parameter displayed on a microcomputer screen, which is used to complete log recording of the parameters.

[0037] For the video data obtained by shooting the screen displayed on the computer screen, since the consecutive image frames contained in the video data are sorted according to the shooting time, the tracking of the same parameter display area is indirectly achieved based on the tracking that there are pixel blocks in multiple consecutive image frames. Then, based on such tracking operations, the changes in the data information of each parameter can be monitored, thereby completing the log recording of the parameters.

[0038] As described above, the embodiment of the present application provides a method for obtaining data displayed on a microcomputer screen on a locomotive. Compared with the prior art that relies on manual transcription to complete data collection operations on parameters, resulting in low efficiency and error-prone technical problems, the present application uses video image processing to realize automatic data collection operations on parameters displayed on the microcomputer screen, which is applied to complete the log recording of parameters without incurring additional development costs for the microcomputer monitoring system of freight trains. In addition, the automated operation improves the efficiency and accuracy of data collection of parameters, thereby also improving the efficiency and accuracy of log recording of parameters.

[0039] For a more detailed explanation, the present application embodiment also provides another method for obtaining the data displayed on the microcomputer screen on the locomotive, such as Figure 2 As shown, the present application embodiment provides the following specific steps:

[0040] 201. Obtain video data corresponding to a display screen on a captured microcomputer screen, wherein the video data includes a plurality of continuous image frames, each image frame corresponds to a captured display frame, and there is at least one parameter display area on the display screen.

[0041] In the embodiment of the present application, this step is explained with reference to step 101 and will not be described again here.

[0042] 202. Determine a first image frame and a last image frame among a plurality of continuous image frames.

[0043] 203. According to the relative position of the parameter display area in the display picture, determine a pixel block corresponding to the parameter display area in the first image frame as a first pixel block corresponding to the parameter display area.

[0044] In a plurality of consecutive image frames obtained by shooting video data, for a parameter display area existing in the display screen, an embodiment of the present application tracks from the first image frame to the last image frame in the plurality of consecutive image frames, and in any two adjacent image frames therein, an affine transformation is performed on a pixel block corresponding to the parameter display area on the previous image frame to a corresponding pixel area on the subsequent image frame (i.e., a pixel block is also obtained accordingly from the pixel area), thereby realizing a tracking operation of a position area (i.e., a pixel block) where the parameter display area exists in a plurality of consecutive image frames based on the affine transformation operation between any two image frames.

[0045] 204. Using a preset inter-frame affine transformation method, map the first pixel block on the first image frame to the next adjacent image frame to obtain a second pixel block corresponding to the parameter display area.

[0046] In a plurality of continuous image frames obtained by shooting video data, a first image frame and a next adjacent image frame are taken as examples, wherein in order to distinguish pixel blocks formed on different image frames in reference to parameter display areas, the present embodiment uses the words "first" and "second" to identify, and identifies the pixel block formed on the first image frame as a "first pixel block", and identifies the pixel block formed on the next adjacent image frame as a "second pixel block". The present embodiment refines 204 to include the following: A1-A5;

[0047] A1. Use a preset feature point extraction method to process the first image frame and the next adjacent image frame respectively to obtain the first feature point information corresponding to the first image frame and the second feature point information corresponding to the next adjacent image frame, wherein the first feature point information includes multiple first key points and a first descriptor corresponding to each first key point, and the second feature point information includes multiple second key points and a second descriptor corresponding to each second key point.

[0048] For the tracking operation of pixel blocks between image frames, the embodiment of the present application first collects the feature point information on the first image frame and the next adjacent image frame respectively. In order to distinguish and refer to them, the embodiment of the present application uses the words "first" and "second" for identification, such as obtaining the first feature point information corresponding to the first image frame and the second feature point information corresponding to the next adjacent image frame.

[0049] Among them, the feature point information on the image frame refers to the pixel points or pixel areas in the image with specificity, discrimination and robustness characteristics. It represents the key information of the objects in the image, such as corner points, textures, edges, patches and other features.

[0050] The embodiment of the present application may, but is not limited to, adopt an ORB (Oriented FAST and Rotated BRIEF) image feature point extraction algorithm to process the first image frame and the next adjacent image frame respectively. The specific extraction steps mainly include: (1) FAST key point selection; (2) BRIEF descriptor construction.

[0051] In the ORB algorithm, key points and descriptors are its core components. Key points, also known as feature points or interest points, are points in an image with significant changes or unique properties. In the ORB algorithm, key points are mainly extracted based on the FAST (Features from Accelerated Segments Test) algorithm. The FAST algorithm is a method for detecting corners that quickly determines key points by comparing the grayscale value difference between a pixel and its surrounding pixels. Specifically, if a pixel has a significant difference in grayscale value with enough pixels around it (exceeding a set threshold), the pixel is considered a key point.

[0052] In the ORB algorithm, the FAST algorithm is improved, mainly by adding the calculation of the main direction of the feature point. This improvement makes the subsequent descriptor rotationally invariant and improves the robustness of feature matching. The calculation of the main direction is usually achieved by calculating the centroid of the image block centered on the key point and determining the vector direction of the key point pointing to the centroid.

[0053] A descriptor is a vector representation used to describe the pixel information around a key point. In the ORB algorithm, the descriptor is constructed based on the BRIEF (Binary Robust Independent Elementary Features) algorithm. The core idea of ​​the BRIEF algorithm is to select N point pairs around the key point in a certain pattern, and then combine the grayscale value comparison results of these point pairs as a descriptor. Specifically, for each key point, the BRIEF algorithm selects a series of point pairs around it and compares the grayscale values ​​of each pair of point pairs. If the grayscale value of the first point is greater than or equal to the grayscale value of the second point, it is recorded as 1; otherwise, it is recorded as 0. In this way, for N point pairs, an N-dimensional binary descriptor can be obtained.

[0054] In order to enhance the rotation invariance of the descriptor, the ORB algorithm rotates the image block according to the main direction of the key point before calculating the descriptor. In this way, no matter how the image is rotated, as long as the pixel value change pattern around the key point is the same, the corresponding descriptor will be consistent.

[0055] In the embodiment of the present application, the first feature point information corresponding to the first image frame and the second feature point information corresponding to the next adjacent image frame are obtained, the first feature point information includes multiple first key points and the first descriptor corresponding to each first key point, and the second feature point information includes multiple second key points and the second descriptor corresponding to each second key point. It should be noted that the identifiers "first" and "second" added before the key points and descriptors are convenient for distinguishing whether their sources are the first feature point information or the second feature point information, and they only serve as an identifier here, and there is no other sorting ambiguity.

[0056] A2. By comparing the first descriptor and the second descriptor, a matching operation is performed on the first key point and the second key point to obtain multiple groups of feature point pairs that match between image frames.

[0057] For the first image frame and the next adjacent image frame, after obtaining the feature point information (i.e., key points and descriptors) on each image frame, the features of the two image frames can be matched based on the key points and descriptors. For example, a key point is selected on each of the two image frames, and the descriptors of the key points are taken out. The number S (Hamming distance) of different elements at the same position in the two N-dimensional vectors is counted. The larger S is, the higher the similarity of the two feature points is. Furthermore, by setting a threshold, it is determined whether the two key points are a matching feature point pair.

[0058] It should be noted that the first feature point information includes multiple first key points and a first descriptor corresponding to each first key point, and the second feature point information includes multiple second key points and a second descriptor corresponding to each second key point. Therefore, the embodiment of the present application is to traverse the subsequent image frame to find key points that can be grouped for multiple key points on the previous image frame, thereby obtaining multiple groups of feature point pairs between two adjacent image frames. For this, the embodiment of the present application can, but is not limited to, provide the following two parallel solutions.

[0059] A solution for traversing and searching for key points that can be grouped is: for the first key point on the first image frame, all the second key points on the next adjacent image frame can be judged one by one, and according to the order of the multiple first key points, they can be compared with the multiple second key points one by one in order.

[0060] Another parallel solution is: for a first key point on the first image frame, it can be first projected onto the next adjacent image frame, and a traversal search is performed in the vicinity of the projection point position on the next adjacent image frame to find a second key point that matches the first key point, thereby speeding up the matching process for each first key point.

[0061] In addition, freight trains will produce significant shaking during operation, and this shaking will cause corresponding shaking in the video images collected by the camera, which will affect the image capture quality. In order to avoid the adverse effects caused by this shaking, the embodiment of the present application can verify the quality of the image frame captured later based on the matching rate of the feature point information between two adjacent image frames, based on the image frame captured previously, thereby implementing an iterative verification operation between multiple consecutive image frames, thereby ensuring the quality of multiple consecutive image frames. Specifically, taking the first image frame and the next adjacent image frame as an example, the implementation steps may include the following: A21-A23;

[0062] A21, determining a percentage value of matching between the first feature point information and the second feature point information according to the multiple sets of feature point pairs obtained between two adjacent image frames;

[0063] A22, if the percentage data does not reach the preset threshold, correcting the offset between the first image frame and the next adjacent image frame;

[0064] A23. After the correction, re-execute the process of obtaining new first feature point information corresponding to the first image frame and new second feature point information corresponding to the next adjacent image frame, so as to obtain multiple groups of feature point pairs matching between the image frames.

[0065] For example, A21-A23 is the verification process adopted between the first image frame and the next adjacent image frame (these two adjacent image frames). Based on the same principle, the embodiment of the present application can implement iterative verification operations on any two image frames between multiple consecutive image frames to avoid the adverse effects of freight locomotive operation jitter on the quality of captured images.

[0066] A3. Determine, from among the multiple groups of feature point pairs, a target feature point pair associated with the first pixel block, wherein the target feature point pair includes a target first key point and a target second key point, wherein the target first key point is associated with the first pixel block, and the target second key point exists in the next adjacent image frame;

[0067] For the first pixel block corresponding to the parameter display area on the first image frame, the embodiment of the present application first determines the feature information covered by the first pixel block (i.e., including the first key point), and then by searching the multiple groups of feature point pairs obtained by A2, it can be further known which second key points on the next adjacent image frame the first key point matches, and such first key points and second key points are combined into a "target feature point pair".

[0068] Based on this, for the same parameter display area, the embodiment of the present application is equivalent to being able to know: which first key points on the first image frame and which second key points on the next adjacent image frame are matched, that is, the target feature point pairs they constitute.

[0069] A4. Based on the target feature point pair, calculate the affine transformation between frames to determine the coordinate data of the first pixel block on the next adjacent image frame;

[0070] A5. Determine a corresponding pixel block in the next adjacent image frame according to the coordinate data as the second pixel block.

[0071] As shown in A4-A5, after obtaining the target feature point pairs, the embodiment of the present application uses these feature pairs to calculate the inter-frame affine transformation, the purpose of which is to determine the pixel block corresponding to the parameter display area on the previous image frame, and the actual pixel coordinate data on the subsequent image frame (i.e., what is the pixel block formed on the subsequent image frame, i.e., obtain the second pixel block), thereby completing the tracking of the same parameter display area on the next adjacent image frame. The working principle of the inter-frame simulation transformation adopted in the embodiment of the present application includes the following:

[0072] Affine transformation in image processing involves mapping an image from one coordinate system to another. It is achieved by applying a linear transformation matrix and a translation vector. Specifically, affine transformation can be expressed as a matrix operation:

[0073]

[0074] in, are the coordinates of the point in the original image, are the coordinates of the point in the transformed image,

[0075] is the linear transformation matrix, is the translation vector.

[0076] like Figure 3 As shown, the embodiment of the present application adopts feature matching and inter-frame affine transformation, which is applied to the video data captured on the microcomputer screen. Taking two adjacent image frames as an example, the effect diagram of tracking the pixel blocks corresponding to the parameter display area is shown.

[0077] 205. Iteratively perform the mapping operation between the next adjacent image frame and the last image frame until a pixel block corresponding to the parameter display area is obtained on each image frame.

[0078] As shown in 203-204, the embodiment of the present application provides that the pixel block tracking of the parameter display area is realized between the first image frame and the next adjacent image frame. Based on the same working principle, the mapping operation can be iteratively performed between the next adjacent image frame to the last image frame, and between any two adjacent image frames, until the pixel block corresponding to the parameter display area is obtained on each image frame. Thus, the tracking operation of the pixel block corresponding to the parameter display area is realized between multiple continuous image frames obtained from the video data.

[0079] 206. In a plurality of continuous image frames, data information on pixel blocks is tracked and collected to obtain data information corresponding to at least one parameter displayed on a microcomputer screen, which is used to complete log recording of the parameters.

[0080] In the embodiment of the present application, this step may be, but is not limited to, detailed as follows:

[0081] B1, converting the image frame into a black and white image;

[0082] The embodiment of the present application first removes noise and interference in the image frame to improve the quality of the image frame, and then uses a binarization method to convert the image frame into a black and white image to simplify subsequent operation steps.

[0083] B2, identify the text area from the pixel block;

[0084] In each image frame, the text that appears can be adjusted to be displayed horizontally to improve subsequent recognition accuracy, and then the pixel blocks can be parsed to identify and extract the area containing the text.

[0085] B3, performing character segmentation processing in the text area to obtain multiple character block areas;

[0086] The text area appearing in the image frame is divided into lines, and the text in each line is further divided into separate character block areas.

[0087] B4, using a preset character recognition model to process the character block area to obtain character information corresponding to the text area;

[0088] The embodiment of the present application can use a classifier and a character pattern to pre-train a character recognition model, thereby identifying a character block area to obtain the character information contained in the character block area.

[0089] In addition, the embodiments of the present application may also provide post-processing to implement correction or verification operations, such as: spelling correction, correcting spelling errors that may occur during the recognition process; format recovery, restoring the original format and layout of the document.

[0090] B5. By collecting character information corresponding to the text area, data information corresponding to the parameters in each image frame is obtained.

[0091] As B1-B5 is a specific implementation method for performing data acquisition operations on parameters on each image frame provided by the embodiment of the present application. Since these image frames are multiple continuous image frames obtained from video data, and each image frame is uniquely corresponding to a frame of microcomputer screen display, it can be seen that during the operation of the freight locomotive, the data information of the parameters displayed on the microcomputer screen is continuously obtained. As the amount of video data increases, the data information of the parameters obtained based on the video data is also huge, and there is data redundancy. Therefore, the embodiment of the present application can adopt the following implementation steps C1-C3 to further manage and visualize these huge data information.

[0092] C1. Import the data information corresponding to the parameters into a preset text template. The preset text template uses a multidimensional data table to display the data information of different parameters.

[0093] C2. In the preset text template, for the same parameter, the data information corresponding to the parameter provided by each image frame is sorted and displayed according to the time sequence of capturing multiple consecutive image frames.

[0094] C3. In the sorting display, the data information corresponding to the parameters provided by each image frame is deduplicated to obtain the data information corresponding to the parameters at different moments.

[0095] As C1-C3, the embodiments of the present application provide Figure 4 The schematic diagram of the data effect obtained by performing the data collection operation on the parameters shown in the figure can, for example, but is not limited to, use an EXCEL table for data display, and since the computer screen display includes multiple parameter display areas, the embodiment of the present application can pre-number each parameter display area digitally, so that based on the unique correspondence between the parameter display area and the pixel block, the multiple pixel blocks formed on each image frame also indirectly have such digital numbers. Then, when the data information collected from the pixel block is filled into the preset text template, it can be relied on such digital numbering to quickly fill it into the specified parameter category in the EXCEL table, thereby improving data management efficiency and avoiding redundant and erroneous filling.

[0096] As the above Figure 1 , Figure 2The embodiment of the present application provides a device for obtaining data displayed on a microcomputer screen on a locomotive. The device embodiment corresponds to the aforementioned method embodiment. For ease of reading, the device embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspond to all the contents of the aforementioned method embodiment. The device is used to realize automatic data collection of parameters on the microcomputer screen and to complete the log recording of the parameters, such as Figure 5 , the device comprises:

[0097] An acquisition unit 31 is used to acquire video data corresponding to a display screen on a captured microcomputer screen, wherein the video data includes a plurality of continuous image frames, each of which corresponds to a captured frame of the display screen, and there is at least one parameter display area on the display screen;

[0098] A tracking processing unit 32, used for tracking the pixel blocks corresponding to the parameter display area in each of the plurality of continuous image frames;

[0099] The acquisition processing unit 33 is used to track and acquire the data information on the pixel blocks in the multiple continuous image frames to obtain the data information corresponding to at least one parameter displayed on the microcomputer screen, which is used to complete the log recording of the parameter.

[0100] In some embodiments, Figure 6 As shown, the tracking processing unit 32 includes:

[0101] A first determining module 321 is used to determine a first image frame and a last image frame among the plurality of continuous image frames;

[0102] A second determining module 322, configured to determine, according to a relative position of the parameter display area in the display picture, a pixel block corresponding to the parameter display area in the first image frame as a first pixel block corresponding to the parameter display area;

[0103] A mapping processing module 323, used to map the first pixel block on the first image frame to the next adjacent image frame by using a preset inter-frame affine transformation method, to obtain a second pixel block corresponding to the parameter display area;

[0104] The execution module 324 is used to iteratively perform the mapping operation between the next adjacent image frame and the last image frame until a pixel block corresponding to the parameter display area is obtained on each of the image frames.

[0105] In some embodiments, the mapping processing module 323 is used to:

[0106] Using a preset feature point extraction method to process the first image frame and the next adjacent image frame respectively, to obtain first feature point information corresponding to the first image frame and second feature point information corresponding to the next adjacent image frame, wherein the first feature point information includes a plurality of first key points and a first descriptor corresponding to each of the first key points, and the second feature point information includes a plurality of second key points and a second descriptor corresponding to each of the second key points;

[0107] By comparing the first descriptor and the second descriptor, a matching operation is performed on the first key point and the second key point to obtain a plurality of groups of feature point pairs matched between image frames;

[0108] Determine, from the plurality of groups of feature point pairs, a target feature point pair associated with the first pixel block, wherein the target feature point pair includes a target first key point and a target second key point, wherein the target first key point is associated with the first pixel block, and the target second key point exists on the next adjacent image frame;

[0109] Based on the target feature point pair, calculating the affine transformation between frames, and determining the coordinate data of the first pixel block on the next adjacent image frame;

[0110] According to the coordinate data, a corresponding pixel block is determined on the next adjacent image frame as a second pixel block.

[0111] In some embodiments, during the execution of the mapping processing module 323, after obtaining a plurality of sets of feature point pairs matched between image frames, the method further includes:

[0112] Determining, based on the plurality of sets of feature point pairs, a percentage value of a match between the first feature point information and the second feature point information;

[0113] If the percentage data does not reach a preset threshold, correcting the offset between the first image frame and the next adjacent image frame;

[0114] After the correction, the process of obtaining new first feature point information corresponding to the first image frame and new second feature point information corresponding to the next adjacent image frame is performed again to obtain a plurality of feature point pairs matched between the image frames.

[0115] In some embodiments, Figure 6 As shown, the acquisition processing unit 33 includes:

[0116] A conversion module 331, used for converting the image frame into a black and white image;

[0117] A recognition module 332, configured to recognize a text area from the pixel block;

[0118] A first processing module 333 is used to perform character segmentation processing in the text area to obtain multiple character block areas;

[0119] The second processing module 334 is used to process the character block area using a preset character recognition model to obtain character information corresponding to the text area;

[0120] The acquisition module 335 is used to acquire the character information corresponding to the text area to obtain the data information corresponding to the parameter in each image frame.

[0121] In some embodiments, Figure 6 As shown, after obtaining the data information corresponding to the parameter on each image frame, the device further includes:

[0122] An import unit 34, used to import the data information corresponding to the parameters into a preset text template, wherein the preset text template displays the data information of different parameters in the form of a multidimensional data table;

[0123] A sorting unit 35 is used to sort and display data information corresponding to the parameter provided by each of the image frames in the preset text template for the same parameter according to the time sequence of capturing the multiple consecutive image frames;

[0124] The deduplication processing unit 36 ​​is used to perform deduplication processing on the data information corresponding to the parameters provided by each of the image frames in the sorting display, so as to obtain the data information corresponding to the parameters at different moments.

[0125] In summary, a device for acquiring display data on a microcomputer screen on a locomotive includes a processor and a memory. The above-mentioned acquisition unit, tracking processing unit and collection processing unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0126] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the automatic data collection operation of the parameters displayed on the microcomputer screen is realized by adjusting the kernel parameters with the help of video image processing, which is applied to complete the log recording of the parameters. The automated operation improves the efficiency and accuracy of the data collection of the parameters, and accordingly also improves the efficiency and accuracy of the log recording of the obtained parameters.

[0127] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for obtaining data displayed on a microcomputer screen on a locomotive is implemented as described above.

[0128] An embodiment of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the method for obtaining display data on a microcomputer screen on a locomotive is implemented as described above.

[0129] The embodiment of the present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes the method steps of obtaining display data on a microcomputer screen on a locomotive.

[0130] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0131] In a typical configuration, the device includes one or more processors (CPU), memory and bus. The device may also include input / output interface, network interface and the like.

[0132] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.

[0133] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0134] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0135] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0136] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for obtaining data displayed on a microcomputer screen on a locomotive, the method comprising: Acquire video data corresponding to a display screen on a captured microcomputer screen, wherein the video data includes a plurality of continuous image frames, each of the image frames corresponds to a captured frame of the display screen, and there is at least one parameter display area on the display screen; In the plurality of continuous image frames, tracking the pixel blocks corresponding to the parameter display area on each of the image frames; In the plurality of continuous image frames, data information on the pixel blocks is tracked and collected to obtain data information corresponding to at least one parameter displayed on the microcomputer screen, which is used to complete the log recording of the parameter.

2. The method according to claim 1, characterized in that In the plurality of continuous image frames, tracking the pixel blocks corresponding to the parameter display area on each of the image frames comprises: Determine a first image frame and a last image frame among the plurality of continuous image frames; According to the relative position of the parameter display area in the display picture, determining a pixel block corresponding to the parameter display area in the first image frame as a first pixel block corresponding to the parameter display area; Using a preset inter-frame affine transformation method, mapping the first pixel block on the first image frame to the next adjacent image frame to obtain a second pixel block corresponding to the parameter display area; The mapping operation is iteratively performed between the next adjacent image frame and the last image frame until a pixel block corresponding to the parameter display area is obtained on each of the image frames.

3. The method according to claim 2, characterized in that The method of mapping the first pixel block on the first image frame to the next adjacent image frame by using a preset inter-frame affine transformation method to obtain a second pixel block corresponding to the parameter display area includes: Using a preset feature point extraction method to process the first image frame and the next adjacent image frame respectively, to obtain first feature point information corresponding to the first image frame and second feature point information corresponding to the next adjacent image frame, wherein the first feature point information includes a plurality of first key points and a first descriptor corresponding to each of the first key points, and the second feature point information includes a plurality of second key points and a second descriptor corresponding to each of the second key points; By comparing the first descriptor and the second descriptor, a matching operation is performed on the first key point and the second key point to obtain a plurality of groups of feature point pairs matched between image frames; Determine, from the plurality of groups of feature point pairs, a target feature point pair associated with the first pixel block, wherein the target feature point pair includes a target first key point and a target second key point, wherein the target first key point is associated with the first pixel block, and the target second key point exists on the next adjacent image frame; Based on the target feature point pair, calculating the affine transformation between frames, and determining the coordinate data of the first pixel block on the next adjacent image frame; According to the coordinate data, a corresponding pixel block is determined on the next adjacent image frame as a second pixel block.

4. The method according to claim 3, characterized in that After obtaining the plurality of sets of feature point pairs matched between the image frames, the method further comprises: Determining, based on the plurality of sets of feature point pairs, a percentage value of a match between the first feature point information and the second feature point information; If the percentage value does not reach the preset threshold, correcting the offset between the first image frame and the next adjacent image frame; After the correction, the process of obtaining new first feature point information corresponding to the first image frame and new second feature point information corresponding to the next adjacent image frame is performed again to obtain a plurality of feature point pairs matched between the image frames.

5. The method according to any one of claims 1 to 4, characterized in that In the plurality of continuous image frames, tracking and collecting data information on the pixel blocks to obtain data information corresponding to at least one of the parameters displayed on the microcomputer screen, including: Converting the image frame into a black and white image; identifying a text region from the pixel block; Performing character segmentation processing in the text area to obtain a plurality of character block areas; Processing the character block area using a preset character recognition model to obtain character information corresponding to the text area; By collecting character information corresponding to the text area, data information corresponding to the parameter in each image frame is obtained.

6. The method according to claim 5, characterized in that After obtaining the data information corresponding to the parameter on each image frame, the method further includes: Importing the data information corresponding to the parameters into a preset text template, wherein the preset text template displays the data information of different parameters in the form of a multidimensional data table; In the preset text template, for the same parameter, the data information corresponding to the parameter provided by each of the image frames is sorted and displayed according to the time sequence of capturing the multiple consecutive image frames; In the sorting display, the data information corresponding to the parameters provided by each of the image frames is deduplicated to obtain the data information corresponding to the parameters at different moments.

7. A device for obtaining data displayed on a microcomputer screen on a locomotive, characterized in that: The device comprises: An acquisition unit is used to acquire video data corresponding to a display screen on a captured microcomputer screen, wherein the video data includes a plurality of continuous image frames, each of which corresponds to a captured frame of the display screen, and there is at least one parameter display area on the display screen; A tracking processing unit, used for tracking the pixel blocks corresponding to the parameter display area in each of the image frames in the plurality of continuous image frames; The acquisition processing unit is used to track and acquire the data information on the pixel blocks in the multiple continuous image frames to obtain the data information corresponding to at least one parameter displayed on the microcomputer screen, which is used to complete the log recording of the parameter.

8. The device according to claim 7, characterized in that The tracking processing unit comprises: A first determining module, used for determining a first image frame and a last image frame among the plurality of continuous image frames; A second determining module is used to determine, according to the relative position of the parameter display area in the display picture, a pixel block corresponding to the parameter display area in the first image frame as a first pixel block corresponding to the parameter display area; A mapping processing module, used to map the first pixel block on the first image frame to the next adjacent image frame by using a preset inter-frame affine transformation method, so as to obtain a second pixel block corresponding to the parameter display area; The execution module is used to iteratively perform the mapping operation between the next adjacent image frame and the last image frame until a pixel block corresponding to the parameter display area is obtained on each of the image frames.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for obtaining display data on a microcomputer screen on a locomotive according to any one of claims 1 to 6 is implemented.

10. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the method for obtaining display data on a microcomputer screen on a locomotive as described in any one of claims 1 to 6 is implemented.

Images