Digital information determination method, device and equipment

Abstract

The invention discloses a digital information determination method, device and equipment. The method comprises the following steps: acquiring a first image and a first base template image, wherein the first base template image comprises preset digital information; dividing the first base template image to obtain a plurality of second base template images and positions of the plurality of second base template images in the first base template image, the size of the plurality of second base template images being the size of the first image; respectively calculating the similarity between the first image and the plurality of second base template images; selecting a target base template image with the maximum similarity from the plurality of second base template images; and determining the digital information of the target base template image at the target position in the first base template image as the digital information in the first image according to the relation information between the preset digital information on the first base template image and the position. And the digital information is determined by using the base template image, so that the accuracy and efficiency of digital information identification are improved.

Description

Technical Field

[0001] The present application belongs to the field of image recognition technology, and in particular, relates to a method, device and equipment for determining digital information. Background Art

[0002] The interconnection relationship of aircraft avionics systems is extremely complex. The cockpit display system is one of the aircraft avionics systems and is the core system for pilots to interact with the aircraft and understand the flight status. Before officially boarding the aircraft, the accuracy and stability of the cockpit display system must be tested on the ground to ensure the flight safety of the aircraft. The correct recognition of digital information such as speed and altitude displayed in the cockpit display system is crucial to the testing of the cockpit display system.

[0003] In the prior art, the recognition of digital information such as speed and altitude displayed in the cockpit display system is mainly through machine vision detection and recognition by the human eye, and the accuracy and efficiency of digital information recognition are low. Summary of the invention

[0004] The embodiments of the present application provide a method, device and equipment for determining digital information, which improves the accuracy and efficiency of digital information recognition by setting a base template image, calculating the similarity between the base template image and a first image, selecting a target base template image with a greater similarity to the first image, and determining that the digital information at a position corresponding to the target base template image is the digital information in the first image, that is, using the base template image to determine the digital information.

[0005] In a first aspect, an embodiment of the present application provides a method for determining digital information, including:

[0006] Acquire a first image and a first base template image, wherein the first base template image includes preset digital information;

[0007] Dividing the first base template image to obtain a plurality of second base template images and positions of the plurality of second base template images in the first base template image, wherein the sizes of the plurality of second base template images are the same as the size of the first image;

[0008] respectively calculating the similarity between the first image and the plurality of second base template images;

[0009] Selecting a target base template image with the greatest similarity from a plurality of second base template images;

[0010] According to the relationship information between the preset digital information and the position on the first base template image, the digital information of the target base template image at the target position in the first base template image is determined to be the digital information in the first image.

[0011] In one possible embodiment, acquiring a first image includes:

[0012] Acquire a second image corresponding to the target area of ​​the display interface;

[0013] The first image at a preset position in the second image is captured.

[0014] In a possible implementation, acquiring a second image corresponding to a target area of ​​a display interface includes:

[0015] Acquire an image acquired by an image acquisition device;

[0016] The size of the image acquired by the image acquisition device is adjusted to a target size to obtain an initial image;

[0017] Extract features from the initial image to obtain a target feature image;

[0018] Determine a target area of ​​the display interface where a target feature value in the target feature image is located;

[0019] A second image corresponding to the target area is captured from the initial image.

[0020] In a possible implementation, extracting features from an initial image to obtain a target feature image includes:

[0021] Inputting the initial image into the target neural network model, and using the target neural network model to extract features of the initial image to obtain a plurality of first feature images;

[0022] Adjusting the sizes and the number of channels of the plurality of first feature images to be consistent to obtain a plurality of second feature images;

[0023] The multiple second characteristic images are superimposed to obtain a target characteristic image.

[0024] In a possible embodiment, capturing a second image corresponding to the target area from the initial image includes:

[0025] intercepting a third image corresponding to the target area from the initial image;

[0026] The vertices of the third image are mapped to the target vertices of the rectangle by using an affine transformation to obtain the second image.

[0027] In a possible implementation, before intercepting the first image at a preset position in the second image, the method further includes:

[0028] Gray-scaling the second image to obtain a gray-scale image;

[0029] Binarize the grayscale image to obtain a binary image;

[0030] De-noise the binary image to obtain a smooth image;

[0031] Capturing a first image at a preset position in a second image, comprising:

[0032] In one possible implementation, the first base template image is divided to obtain a plurality of second base template images, including:

[0033] A sliding window is established on the first base template image, and the sliding window is moved according to a preset step length to divide the first base template image into a plurality of second base template images, and the size of the sliding window is the size of the first image.

[0034] In a possible implementation, respectively calculating the similarity of the first image and the plurality of second base template images includes:

[0035] Calculating the Hu invariant moment of the first image and the Hu invariant moment of a plurality of second base template images;

[0036] The similarity between the first image and the plurality of second base template images is determined according to the difference between the Hu invariant moment of the first image and the Hu invariant moment of the plurality of second base template images.

[0037] In a second aspect, an embodiment of the present application provides a device for determining digital information, including:

[0038] An acquisition module, used for acquiring a first image and a first base template image, wherein the first base template image includes preset digital information;

[0039] A division module, used for dividing the first base template image to obtain a plurality of second base template images and positions of the plurality of second base template images in the first base template image, wherein the size of the plurality of second base template images is the size of the first image;

[0040] A calculation module, used for respectively calculating the similarity between the first image and a plurality of second base template images;

[0041] A selection module, used for selecting a target base template image with the greatest similarity from a plurality of second base template images;

[0042] The determination module is used to determine that the digital information of the target base template image at the target position in the first base template image is the digital information in the first image according to the relationship information between the preset digital information and the position on the first base template image.

[0043] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising:

[0044] a processor and a memory storing computer program instructions;

[0045] When the processor executes the computer program instructions, any one of the above methods for determining digital information is implemented.

[0046] In a fourth aspect, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, any of the above-mentioned methods for determining digital information is implemented.

[0047] In a fifth aspect, an embodiment of the present application provides a computer program product, characterized in that when the instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to execute any one of the above-mentioned methods for determining digital information.

[0048] The method, device and equipment for determining digital information of the embodiment of the present application include: obtaining a first image and a first base template image, wherein the first base template image includes preset digital information; dividing the first base template image to obtain a plurality of second base template images and the positions of the plurality of second base template images in the first base template image, wherein the size of the plurality of second base template images is the size of the first image; respectively calculating the similarity of the first image and the plurality of second base template images; selecting a target base template image with the greatest similarity from the plurality of second base template images; and determining the digital information at the target position of the target base template image in the first base template image as the digital information in the first image according to the relationship information between the preset digital information and the position on the first base template image. In this way, by setting the base template image, calculating the similarity between the base template image and the first image, selecting the target base template image with the greater similarity to the first image, and determining the digital information at the position corresponding to the target base template image as the digital information in the first image, the digital information is determined by using the base template image, thereby improving the accuracy and efficiency of digital information recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0050] Figure 1 It is a flowchart of a method for determining digital information provided by an embodiment of the present application;

[0051] Figure 2 is a first base template image provided by another embodiment of the present application;

[0052] Figure 3 is a flowchart of a method for determining digital information provided by another embodiment of the present application;

[0053] Figure 4A second image corresponding to a target area of ​​a display interface provided by another embodiment of the present application;

[0054] Figure 5 is a flowchart of a method for determining digital information provided by yet another embodiment of the present application;

[0055] Figure 6 is a flowchart of a method for determining digital information provided by yet another embodiment of the present application;

[0056] Figure 7 is a schematic diagram of an interactive process of digital information recognition provided by yet another embodiment of the present application;

[0057] Figure 8 is a structural schematic diagram of a device for determining digital information provided by yet another embodiment of the present application;

[0058] Fig. 9 It is a structural diagram of an electronic device provided in yet another embodiment of the present application. DETAILED DESCRIPTION

[0059] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0060] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0061] The interconnection relationship of aircraft avionics systems is extremely complex. The cockpit display system is one of the aircraft avionics systems and is the core system for pilots to interact with the aircraft and understand the flight status. Before officially boarding the aircraft, the accuracy and stability of the cockpit display system must be tested on the ground to ensure the flight safety of the aircraft. The correct recognition of digital information such as speed and altitude displayed in the cockpit display system is crucial to the testing of the cockpit display system.

[0062] In the prior art, the recognition of digital information such as speed and altitude displayed in the cockpit display system is mainly through machine vision detection and recognition and human eye recognition. The staff in the cockpit can observe the cockpit display system with the naked eye and directly read the digital information on the display interface; or use an image acquisition device (such as a camera) to aim at the cockpit display system, take a picture of the area where the digital information is located, and transmit it to the remote device where the staff is located through a high-speed transmission network, and then the staff can read the digital information on the display interface in the captured image.

[0063] However, the testing of cockpit display systems is relatively complex and time-consuming. It takes a lot of time for workers to recognize the digital information on the display interface with their naked eyes, which is prone to fatigue and recognition errors, seriously reducing the reading efficiency, reliability and accuracy of the digital information displayed in the cockpit display system and increasing labor costs. Therefore, the accuracy and efficiency of recognizing digital information by human eyes are low.

[0064] At present, the recognition of characters on the display interface can be performed through deep learning tools such as tessortocr and paddleocr to recognize text in images. However, its recognition accuracy is extremely low for small targets, single characters, and incomplete digital information; and when multiple incomplete digital information appears at the same time, it cannot accurately judge their mutual relationship, and the recognition accuracy is poor, which cannot meet the test requirements at all.

[0065] In order to solve the problems of the prior art, the embodiments of the present application provide a method, device and apparatus for determining digital information. The method for determining digital information in the embodiments of the present application sets a base template image, calculates the similarity between the base template image and the first image, selects a target base template image with a greater similarity to the first image, and determines that the digital information at the position corresponding to the target base template image is the digital information in the first image, that is, the digital information is determined using the base template image, thereby improving the accuracy and efficiency of digital information recognition.

[0066] The following first introduces the method for determining digital information provided in the embodiment of the present application.

[0067] Figure 1 A schematic flow chart of a method for determining digital information provided by an embodiment of the present application is shown.

[0068] As Figure 1 shown, the method for determining digital information provided by the embodiment of the present application includes the following steps.

[0069] S110. Obtain a first image and a first base template image, where the first base template image includes preset digital information.

[0070] Here, the first image is an image of the area where the display interface is located in the cockpit display system, and the first image includes complete or incomplete digital information to be determined. The preset digital information is complete or incomplete digital information. The specific category of the preset digital information is not limited here. The preset digital information can be pure numerical digital information or capitalized digital information. For example, "1" or "one".

[0071] In some embodiments, the first base template image is set in advance. For example, as Figure 2 shown, it is a preset first base template image. From top to bottom, the first base template image is arranged as the number 1, the number 0, the number 9 to the number 1, and then the number 0. The first base template image can be set according to the category of the first image. A preset base template image can also be set in advance, and the first base template image is selected from the preset base template image. The category of the image here can be a color image, a binary image, etc., or the image category can be divided according to different regions included in the image, or the image category can be divided according to different categories of digital information.

[0072] As an example, receive a selection input from the user for the first base template image in the preset base template image. In response to the selection input, select the first base template image from the preset base template image.

[0073] S120. Divide the first base template image to obtain a plurality of second base template images and the positions of the plurality of second base template images in the first base template image, where the sizes of the plurality of second base template images are the same as the size of the first image.

[0074] Here, the second base template image is a part of the first base template image.

[0075] In some embodiments, dividing the first base template image can be directly splitting the first base template image, and the obtained plurality of second base template images can be spliced into a complete first base template image. It can also be intercepting a second base template image on the first base template image every preset length, and the preset length is less than the length or width of the first image.

[0076] In some embodiments, the first base template image includes coordinate information. According to the coordinate information in the first base template image and the coordinate information of the second base template image obtained by dividing the first base template image, the positions of the plurality of second base template images in the first base template image can be determined.

[0077] S130 , respectively calculating similarities between the first image and a plurality of second base template images.

[0078] Here, the similarity may include, but is not limited to, cosine similarity and similarity obtained by calculating Hu invariant moments.

[0079] In some embodiments, the number of similarities is the same as the number of the second base template images, and one similarity corresponds to one second base template image.

[0080] S140 . Selecting a target base template image with the greatest similarity from a plurality of second base template images.

[0081] Here, the plurality of second base template images include a target base template image.

[0082] In some embodiments, the similarities corresponding to the plurality of second base template images are sorted, and a target base template image with the greatest similarity is selected from the plurality of second base template images.

[0083] S150. Determine, based on the relationship information between the preset digital information and the position on the first base template image, that the digital information of the target base template image at the target position in the first base template image is the digital information in the first image.

[0084] Here, the relationship between the preset digital information and the position on the first base template image is set in advance. Different positions correspond to a complete digital information respectively. The position can be represented by coordinate information.

[0085] In some embodiments, the target position of the target base template image in the first base template image is determined, and the digital information at the target position is determined to be the digital information in the first image according to the relationship information between the preset digital information and the position on the first base template image.

[0086] In this way, by setting the base template image, calculating the similarity between the base template image and the first image, selecting the target base template image with greater similarity to the first image, and determining that the digital information at the position corresponding to the target base template image is the digital information in the first image, that is, using the base template image to determine the digital information, the accuracy and efficiency of digital information recognition are improved.

[0087] Based on this, in some embodiments, such as Figure 3 As shown, in the above S110, acquiring the first image may include:

[0088] S111, acquiring a second image corresponding to a target area of ​​a display interface;

[0089] S112: intercepting the first image at a preset position in the second image.

[0090] Here, the preset position is set in advance. The preset position may be an area of ​​interest, and according to the user's preset, the first image of interest to the user can be intercepted from the second image.

[0091] In some embodiments, the target area of ​​the display interface may be an area where digital information is located in the display interface of the cockpit display system. Different areas of the second image are intercepted according to the units, tens, and percentiles, and the first image at a preset position in the second image is intercepted to reduce the identified area.

[0092] As an example, Figure 4 The second image corresponding to the target area of ​​the display interface is shown. Figure 4 As shown, the second image includes a plurality of digital information. The digital information may be complete or incomplete. When intercepting, a column of digital information in the second image is intercepted.

[0093] In this way, the first image can be captured according to the preset position. Even if the acquired image cannot be directly calculated for similarity with the second base template image, a suitable first image can be captured, the similarity between the first image and the second base template image can be calculated, and the digital information of the first image can be determined, thereby improving the accuracy of digital information recognition.

[0094] Based on this, in some embodiments, such as Figure 5 As shown, the above S111 may specifically include:

[0095] S101, acquiring an image acquired by an image acquisition device;

[0096] S102, adjusting the size of the image acquired by the image acquisition device to a target size to obtain an initial image;

[0097] S103, extracting features from the initial image to obtain a target feature image;

[0098] S104, determining a target area of ​​the display interface where a target feature value in the target feature image is located;

[0099] S105: intercepting a second image corresponding to the target area from the initial image.

[0100] Here, the image acquisition device may be a high frame rate industrial camera. The image acquired by the image acquisition device is the original image taken. In order to input the original image into the network for feature extraction, the size of the image acquired by the image acquisition device needs to be adjusted to the target size, thereby obtaining the initial image.

[0101] It is understandable that after the second image corresponding to the target area is intercepted from the initial image, the size of the second image can be adjusted back to the original size according to the ratio before and after the adjustment. Here, the original size refers to the size of the area occupied by the second image in the image captured by the image acquisition device.

[0102] In some embodiments, the high frame rate industrial camera captures the entire display interface and part of the cockpit cabin, while the area where the digital information is located only occupies a small part of the entire display interface. Therefore, it is necessary to first separate the area where the digital information is located from the background. The image size captured by the high frame rate industrial camera is large, which is not convenient for storage and calculation. In order to speed up the processing speed, the image obtained from the video stream captured by the high frame rate industrial camera is first resized to a suitable size. The size of the image captured by the image acquisition device is adjusted to the target size to obtain the initial image.

[0103] In some embodiments, multiple features may be extracted from the initial image, and the obtained multiple feature images may be superimposed to obtain a target feature image.

[0104] In some embodiments, the feature values ​​of the cockpit body and the display interface or the target area of ​​the display interface are different. The target area of ​​the display interface where the target feature value is located in the target feature image is determined, the cockpit body is segmented as the background, only the target area of ​​the display interface is extracted, and a second image corresponding to the target area is captured from the initial image.

[0105] It should be noted that the target area may be the entire area of ​​the display interface or a partial area of ​​the display interface.

[0106] In this way, feature extraction is performed on the captured image, and the second image corresponding to the target area is determined according to the different feature values, and the identified area is reduced; the first image can be intercepted according to the preset position, and the identified area is further reduced. Even if the acquired image cannot be directly calculated for similarity with the second base template image, a suitable first image can be intercepted, the similarity between the first image and the second base template image can be calculated, and the digital information of the first image can be determined, thereby improving the accuracy of digital information recognition. Moreover, various tests can be performed through the same image acquisition device, which is not limited to the test of the cockpit display system. There is no need to set up an image acquisition device for the cockpit display system separately, which saves costs.

[0107] Based on this, in some embodiments, the above S103 may specifically include:

[0108] Inputting the initial image into the target neural network model, and using the target neural network model to extract features of the initial image to obtain a plurality of first feature images;

[0109] Adjusting the sizes and the number of channels of the plurality of first feature images to be consistent to obtain a plurality of second feature images;

[0110] The multiple second characteristic images are superimposed to obtain a target characteristic image.

[0111] In some embodiments, the target neural network model can be a fully convolutional neural network model (FCN). The FCN neural network is used, and the overall network is based on the VGG16 pre-trained model. On the basis of the basic model, the feature image after each layer of pooling is taken out and superimposed layer by layer. In order to ensure that the size and number of channels of the two added feature images are consistent, the high-dimensional feature image must be deconvolved first and then added. For the superimposed feature image, the target feature image is output through 1x1 convolution.

[0112] In this way, by extracting features multiple times and superimposing the extracted multiple feature images, the accuracy of the target feature image is improved, thereby improving the accuracy of the second image determination.

[0113] Based on this, in some embodiments, the above S105 may specifically include:

[0114] intercepting a third image corresponding to the target area from the initial image;

[0115] The vertices of the third image are mapped to the target vertices of the rectangle by using an affine transformation to obtain the second image.

[0116] In some embodiments, due to the camera angle error and lens distortion, the acquired initial image may be a tilted quadrilateral, and the area where the digital information is located in the third image corresponding to the target area intercepted from the initial image may be a tilted quadrilateral, so it is necessary to correct the third image corresponding to the target area intercepted from the initial image. Using affine transformation, the vertices of the third image are mapped to the target vertices of the rectangle to obtain the second image. As a transformation model commonly used in the field of image graphics, affine transformation mainly describes a linear transformation between two-dimensional coordinate point pairs, which maintains the "flatness" (i.e., straight lines are still straight lines after transformation) and "parallelism" (i.e., parallel lines are still parallel lines after transformation) of two-dimensional graphics.

[0117] In some embodiments, the third image includes horizon data on the display interface. The horizon data is used as a reference. If the horizon is not horizontal, it means that the content in the third image is not horizontal. Using affine transformation, the content in the third image is adjusted to be horizontal, that is, the area where the horizon is located is corrected, and the third image is corrected. Here, it is also possible to adjust the third image directly without relying on the horizon data, and use affine transformation to map the vertices of the third image to the target vertices of the rectangle to obtain the second image.

[0118] In this way, the image can be corrected using affine transformation, eliminating the error in camera angle and the influence of lens distortion, and preventing the problem of digital information tilt.

[0119] Based on this, in some embodiments, before the above S112, the method may further include:

[0120] Gray-scaling the second image to obtain a gray-scale image;

[0121] Binarize the grayscale image to obtain a binary image;

[0122] De-noise the binary image to obtain a smooth image;

[0123] The above S112 may specifically include:

[0124] A first image at a preset position in the smoothed image is captured.

[0125] In some embodiments, the second image is grayscaled and the obtained grayscale image is binarized using the Otsu method (OTSU) to simplify subsequent data processing.

[0126] In some embodiments, when an industrial camera collects an image, a large number of noise points are generated. Bilateral filtering and opening operations are used to denoise the binary image to obtain a smooth image. Bilateral filtering and opening operations are used to denoise the binary image, while removing small noise points, preserving the image edge information and smoothing the image.

[0127] In this way, the captured image is converted into a binary image, the categories of the preset base template images can be reduced, saving resources; and the influence of color is eliminated, the noise points caused by the image acquisition device are removed, and the first image at a preset position in the smooth image is intercepted. When calculating the similarity between the first image and the second base template image and determining the digital information of the first image, the influence of the noise points is eliminated, thereby improving the accuracy of digital information recognition.

[0128] Based on this, in some embodiments, in the above S120, dividing the first base template image to obtain a plurality of second base template images may include:

[0129] A sliding window is established on the first base template image, and the sliding window is moved according to a preset step length to divide the first base template image into a plurality of second base template images, and the size of the sliding window is the size of the first image.

[0130] In some embodiments, a sliding window is established on the first base template image, and the size of the sliding window is the size of the first image. The sliding window can slide from top to bottom on the base template with a specified pixel as a step size. With each step forward, a second base template image is obtained. The similarity between the first image and the area in the sliding window is calculated. The target base template image with the highest similarity on the first base template image is selected, and the digital information of the target position of the target base template image is determined to be the digital information in the first image.

[0131] As an example, the sliding window width is the template pixel width, and the sliding window height is the pixel height of 2 characters in the template.

[0132] In this way, by setting the sliding window, the first base template image is accurately divided, thereby improving the efficiency and consistency of the division of the first base template image.

[0133] Based on this, in some embodiments, the above S130 may specifically include:

[0134] Calculating the Hu invariant moment of the first image and the Hu invariant moment of a plurality of second base template images;

[0135] The similarity between the first image and the plurality of second base template images is determined according to the difference between the Hu invariant moment of the first image and the Hu invariant moment of the plurality of second base template images.

[0136] In some embodiments, due to the width of the sliding window, the aspect ratio of the digital information is different from that of the digital information in the first image, and the traditional similarity calculation method is difficult to measure accurately. However, the Hu moment can still maintain the invariance of the moment after image rotation, scaling, translation and other operations, so the Hu invariant moment is used as the similarity metric. The 7 Hu invariant moments of the first image and the 7 Hu invariant moments of the multiple second base template images are calculated respectively. The difference between the Hu invariant moment of the first image and the Hu invariant moment of the multiple second base template images is calculated respectively, one Hu invariant moment corresponds to one difference, the squares of the 7 differences are calculated and summed, and the similarity between the first image and the multiple second base template images is obtained.

[0137] In this way, by using the Hu invariant moment of the image to calculate the similarity between the first image and the plurality of second base template images, digital information in the first image can be quickly and accurately identified.

[0138] In the embodiments provided in this application, Figure 6 As shown, the specific process of the method for determining digital information includes:

[0139] S210, acquiring an image acquired by an image acquisition device;

[0140] S220, extracting features from the initial image using the target neural network model to determine a third image corresponding to the target area;

[0141] S230, using affine transformation, mapping the vertices of the third image to the target vertices of the rectangle to obtain a second image;

[0142] S240, graying the second image to obtain a grayscale image;

[0143] S250, binarizing the grayscale image to obtain a binarized image;

[0144] S260, denoising the binary image to obtain a smoothed image;

[0145] S270, intercepting a first image at a preset position in the smoothed image;

[0146] S280, combining the hu invariant moment and the base template sliding window to determine the digital information in the first image.

[0147] The embodiment of the present application provides a method based on machine vision and image processing, which replaces the human eye to identify the digital information on the display interface of the aircraft cockpit display system in real time, solves the problem of accurate recognition of digital information in the cockpit display system, thereby improving the efficiency and reliability of the automated test of the cockpit display system and reducing labor costs. The recognition process includes positioning (intercepting the first image) and recognition (calculating similarity based on the base template).

[0148] In the embodiment provided by the present application, by combining Hu invariant moment and base template sliding window, digital information can be quickly and accurately identified. The embodiment provided by the present application proposes a recognition method based on Hu invariant moment and base template sliding window, positioning the image to the area where the digital information is located, and identifying the digital information, which can accurately identify the digital information in real time. Compared with the long recognition time of deep learning tools, the recognition time of the embodiment provided by the present application is greatly shortened, and both accuracy and speed meet the test requirements of avionics systems.

[0149] In some embodiments, Figure 7 The interactive process of digital information recognition is shown. Figure 7 As shown, when identifying digital information, the high frame rate industrial camera 310 is aimed at the display interface of the aircraft cockpit display system, capturing the image in real time, and transmitting it to the image recognition device 320 via the avionics network. The image recognition device 320 identifies the digital information through the digital information determination method provided in the embodiment of the present application, and returns it to the avionics system test tool 330.

[0150] Based on the digital information determination method provided in the above embodiment, the present application also provides a specific implementation of the digital information determination device. Please refer to the following embodiment.

[0151] See first Figure 8 , the digital information determination device 400 provided in the embodiment of the present application includes:

[0152] An acquisition module 410 is used to acquire a first image and a first base template image, where the first base template image includes preset digital information;

[0153] A division module 420 is used to divide the first base template image to obtain a plurality of second base template images and positions of the plurality of second base template images in the first base template image, wherein the size of the plurality of second base template images is the size of the first image;

[0154] A calculation module 430, used to calculate the similarity between the first image and the plurality of second base template images respectively;

[0155] A selection module 440 is used to select a target base template image with the greatest similarity from a plurality of second base template images;

[0156] The determination module 450 is used to determine that the digital information of the target base template image at the target position in the first base template image is the digital information in the first image according to the relationship information between the preset digital information and the position on the first base template image.

[0157] Based on this, in some embodiments, the acquisition module 410 may include:

[0158] An acquisition submodule, used to acquire a second image corresponding to a target area of ​​a display interface;

[0159] The capture submodule is used to capture the first image at a preset position in the second image.

[0160] Based on this, in some embodiments, the acquisition submodule may include:

[0161] An acquisition unit, used for acquiring an image acquired by an image acquisition device;

[0162] An adjusting unit, used for adjusting the size of the image acquired by the image acquisition device to a target size to obtain an initial image;

[0163] An extraction unit, used for extracting features from the initial image to obtain a target feature image;

[0164] A determination unit, used to determine a target area of ​​the display interface where a target feature value in the target feature image is located;

[0165] The interception unit is used to intercept a second image corresponding to the target area from the initial image.

[0166] Based on this, in some embodiments, the extraction unit can be specifically used to:

[0167] Inputting the initial image into the target neural network model, and using the target neural network model to extract features of the initial image to obtain a plurality of first feature images;

[0168] Adjusting the sizes and the number of channels of the plurality of first feature images to be consistent to obtain a plurality of second feature images;

[0169] The multiple second characteristic images are superimposed to obtain a target characteristic image.

[0170] Based on this, in some embodiments, the interception unit can be specifically used to:

[0171] intercepting a third image corresponding to the target area from the initial image;

[0172] The vertices of the third image are mapped to the target vertices of the rectangle by using an affine transformation to obtain the second image.

[0173] Based on this, in some embodiments, the acquisition module 410 may further include:

[0174] A determination submodule is used to grayscale the second image to obtain a grayscale image before intercepting the first image at a preset position in the second image;

[0175] The determination submodule is also used to binarize the grayscale image to obtain a binary image;

[0176] The determination submodule is also used to denoise the binary image to obtain a smoothed image;

[0177] The interception submodule can be used specifically for:

[0178] A first image at a preset position in the smoothed image is captured.

[0179] Based on this, in some embodiments, the partitioning module 420 may be specifically used to:

[0180] A sliding window is established on the first base template image, and the sliding window is moved according to a preset step length to divide the first base template image into a plurality of second base template images, and the size of the sliding window is the size of the first image.

[0181] Based on this, in some embodiments, the calculation module 430 may be specifically used for:

[0182] Calculating the Hu invariant moment of the first image and the Hu invariant moment of a plurality of second base template images;

[0183] The similarity between the first image and the plurality of second base template images is determined according to the difference between the Hu invariant moment of the first image and the Hu invariant moment of the plurality of second base template images.

[0184] The various modules of the digital information determination device provided in the embodiment of the present application can realize the functions of the various steps of the digital information determination method provided above and achieve the corresponding technical effects, which will not be described in detail here for the sake of brevity.

[0185] Based on the same inventive concept, an embodiment of the present application also provides an electronic device.

[0186] Fig. 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0187] The electronic device may include a processor 901 and a memory 902 storing computer program instructions.

[0188] Specifically, the processor 901 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0189] The memory 902 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 902 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 902 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 902 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 902 is a non-volatile solid-state memory.

[0190] The memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present disclosure.

[0191] The processor 901 implements any one of the methods for determining digital information in the above embodiments by reading and executing computer program instructions stored in the memory 902 .

[0192] In one example, the electronic device may further include a communication interface 903 and a bus 910. Fig. 9 As shown, the processor 901, the memory 902, and the communication interface 903 are connected via a bus 910 and communicate with each other.

[0193] The communication interface 903 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0194] The bus 910 includes hardware, software or both, coupling the components of the electronic device to each other. For example, but not limitation, the bus may include an accelerated graphics port (Accelerated Graphics Port, AGP) or other graphics bus, an enhanced industry standard architecture (Extended Industry Standard Architecture, EISA) bus, a front side bus (Front Side Bus, FSB), a hypertransport (Hyper Transport, HT) interconnection, an industry standard architecture (Industry Standard Architecture, ISA) bus, an infinite bandwidth interconnection, a low pin count (Linear Predictive Coding, LPC) bus, a memory bus, a micro channel architecture (MicroChannel Architecture, MCA) bus, a peripheral component interconnect (Peripheral Component Interconnect, PCI) bus, a PCI-Express (Peripheral Component Interconnect-X, PCI-X) bus, a serial advanced technology attachment (Serial Advanced Technology Attachment, SATA) bus, a video electronics standard association local (VESA Local Bus, VLB) bus or other suitable bus or a combination of two or more of these. Where appropriate, the bus 910 may include one or more buses. Although the embodiments of the present application describe and illustrate a specific bus, the present application considers any suitable bus or interconnection. The electronic device can execute the method for determining digital information in the embodiments of the present invention, thereby implementing the above-mentioned method for determining digital information.

[0195] In addition, in combination with the method for determining digital information in the above embodiments, the present application embodiment may provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the methods for determining digital information in the above embodiments is implemented.

[0196] The present application also provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes each process of implementing any one of the above-mentioned digital information determination method embodiments.

[0197] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0198] The functional blocks shown in the structural block diagram described above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), appropriate firmware, plug-in, function card, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or communication link by a data signal carried in a carrier. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memory (Read-Only Memory, ROM), flash memory, erasable read-only memory (Erasable ReadOnly Memory, EROM), floppy disks, compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical discs, hard disks, optical fiber media, radio frequency (Radio Frequency, RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0199] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0200] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0201] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A method for determining digital information, characterized in that: include: Acquire a first image and a first base template image, wherein the first base template image includes preset digital information; Dividing the first base template image to obtain a plurality of second base template images and positions of the plurality of second base template images in the first base template image, wherein the size of the plurality of second base template images is the size of the first image; respectively calculating the similarities between the first image and a plurality of the second base template images; Selecting the target base template image with the greatest similarity from the plurality of second base template images; According to the relationship information between the preset digital information and the position on the first base template image, the digital information of the target base template image at the target position in the first base template image is determined to be the digital information in the first image.

2. The method for determining digital information according to claim 1, characterized in that: The acquiring of the first image comprises: Acquire a second image corresponding to the target area of ​​the display interface; The first image at a preset position in the second image is captured.

3. The method for determining digital information according to claim 2, characterized in that: The acquiring a second image corresponding to the target area of ​​the display interface includes: Acquire an image acquired by an image acquisition device; Adjusting the size of the image captured by the image acquisition device to a target size to obtain an initial image; Performing feature extraction on the initial image to obtain a target feature image; Determine a target area of ​​the display interface where a target feature value in the target feature image is located; A second image corresponding to the target area is captured from the initial image.

4. The method for determining digital information according to claim 3, characterized in that: The step of extracting features from the initial image to obtain a target feature image includes: Inputting the initial image into a target neural network model, and using the target neural network model to perform feature extraction on the initial image to obtain a plurality of first feature images; Adjusting the sizes and the number of channels of the plurality of first feature images to be consistent to obtain a plurality of second feature images; A plurality of the second characteristic images are superimposed to obtain a target characteristic image.

5. The method for determining digital information according to claim 3, characterized in that: The step of intercepting a second image corresponding to the target area from the initial image includes: intercepting a third image corresponding to the target area from the initial image; Using affine transformation, the vertices of the third image are mapped to target vertices of the rectangle to obtain a second image.

6. The method for determining digital information according to claim 2, characterized in that: Before intercepting the first image at a preset position in the second image, the method further includes: gray-scale the second image to obtain a gray-scale image; Binarizing the grayscale image to obtain a binary image; De-noising the binary image to obtain a smoothed image; The intercepting the first image at a preset position in the second image comprises: A first image at a preset position in the smoothed image is captured.

7. The method for determining digital information according to claim 1, characterized in that: The step of dividing the first base template image to obtain a plurality of second base template images comprises: A sliding window is established on the first base template image, and the sliding window is moved according to a preset step length to divide the first base template image into a plurality of second base template images, wherein the size of the sliding window is the size of the first image.

8. The method for determining digital information according to claim 1, characterized in that: The respectively calculating the similarities between the first image and the plurality of second base template images comprises: Calculating the Hu invariant moment of the first image and the Hu invariant moments of a plurality of the second base template images; The similarity between the first image and the plurality of second base template images is determined according to the difference between the Hu invariant moments of the first image and the Hu invariant moments of the plurality of second base template images.

9. A device for determining digital information, characterized in that: include: An acquisition module, used to acquire a first image and a first base template image, wherein the first base template image includes preset digital information; a division module, configured to divide the first base template image to obtain a plurality of second base template images and positions of the plurality of second base template images in the first base template image, wherein the size of the plurality of second base template images is the size of the first image; A calculation module, used for respectively calculating the similarity between the first image and a plurality of the second base template images; A selection module, used for selecting the target base template image with the greatest similarity from a plurality of the second base template images; A determination module is used to determine that the digital information of the target base template image at the target position in the first base template image is the digital information in the first image according to the relationship information between the preset digital information and the position on the first base template image.

10. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the method for determining digital information according to any one of claims 1 to 8 is implemented.

Images