Instrument panel data determination method and device, storage medium and electronic device

Abstract

The invention discloses an instrument panel data determination method and device, a storage medium and an electronic device.The method comprises the steps that an inspection image from a user interaction interface is received, and the inspection image is obtained after image collection is conducted on an inspection area; determining an identification result obtained by performing image identification on the inspection image by using a target identification model, and determining an instrument image of an instrument panel in the inspection area from the identification result; and determining a target instrument program corresponding to the instrument type of the instrument panel from preset instrument programs, and performing image processing on the instrument image according to an image recognition algorithm corresponding to the target instrument program to obtain instrument panel data of the instrument panel. By adopting the technical scheme, the problem that the accuracy of the instrument panel data is relatively low when the instrument panel data is automatically read is solved.

Description

Technical Field

[0001] The present application relates to the technical field of instrument identification, and in particular to a method and device for determining instrument panel data, a storage medium and an electronic device. Background Art

[0002] In modern industry and transportation, dashboards are an important part of vehicles, ships, airplanes and other transportation tools and various equipment. The dashboard integrates various instruments and indicator lights to display key parameters such as speed, temperature, pressure, fuel level, etc. that reflect the operating status of transportation tools or equipment in real time.

[0003] Generally, the parameters displayed on a physical instrument panel or a digital instrument panel are read manually. For example, for a physical instrument panel, the data displayed by the pointer and the dial can be read. For a digital instrument panel, the data displayed by the liquid crystal display or the organic light-emitting diode can be read directly. However, the manual reading method is inefficient, so a method for automatically identifying instrument panel data is proposed in the related art, such as using computer vision methods to identify key parameters in an instrument panel image. Alternatively, intelligent models such as convolutional networks are used to identify the position of the instrument panel in the instrument panel image and specific elements on the instrument panel, such as the speedometer and fuel gauge, but the recognition results of this automatic recognition are not accurate enough and it is easy to read the wrong data. Therefore, when automatically reading instrument panel data, there is a technical problem of low accuracy of instrument panel data.

[0004] In the related art, there is no effective solution to the problem of low accuracy of dashboard data when the dashboard data is automatically read.

[0005] Therefore, it is necessary to improve the related technology to overcome the above-mentioned defects in the related technology. Summary of the invention

[0006] The embodiments of the present application provide a method and device for determining dashboard data, a storage medium, and an electronic device to at least solve the problem of low accuracy of dashboard data when the dashboard data is automatically read.

[0007] According to one aspect of an embodiment of the present application, a method for determining dashboard data is provided, comprising: receiving an inspection image from a user interaction interface, wherein the inspection image is obtained after image acquisition of an inspection area; determining a recognition result of image recognition of the inspection image using a target recognition model, and determining an instrument image of a dashboard in the inspection area from the recognition result; determining a target instrument program corresponding to the instrument type of the dashboard from preset instrument programs, and performing image processing on the instrument image according to an image recognition algorithm corresponding to the target instrument program to obtain dashboard data of the dashboard.

[0008] In an exemplary embodiment, before determining the recognition result of performing image recognition on the inspection image using the target recognition model, the method also includes: determining a training image from a pre-generated training data set, performing an image enhancement operation on the training image to obtain an enhanced image; splicing sub-images obtained after cropping the enhanced image to obtain a new training image, wherein the training image and the new training image have the same image label; determining a new training data set based on multiple new training images, and using the new training data set to train the initial recognition model to obtain the target recognition model.

[0009] In an exemplary embodiment, the instrument image is processed according to the image recognition algorithm corresponding to the target instrument program to obtain the instrument panel data of the instrument panel, including: performing image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result, the recognition result including at least one of the following: pointer data of the instrument panel, scale data of the instrument panel and character data of the instrument panel; determining the instrument panel data using the pointer data, the scale data and the character data.

[0010] In an exemplary embodiment, image recognition is performed on the instrument image according to the image recognition algorithm to obtain a recognition result, including: using the image recognition algorithm to perform Hough circle detection on the instrument image to identify the dial outline in the instrument image; determining the circumscribed rectangle of all dial outlines, and filtering out the pointer outline from all dial outlines according to the geometric information of the circumscribed rectangle; generating a mask image corresponding to the pointer outline, performing morphological operations on the first mask image to obtain a first morphological image, and obtaining different Hough line segments obtained by performing Hough line detection on the first morphological image, determining the Hough line segment with the maximum line segment length from the different Hough line segments, and determining the pointer data according to the fitted line segments of the Hough line segments.

[0011] In an exemplary embodiment, image recognition is performed on the instrument image according to the image recognition algorithm to obtain a recognition result, including: converting the image format of the instrument image into a brightness display format, wherein the pointer of the instrument panel has a first brightness value, and the dial of the instrument panel has a second brightness value; for a first pointer whose line segment width is greater than a preset width, determining the pointer contour of the first pointer, performing least squares fitting on the first pointer according to an external matrix of the pointer contour of the first pointer to obtain a fitted straight line, and determining the pointer data based on the fitted straight line; for a second pointer whose line segment width is less than a preset width, determining the pointer contour of the second pointer, determining a target contour point from the pointer contour of the second pointer, and determining the pointer data based on a line between the target contour point and the center of a Hough circle for performing Hough circle detection, wherein the target contour point is farthest from the Hough circle.

[0012] In an exemplary embodiment, image recognition is performed on the instrument image according to the image recognition algorithm to obtain a recognition result, including: determining a character recognition model used by the image recognition algorithm, the character recognition model including a text recognition model; inputting the instrument image into the text recognition model to obtain a text sequence output by the text recognition model, and determining the character data based on the text characters in the text sequence.

[0013] In an exemplary embodiment, the method further includes: inputting the instrument image into the text positioning model to obtain the text position output by the text positioning model, wherein the text characters in the text sequence all correspond to text positions; when it is determined that the text character is consistent with a preset starting character, obtaining the first text position of the text character, and determining the vector between the center point of the circumscribed rectangle where the first text position is located and the center of the instrument panel as the starting vector; when it is determined that the text character is consistent with a preset ending character, obtaining the second text position of the text character, and determining the vector between the center point of the circumscribed rectangle where the second text position is located and the center of the instrument panel as the ending vector; and determining the starting position and ending position of the pointer of the instrument panel based on the starting vector and the ending vector.

[0014] According to another aspect of an embodiment of the present application, a device for determining dashboard data is also provided, including: a receiving module for receiving an inspection image from a user interaction interface, wherein the inspection image is obtained after image acquisition of an inspection area; a determining module for determining a recognition result of image recognition of the inspection image using a target recognition model, and determining an instrument image of the dashboard of the inspection area from the recognition result; an obtaining module for determining a target instrument program corresponding to the instrument type of the dashboard from a preset instrument program, and performing image processing on the instrument image according to an image recognition algorithm corresponding to the target instrument program, to obtain dashboard data of the dashboard.

[0015] According to another aspect of the embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for determining dashboard data when running.

[0016] According to another aspect of an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for determining the dashboard data through the computer program.

[0017] According to another aspect of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0018] Through this application, by receiving the inspection image from the user interaction interface, wherein the inspection image is obtained after image acquisition of the inspection area; determining the recognition result of the inspection image using the target recognition model, and determining the instrument image of the instrument panel of the inspection area from the recognition result; determining the target instrument program corresponding to the instrument type of the instrument panel from the preset instrument program, and performing image processing on the instrument image according to the image recognition algorithm corresponding to the target instrument program, the instrument panel data of the instrument panel is obtained. It not only solves the problem of low accuracy of the instrument panel data when automatically reading the instrument panel data, but also improves the accuracy of the instrument panel data, thereby realizing more accurate monitoring, predictive maintenance and fault diagnosis of the equipment or vehicle installed with the instrument panel, and improving safety and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] Figure 1 It is a hardware structure block diagram of a mobile terminal of a method for determining instrument panel data in an embodiment of the present application;

[0022] Figure 2 is a flow chart of a method for determining dashboard data according to an embodiment of the present application;

[0023] Figure 3 is a flow chart of a method for determining dashboard data according to an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of an operation flow of an instrument program according to an embodiment of the present application;

[0025] Figure 5 It is a structural block diagram of a device for determining dashboard data according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal of a method for determining dashboard data in an embodiment of the present application. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor (Central Processing Unit, MCU) or a programmable logic device (Field Programmable Gate Array, FPGA)) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0029] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the dashboard data in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, a method for determining dashboard data is provided. Figure 2 is a flow chart of a method for determining dashboard data according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps:

[0032] Step S202: receiving an inspection image from a user interaction interface, wherein the inspection image is obtained after image acquisition of an inspection area;

[0033] Optionally, you can use PyQt to design a GUI interactive interface (i.e. user interactive interface) for instrument identification. The specific process may include:

[0034] 1. You need to install the PyQt library. You can install PyQt5 through pip, the command is: pip install PyQt5.

[0035] 2. Import PyQt5 modules in the Python script, for example: from PyQt5.QtWidgets importQApplication,QWidget,QVBoxLayout,QPushButton,QLabel,QFileDialog, etc.

[0036] 3. Use PyQt's layout manager (such as QVBoxLayout) to organize interface elements such as buttons, labels, and input boxes.

[0037] 4. Add interface elements as needed, such as buttons (QPushButton), labels (QLabel), input boxes (QLineEdit), etc. Set properties for each element, such as text, size, and position.

[0038] 5. Add event handling functions to interface elements, such as button click events. Implement instrument recognition logic in these functions, such as reading images, identifying instruments, and displaying results.

[0039] 6. Create a QWidget as the main window and set the window title, size and other properties.

[0040] 7. Create a QApplication instance and start the event loop. Display the main window and wait for user operation.

[0041] Optionally, the user interaction interface includes a file upload module, a type matching module and a result display module. The file upload module supports users to upload multiple pictures. The type matching module can automatically match the instrument type and call the relevant detection program (i.e., the target instrument program) to output the running results of the detection program, which at least includes the detected dashboard data.

[0042] Step S204: determining a recognition result of performing image recognition on the inspection image using a target recognition model, and determining an instrument image of an instrument panel in the inspection area from the recognition result;

[0043] Step S206: determining a target instrument program corresponding to the instrument type of the instrument panel from preset instrument programs, and performing image processing on the instrument image according to an image recognition algorithm corresponding to the target instrument program to obtain instrument panel data of the instrument panel.

[0044] Optionally, the instrument type of the instrument panel includes a disc type and a non-disc type. For the disc type, the pointer, line segment and character in the instrument panel are identified in sequence, and for the non-disc type, the instrument panel outline needs to be identified from the instrument image first, and then the pointer, line segment and character in the instrument panel are identified in sequence.

[0045] Through the above steps, by receiving the inspection image from the user interaction interface, wherein the inspection image is obtained after image acquisition of the inspection area; determining the recognition result of the inspection image using the target recognition model, and determining the instrument image of the instrument panel of the inspection area from the recognition result; determining the target instrument program corresponding to the instrument type of the instrument panel from the preset instrument program, and performing image processing on the instrument image according to the image recognition algorithm corresponding to the target instrument program, to obtain the instrument panel data of the instrument panel. It not only solves the problem of low accuracy of the instrument panel data when automatically reading the instrument panel data, but also improves the accuracy of the instrument panel data, thereby realizing more accurate monitoring, predictive maintenance and fault diagnosis of the equipment or vehicles installed with the instrument panel, and improving safety and efficiency.

[0046] In this application, a data set of instruments can be constructed first, and the data set can be further trained based on the existing weights of the target recognition model YOLOv7 to improve the accuracy of subsequent recognition of the model. After receiving the inspection image, YOLOv7 is used to locate the position of the instrument panel from the inspection image, and after extracting the instrument image of the instrument panel, the dial area of ​​the instrument panel is read.

[0047] In the process of image recognition, the recognition of pointers, scales and characters is mainly involved. For pointers and scales, two methods, Hough line detection and color segmentation, are proposed for recognition, which can improve the recognition accuracy. For characters, the inspection image needs to be input into a pre-trained OCR model, and the position of the starting scale is located according to the image information and text sequence output by the model. Among them, the OCR model can be trained with file data of multiple characters.

[0048] Optionally, the present application also uses Pyqt to design an interactive interface for instrument identification. This interactive interface can receive batch uploads of corresponding pictures by users, automatically call the corresponding instrument identification program according to the instrument type, automatically identify the dashboard data and display the results.

[0049] Optionally, the instruments can be specifically divided into oil thermometers, oil level gauges, lightning arresters and on-load tap-changing transformers according to the application scenarios of the instruments in the inspection images and the different contents of the dials, and corresponding algorithm processes can be designed for different instruments.

[0050] In an exemplary embodiment, before determining the recognition result of image recognition of the inspection image using the target recognition model, the following steps can also be implemented: determine a training image from a pre-generated training data set, perform image enhancement operation on the training image to obtain an enhanced image; splice the sub-images obtained after cropping the enhanced image to obtain a new training image, wherein the training image and the new training image have the same image label; determine a new training data set based on multiple new training images, and use the new training data set to train the initial recognition model to obtain the target recognition model. This embodiment proposes a method for training a target recognition model. During the training process, the training data set is optimized to further improve the recognition accuracy of the model.

[0051] The image enhancement operation includes flipping, rotating, shifting, etc. The above initial recognition model is, for example, a basic model such as a convolutional neural network.

[0052] In one embodiment, the training process of the recognition model is specifically described through the following process. First, a data set of instruments is constructed, and the instrument images therein are subjected to processing operations including flipping, rotation, and shifting, so as to achieve single-sample data enhancement, and then the images in the training set are cropped and spliced ​​to synthesize a new training sample. At this time, the label of the new training sample is still the label of the original training sample, thereby achieving multi-sample data enhancement. After using the new training set for model training, a new weight file can be obtained, which is then used to perform reasoning on new images. During the reasoning process, the bounding box and other information of the identified target are written into the json file to facilitate the subsequent image segmentation processing.

[0053] In an exemplary embodiment, the process of performing image processing on the instrument image according to the image recognition algorithm corresponding to the target instrument program to obtain the instrument panel data of the instrument panel specifically includes: performing image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result, the recognition result includes at least one of the following: pointer data of the instrument panel, scale data of the instrument panel and character data of the instrument panel; determining the instrument panel data using the pointer data, scale data and character data. Through this embodiment, it is proposed to recognize the pointer, scale and character respectively in the process of image recognition, thereby improving the accuracy of instrument panel recognition.

[0054] In an exemplary embodiment, other implementation schemes are further proposed for performing image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result, specifically including: using the image recognition algorithm to perform Hough circle detection on the instrument image to identify the dial outline in the instrument image; determining the circumscribed rectangle of all dial outlines, and filtering out the pointer outline from all dial outlines according to the geometric information of the circumscribed rectangle; generating a mask image corresponding to the pointer outline, performing morphological operations on the first mask image to obtain a first morphological image, and obtaining different Hough line segments obtained by performing Hough line detection on the first morphological image, determining the Hough line segment with the maximum line segment length from the different Hough line segments, and determining the pointer data according to the fitted line segments of the Hough line segments.

[0055] The geometric information of the circumscribed rectangle includes, for example, the aspect ratio and the center point position.

[0056] In one embodiment, specific processing steps using Hough line detection are proposed. First, according to the image position information, the ROI area is extracted to obtain the instrument image, and the Hough circle detection is performed on the instrument image after image preprocessing, and then the center position and radius of the dial are extracted, and a mask image corresponding to the dial is generated to ensure that the area detected by the Hough line is within the dial.

[0057] The image preprocessing process specifically includes grayscale processing, Gaussian filtering and Canny edge detection.

[0058] Then, after performing Canny edge detection on the mask image, a contour set is obtained, and the circumscribed rectangle corresponding to the contour is drawn. The contour is preliminarily screened according to the length, width and center point position of the circumscribed rectangle, and the scale contour and pointer contour are retained.

[0059] After extracting the corresponding contours, mask images of the scale and pointer are generated respectively (equivalent to the above-mentioned mask image), and further morphological operations such as dilation and corrosion are performed to fill small holes where pixels are missing in the mask image and eliminate some noise.

[0060] Next, perform Hough line detection on the mask images of the scale and pointer respectively, and perform length screening on the obtained approximate Hough lines. For example, when identifying the pointer, the length screening can be further used to obtain the fitting straight line of the pointer. Among them, the longest and second longest ones are retained for the dual-pointer instrument, and the longest one is retained for the single-pointer instrument.

[0061] After the Hough line detection, the screening process of the approximate Hough lines may include: arranging the identified Hough lines in descending order by length, then fitting the straight line equations of each line segment after comparing each line segment backwards in turn, merging the line segments, and only retaining the line segments where the distance from the straight line to the center of the circle is less than a certain threshold, otherwise, it is not the pointer position.

[0062] Optionally, use length for filtering as follows:

[0063] 1. If the slopes of two lines are similar and the distance between the two lines is less than a certain set value, they need to be filtered. The reason for filtering at this time is that for some pointers with wider pixels, one pointer may be identified as multiple similar straight lines, and these similar straight lines need to be merged.

[0064] 2. If two straight lines intersect and the lengths of the two line segments are significantly different, they need to be filtered. The reason for filtering is that after the morphological operation, some noise points may still exist, causing the original pointer part to be connected to the noise points and be recognized as a straight line.

[0065] After screening and merging, the remaining straight lines can be screened again, and the remaining line segments can be extended. If the straight line where the line segment is located is within a certain distance from the center of the circle, the straight line is retained, otherwise it is discarded.

[0066] It should be noted that, in the present application, the method for identifying the scale can refer to the method for identifying the pointer, which will not be repeated here.

[0067] In an exemplary embodiment, for performing image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result, the following implementation steps may also be included: converting the image format of the instrument image into a brightness display format, wherein the pointer of the instrument panel has a first brightness value, and the dial of the instrument panel has a second brightness value; for a first pointer whose line segment width is greater than a preset width, determining the pointer contour of the first pointer, performing least squares fitting on the first pointer according to an external matrix of the pointer contour of the first pointer to obtain a fitted straight line, and determining the pointer data based on the fitted straight line; for a second pointer whose line segment width is less than a preset width, determining the pointer contour of the second pointer, determining a target contour point from the pointer contour of the second pointer, and determining the pointer data based on a line between the target contour point and the center of a Hough circle for performing Hough circle detection, wherein the target contour point is farthest from the Hough circle.

[0068] In one embodiment, the steps of using the color recognition method to identify the pointer are as follows: first, according to the dial color and the pointer color, the dial image is converted into the HSV color space, the upper and lower limits of the target color are set, and the pointer area is segmented according to the target color. Among them, the display format of the image in the HSV color space corresponds to the above-mentioned brightness display format.

[0069] Then, for the thicker pointer, a straight line fitting is performed directly. Specifically, the thicker pointer is fitted with the least squares method according to the circumscribed rectangle of the contour, and the fitted straight line is used as the pointer. For the thinner pointer, the points on the pointer contour can be traversed, and the point on the contour farthest from the previously detected Hough circle is used as the far point of the pointer. The line connecting the far point and the center of the circle is the pointer.

[0070] In an exemplary embodiment, further, the process of performing image recognition on the instrument image according to the image recognition algorithm to obtain the recognition result may also include: determining a character recognition model used by the image recognition algorithm, the character recognition model including a text recognition model; inputting the instrument image into the text recognition model to obtain a text sequence output by the text recognition model, and determining the character data based on the text characters in the text sequence.

[0071] The character recognition model may specifically include a text positioning model and a text recognition model. The character recognition model may be used to recognize characters and determine the situation where the starting point of the pointer is not at the center of the disk.

[0072] Among them, the text positioning model is, for example, the CnSTD (Chinese Scene Text Detection) model, and the text recognition model is, for example, the CnOCR (Chinese Optical Character Recognition) model. The two models, CnSTD and CnOCR, are usually used in combination to implement the complete process of extracting and recognizing Chinese text from an image. First, CnSTD locates the text area in the image; then, CnOCR recognizes the characters in these areas. This combination can be applied to a variety of practical scenarios, such as automatic translation, information extraction, and traffic sign recognition for autonomous vehicles.

[0073] In an exemplary embodiment, further, the instrument image is input into the text positioning model to obtain the text position output by the text positioning model, wherein the text characters in the text sequence all correspond to text positions; when it is determined that the text character is consistent with a preset starting character, the first text position of the text character is obtained, and the vector between the center point of the circumscribed rectangle where the first text position is located and the center of the instrument panel is determined as the starting vector; when it is determined that the text character is consistent with a preset ending character, the second text position of the text character is obtained, and the vector between the center point of the circumscribed rectangle where the second text position is located and the center of the instrument panel is determined as the ending vector; the starting position and ending position of the pointer of the instrument panel are determined based on the starting vector and the ending vector.

[0074] Among them, the process of determining the starting vector and the ending vector can be further described in combination with the following process: if the character recognized is a starting character such as "0" or "MIN" (corresponding to the preset starting character), the corresponding character position is returned, for example, the circumscribed rectangle containing its four endpoints of upper left, lower left, upper right, and lower right, and the center point of the circumscribed rectangle and the center vector of the circle are taken as the starting vector. Similarly, if the largest number or the ending character such as "MAX" is recognized, the corresponding position of the character is returned, for example, the circumscribed rectangle containing its four endpoints of upper left, lower left, upper right, and lower right, and the center point of the circumscribed rectangle and the center vector of the circle are taken as the ending vector.

[0075] Furthermore, for the case where the straight line where the pointer is located does not pass through the center of the dial, the center points of different scale positions can be determined and identified, and the perpendicular bisectors of the line segments connecting the center points can be calculated in turn. When the straight lines where multiple perpendicular bisectors are located are intersected with the straight line where the pointer is located, several intersection points will be obtained. The average value of these intersection points is calculated to obtain the fitted pointer starting point.

[0076] Obviously, the embodiments described above are only some embodiments of the present application, not all embodiments. In order to better understand the above method, the above process is described below in conjunction with the embodiments, but it is not intended to limit the technical solutions of the embodiments of the present application, specifically:

[0077] In an optional embodiment, the present application combines Figure 3 The process shown in the figure explains the process of identifying dashboard data. The specific steps are as follows:

[0078] Step 1: Run YOLOv7 to extract the dial position and segment it.

[0079] It should be noted that, unlike the two-stage detector Faster-RCNN, YOLOv7 is a single-stage target detection algorithm based on convolutional neural network. It can directly predict the target category and bounding box position at the output layer at the same time, and divide the input image into fixed grids, where each grid unit is responsible for detecting the target in its own area and outputting the result after detection. From the perspective of the process after the image is input, the YOLOv7 network mainly includes four parts: Input, Backbone, Neck, and Head. First, after the image is pre-processed by a series of operations such as data enhancement in the input part, it is sent to the backbone network, and the backbone network extracts features from the processed image; then, the extracted features are processed by the Neck module feature fusion to obtain features of three sizes: large, medium, and small. Finally, the fused features are sent to the detection head for detection and results are obtained.

[0080] In the process of using the YOLOv7 model to extract images, we first collected relevant instrument data sets from the Internet and used Labelme to annotate the instruments. The instrument data sets mainly include disk instruments. We performed data enhancement processing on the data sets (including cropping, rotation, flipping, etc.), and then divided the data sets into training sets, test sets, and validation sets in a ratio of 8:1:1.

[0081] Store the XML file of the annotated dataset in the Annotations folder, convert the XML file with the annotation result into a txt file, which contains the bounding box position information of the target area corresponding to the image, and store it in the labels folder. Put the image under Image and the image path under data.

[0082] Run the model training file train.py, train on the basis of the original model, get new weights, modify the weights in the detection program detect.py file to the new weights, import the data set to be detected into the model, run the detection program, identify and return information such as category, location information, confidence, etc. In this information, the index target is the instrument class, return the location information and write it in a json file, and use the json file to segment the instrument before processing the instrument image.

[0083] Step 2: Determine whether the instrument panel is a disc-shaped instrument. If yes, proceed to step 3, otherwise proceed to step 4.

[0084] Step 3: Perform Hough circle detection on the instrument image to further extract the dial position.

[0085] After segmenting and extracting the dial area, if the obtained area is a rectangular area, for a disc-type instrument, Hough circle detection is required to extract the dial area.

[0086] Among them, the pre-steps of Hough circle detection usually include image grayscale operation and Canny edge detection operation.

[0087] The principle of image grayscale is to adjust the saturation of each primary color in the RGB image to zero, forming an image without color distinction, which is the grayscale image of the original image.

[0088] The principle of Canny edge detection is as follows:

[0089] (1) A Gaussian filter is used to perform Gaussian smoothing on the grayscale image to make the pixel value of each point close to the true value and remove isolated noise. The two-dimensional normal distribution formula used to determine the weight is:

[0090]

[0091] Here, σ is the standard deviation of the Gaussian distribution and is used to determine the blurriness of the filter.

[0092] (2) Gradient calculation. The Sobel operator is used to calculate the gradient intensity direction of the image for preliminary edge positioning. The Sobel operator is used to calculate the gradients in the horizontal and vertical directions respectively:

[0093]

[0094] Through the convolution operation, the horizontal gradient G is obtained x and G Y , and calculate the gradient magnitude and gradient direction as:

[0095]

[0096] (3) Non-maximum suppression: Only pixels with local gradient maxima in the region are retained and suppressed along the gradient direction. If a pixel is not the maximum value in the gradient direction, its value is suppressed to 0.

[0097] (4) Dual threshold detection. Potential edges are determined by setting thresholds. The high threshold is used to mark strong edges, and the low threshold is used to mark weak edges.

[0098] (5) Edge connection. A strong edge determined by a high threshold will be used as the final edge, while a weak edge below the high threshold but above the low threshold will also be retained as an edge point if it is connected to a strong edge.

[0099] The principle of Hough circle detection is to map the edge points in the image to the parameter space and determine the parameters of the circle through accumulator voting. The standard equation of the circle is:

[0100] (x) 2 +(yb) 2 =r 2 .

[0101] In Hough circle detection, edge points (x, y) are calculated by the formula

[0102]

[0103] Calculate the possible center coordinates (a, b). The accumulator is used to record the votes for each possible combination (a, b, r). Finally, the peak value in the accumulator represents the detected circle. Just like the previous step, Canny detection is performed to obtain edge points, and then voting in the parameter space is performed to detect circular targets in the image. In this way, the dial area is extracted, and based on the position information, the dial area is further intercepted from the original image to create the corresponding mask image.

[0104] Step 4: After preprocessing the image, perform contour recognition and conduct preliminary contour screening based on the position and aspect ratio of the contour.

[0105] Specifically, the preprocessing process of grayscale conversion and image threshold adaptive binarization is first performed. Then contour recognition is performed, and the bounding rectangles of all recognized contours are drawn. According to the aspect ratio and center point position of the bounding rectangle, the contours are screened, and the areas that may be scales or pointers are identified to facilitate the next step of screening.

[0106] Among them, the principle of binarization is: calculate the mean of the grayscale values ​​of all pixels in the pixel matrix in the grayscale image. In this application, the mean is calculated using the Gaussian mean. Similar to the Gaussian filtering process, the adaptive threshold of a pixel is equal to the weighted weight calculated according to the two-dimensional normal distribution in its neighborhood. The part of the pixel value greater than the threshold is set to 255, and the part less than the threshold is set to 0 to generate a binary image.

[0107] Step 5: Use Hough line detection to identify the pointer and scale lines.

[0108] Step 6: Filter the line segments based on the slope, the distance between the straight lines where the line segments are located, and the position of the line segments.

[0109] After identifying the scale and pointer, the mask images of the two are generated respectively. For the mask image of the pointer, the lines in the grayscale image are detected based on HoughlinesP, and all the line segments contained in the mask image are detected. After cross-screening the detected line segments, they are sorted in descending order by length. The longest one is generally the pointer line segment. The conditions for screening the line segments are as follows:

[0110] (1) The slopes of the two lines are similar, and the distance between the two lines is less than a certain set value. This screening is because: for some pointers with wider pixels, one pointer may be identified as multiple similar straight lines, which need to be merged.

[0111] (2) Two straight lines intersect, and the lengths of the two line segments are significantly different. This screening is because: after the morphological operation, some noise points may still exist, causing the original pointer part to be connected to the noise points and be recognized as a straight line.

[0112] After screening and merging, the remaining lines are screened a second time, and the remaining line segments are extended. If the line where the line segment is located is within a certain distance from the center of the circle, the line is retained, otherwise it is discarded.

[0113] Step 7: If the recognition effect is not good, the color segmentation method is used to identify the pointer area and a straight line or least squares fit is used.

[0114] When the image pixels are not enough and the preprocessing effect is not good, there are many holes, and the morphological processing cannot fill them. Increasing the number of dilation times may cause more noise points, resulting in misrecognition. Therefore, the number of dilation processing images is limited. This may lead to the situation that the number of line segments detected by the above Hough lines is insufficient, so the color segmentation method is adopted.

[0115] Taking the dual-pointer instrument as an example, the historical maximum indication scale of the indication scale is red, and the upper and lower limits of the red value in the HSV median are pre-set. The original image is converted into the HSV space, and segmented according to the pre-set color range to generate the corresponding mask image, in which the ROI area is white and the other parts are black.

[0116] After identifying the corresponding contour, if the corresponding pointer part is thin, the Hough line detection is directly used to fit the corresponding straight line. For a wider pointer type (i.e., a nonlinear shape), the least squares method is used to fit the straight line as the straight line where the direction vector of the pointer is located.

[0117] Step 8: Use the CnSTD model and CnOCR model to identify the digital readings or dial information to determine the start and end positions of the pointer instrument.

[0118] Specifically, the existing CnSTD model and CnOCR model are used to locate and recognize numbers and English characters.

[0119] Among them, CnSTD uses CNN as a feature extractor to extract feature maps from the input image. Through convolution operations, low-level features (such as edges, corners, etc.) are extracted, and higher-level features are formed in deeper network layers to distinguish text from background. Among them, the convolution formula is:

[0120]

[0121] Among them, h i，j is the value of the output feature map, x i+m，j+n is the pixel value of the input image, w m，n is the convolution kernel, b is the bias term, and σ is the activation function.

[0122] After identifying the text area, the CnSTD model uses the bounding box regression method to correct the initial predicted text box by learning the offset of the bounding box, so that the text box is closer to the actual text prediction position. The formula for bounding box regression is:

[0123]

[0124] Among them, (r i , c i ) is the row and column position of the predicted text box, (h i , w i ) are the height and width respectively, and the characters with superscripts are the actual bounding box parameters.

[0125] Since the model may detect multiple similar text boxes in the same area, CnSTD uses non-maximum suppression (NMS) to remove redundant detection boxes. By calculating the overlap of candidate boxes, boxes with lower overlap are retained and boxes with higher overlap are removed. At the same time, the CnSTD model can identify text regions of different sizes through multi-scale processing, which improves the robustness of the model. Finally, the CnSTD model identifies the text region and returns the text box of the text region.

[0126] As described above, on the basis of CnSTD, CnOCR is used to perform character recognition.

[0127] The principle of CnOCR is to combine the convolutional neural network (CNN) and the recurrent neural network (RNN) detection, and use the CTC loss function for sequence-to-sequence training. First, the input text image is converted into a set of high-dimensional feature representations through the convolution layer and the pooling layer. For text recognition, the main function of the convolution layer is to capture the local spatial information of the characters, and the pooling layer is used for downsampling to reduce computational complexity and overfitting.

[0128] After extracting character features, CnOCR uses recurrent neural networks, especially bidirectional long short-term memory networks, to establish sequence modeling problems to capture the contextual relationship of character sequences.

[0129] Among them, the output of the recurrent neural network RNN ​​is:

[0130] h t =σ(W h *h t-1 +W x *x t +b h ).

[0131] After considering the information before and after, the output of LSTM is:

[0132] h t =LSTM f (x t-1 ,h t )+LSTM b (x,h t+1 ).

[0133] Among them, h t is the hidden state at time step t, W h and W t is the weight matrix of RNN, b h is bias.

[0134] Since the length and position of characters are not fixed, CnOCR uses the CTC loss function to solve the alignment problem. CTC allows the character sequence output by the network to not match the world target length, but instead generates an intermediate probability matrix in which each character has a probability distribution. Through decoding, the character sequence can be derived from the output probability matrix.

[0135] Through character recognition, the starting or ending position of different types of meters can be determined.

[0136] For example, in the oil thermometer, we can index the key with the text key value of '0' from the output dictionary, extract its location information, find the center point of the area outlined by its bounding box, and use the line connecting this point and the center of the circle as the starting vector.

[0137] In the oil level gauge, the key corresponding to the text value "MIN" and "MAX" is indexed from the dictionary outputted by the result, and the center point and the line connecting the center point of the circle are taken as the starting vector and the ending vector. In addition, based on the distance measurement and the scale line recognition method mentioned above, the scale line closest to the detection scale is selected as the pointing point of the starting or ending vector.

[0138] In addition, character recognition has another application scenario, that is, to determine that the scale line is an arc, but the center of the circle is not on the Hough circle, and the instrument is a single pointer, and the center of the circle cannot be found through the intersection of the pointers. For example, in the instrument type of the lightning arrester, this situation may occur, so find symmetrical characters, find the center point connection line, and draw the perpendicular bisector of the connection line. The intersection of the perpendicular bisector or the intersection of the perpendicular bisector and the straight line where the pointer is located is the center of the circle to be found.

[0139] Step 9: Calculate the reading using the angle method. Based on the above recognition process, for digital instruments, CnSTD and CnOCR can be used to directly identify the corresponding numbers. For pointer instruments, after identifying the starting vector and the direction vector on the current pointer line, calculate the angle between the vectors and multiply it by the division value corresponding to the unit scale to get the instrument reading. If it is a non-circular instrument, skip the Hough circle detection process and directly process the image obtained after YOLOv7 locates the dial segmentation. After preprocessing the image, detect the pointer and scale, recognize the characters at the same time, and use the angle method to get the reading.

[0140] In other embodiments, Figure 4 As shown, you can also identify the dashboard by following these steps:

[0141] Step S11, obtaining pictures uploaded in batches by users based on the pygt interface;

[0142] Step S12: After the inspection device acquires the inspection image, the YOLOv7 model is used to locate and segment the instrument image.

[0143] Step S13, automatically identifying the instrument type.

[0144] Step S14, calling the corresponding type of instrument program to perform reading recognition and output the result.

[0145] Based on YOLOv7 target detection technology, OCR text recognition technology and digital image processing technology, this application designs an interactive interface that allows users to upload pictures in batches, automatically identifies the instrument type and calls the corresponding type of instrument program for reading recognition, and finally displays the reading in the result box of the interactive interface. It not only solves the problem of low accuracy of dashboard data when automatically reading dashboard data, but also improves the accuracy of dashboard data, thereby realizing more accurate monitoring, predictive maintenance and fault diagnosis of equipment or vehicles equipped with dashboards, and improving safety and efficiency.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0147] In this embodiment, a device for determining dashboard data is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "module" may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0148] Figure 5 : is a structural block diagram of a device for determining dashboard data according to an embodiment of the present application, the device comprising:

[0149] The receiving module 52 is used to receive the inspection image from the user interaction interface, wherein the inspection image is obtained after collecting images of the inspection area;

[0150] A determination module 54 is used to determine a recognition result of image recognition performed on the inspection image using a target recognition model, and determine an instrument image of the instrument panel in the inspection area from the recognition result;

[0151] The obtaining module 56 is used to determine the target instrument program corresponding to the instrument type of the instrument panel from the preset instrument programs, and perform image processing on the instrument image according to the image recognition algorithm corresponding to the target instrument program to obtain the instrument panel data of the instrument panel.

[0152] Through the above device, by receiving the inspection image from the user interaction interface, wherein the inspection image is obtained after image acquisition of the inspection area; determining the recognition result of the inspection image using the target recognition model, and determining the instrument image of the instrument panel of the inspection area from the recognition result; determining the target instrument program corresponding to the instrument type of the instrument panel from the preset instrument program, and performing image processing on the instrument image according to the image recognition algorithm corresponding to the target instrument program, to obtain the instrument panel data of the instrument panel. It not only solves the problem of low accuracy of the instrument panel data when automatically reading the instrument panel data, but also improves the accuracy of the instrument panel data, thereby realizing more accurate monitoring, predictive maintenance and fault diagnosis of the equipment or vehicles installed with the instrument panel, and improving safety and efficiency.

[0153] In an exemplary embodiment, the determination module, before determining the recognition result of image recognition of the inspection image using the target recognition model, is also used to: determine a training image from a pre-generated training data set, perform an image enhancement operation on the training image to obtain an enhanced image; splice sub-images obtained after cropping the enhanced image to obtain a new training image, wherein the training image and the new training image have the same image label; determine a new training data set based on multiple new training images, and use the new training data set to train the initial recognition model to obtain the target recognition model.

[0154] In an exemplary embodiment, the obtaining module is also used to: perform image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result, and the recognition result includes at least one of the following: pointer data of the instrument panel, scale data of the instrument panel and character data of the instrument panel; determine the instrument panel data using the pointer data, the scale data and the character data.

[0155] In an exemplary embodiment, the obtaining module is also used to: use the image recognition algorithm to perform Hough circle detection on the instrument image to identify the dial outline in the instrument image; determine the circumscribed rectangle of all dial outlines, and filter out the pointer outline from all dial outlines based on the geometric information of the circumscribed rectangle; generate a mask image corresponding to the pointer outline, perform morphological operations on the first mask image to obtain a first morphological image, and obtain different Hough line segments obtained by performing Hough line detection on the first morphological image, determine the Hough line segment with the maximum line segment length from the different Hough line segments, and determine the pointer data based on the fitting line segment of the Hough line segment.

[0156] In an exemplary embodiment, the obtaining module is also used to: convert the image format of the instrument image into a brightness display format, wherein the pointer of the instrument panel has a first brightness value, and the dial of the instrument panel has a second brightness value; for a first pointer whose line segment width is greater than a preset width, determine the pointer contour of the first pointer, perform least squares fitting on the first pointer according to an external matrix of the pointer contour of the first pointer to obtain a fitted straight line, and determine the pointer data based on the fitted straight line; for a second pointer whose line segment width is less than a preset width, determine the pointer contour of the second pointer, determine a target contour point from the pointer contour of the second pointer, and determine the pointer data based on a line between the target contour point and the center of a Hough circle for Hough circle detection, wherein the target contour point is farthest from the Hough circle.

[0157] In an exemplary embodiment, the module is also used to: determine the character recognition model used by the image recognition algorithm, the character recognition model includes a text recognition model; input the instrument image into the text recognition model to obtain a text sequence output by the text recognition model, and determine the character data based on the text characters in the text sequence.

[0158] In an exemplary embodiment, the obtaining module is also used to: input the instrument image into the text positioning model to obtain the text position output by the text positioning model, wherein the text characters in the text sequence all correspond to text positions; when it is determined that the text character is consistent with a preset starting character, obtain the first text position of the text character, and determine the vector between the center point of the circumscribed rectangle where the first text position is located and the center of the instrument panel as the starting vector; when it is determined that the text character is consistent with a preset ending character, obtain the second text position of the text character, and determine the vector between the center point of the circumscribed rectangle where the second text position is located and the center of the instrument panel as the ending vector; determine the starting position and ending position of the pointer of the instrument panel based on the starting vector and the ending vector.

[0159] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0160] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0161] S1, receiving an inspection image from a user interaction interface, wherein the inspection image is obtained after image acquisition of an inspection area;

[0162] S2, determining a recognition result of performing image recognition on the inspection image using a target recognition model, and determining an instrument image of the instrument panel in the inspection area from the recognition result;

[0163] S3, determining a target instrument program corresponding to the instrument type of the instrument panel from preset instrument programs, and performing image processing on the instrument image according to an image recognition algorithm corresponding to the target instrument program to obtain instrument panel data of the instrument panel.

[0164] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0165] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0166] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0167] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0168] S1, receiving an inspection image from a user interaction interface, wherein the inspection image is obtained after image acquisition of an inspection area;

[0169] S2, determining a recognition result of performing image recognition on the inspection image using a target recognition model, and determining an instrument image of the instrument panel in the inspection area from the recognition result;

[0170] S3, determining a target instrument program corresponding to the instrument type of the instrument panel from preset instrument programs, and performing image processing on the instrument image according to an image recognition algorithm corresponding to the target instrument program to obtain instrument panel data of the instrument panel.

[0171] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0172] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0173] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0174] An embodiment of the present application also provides a computer program, which includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in any one of the above method embodiments.

[0175] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0176] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0177] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for determining dashboard data, characterized in that: include: Receiving an inspection image from a user interaction interface, wherein the inspection image is obtained after image acquisition of an inspection area; Determine a recognition result of image recognition of the inspection image using a target recognition model, and determine an instrument image of an instrument panel in the inspection area from the recognition result; A target instrument program corresponding to the instrument type of the instrument panel is determined from preset instrument programs, and image processing is performed on the instrument image according to an image recognition algorithm corresponding to the target instrument program to obtain instrument panel data of the instrument panel.

2. The method according to claim 1, characterized in that Before determining a recognition result of performing image recognition on the inspection image using the target recognition model, the method further includes: Determine a training image from a pre-generated training data set, and perform an image enhancement operation on the training image to obtain an enhanced image; The sub-images obtained after the cropping operation on the enhanced image are spliced ​​to obtain a new training image, wherein the training image and the new training image have the same image label; a new training data set is determined based on multiple new training images, and the new training data set is used to train the initial recognition model to obtain the target recognition model.

3. The method according to claim 2, characterized in that The instrument image is processed according to the image recognition algorithm corresponding to the target instrument program to obtain instrument panel data of the instrument panel, including: Performing image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result, wherein the recognition result includes at least one of the following: pointer data of the instrument panel, scale data of the instrument panel, and character data of the instrument panel; The instrument panel data is determined using the pointer data, the scale data and the character data.

4. The method according to claim 3, characterized in that Performing image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result includes: Using the image recognition algorithm to perform Hough circle detection on the instrument image to identify the dial outline in the instrument image; Determine the circumscribed rectangles of all dial outlines, and select the pointer outlines from all dial outlines according to the geometric information of the circumscribed rectangles; A mask image corresponding to the pointer contour is generated, a morphological operation is performed on the first mask image to obtain a first morphological image, and different Hough line segments obtained by performing Hough line detection on the first morphological image are obtained, a Hough line segment with a maximum line segment length is determined from the different Hough line segments, and the pointer data is determined according to a fitting line segment of the Hough line segment.

5. The method according to claim 3, characterized in that: Performing image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result includes: Converting the image format of the instrument image into a brightness display format, wherein the pointer of the instrument panel has a first brightness value and the dial of the instrument panel has a second brightness value; For a first pointer whose line segment width is greater than a preset width, determining a pointer outline of the first pointer, performing least square fitting on the first pointer according to an external matrix of the pointer outline of the first pointer to obtain a fitted straight line, and determining the pointer data based on the fitted straight line; For a second pointer whose line segment width is less than a preset width, determine the pointer outline of the second pointer, determine a target outline point from the pointer outline of the second pointer, and determine the pointer data based on a line between the target outline point and the center of a Hough circle for Hough circle detection, wherein the target outline point is farthest from the Hough circle.

6. The method according to claim 3, characterized in that Performing image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result includes: Determining a character recognition model used by the image recognition algorithm, wherein the character recognition model includes a text recognition model; The instrument image is input into the text recognition model to obtain a text sequence output by the text recognition model, and the character data is determined based on text characters in the text sequence.

7. The method according to claim 6, characterized in that The method further comprises: Inputting the instrument image into the text positioning model to obtain the text position output by the text positioning model, wherein each text character in the text sequence corresponds to a text position; When it is determined that the text character is consistent with the preset starting character, a first text position of the text character is obtained, and a vector between a center point of a circumscribed rectangle where the first text position is located and a center point of the instrument panel is determined as a starting vector; When it is determined that the text character is consistent with the preset termination character, a second text position of the text character is obtained, and a vector between a center point of a circumscribed rectangle where the second text position is located and a center point of the instrument panel is determined as a termination vector; A starting position and an ending position of a pointer of the instrument panel are determined based on the starting vector and the ending vector.

8. A device for determining dashboard data, characterized in that: include: A receiving module, used to receive an inspection image from a user interaction interface, wherein the inspection image is obtained after image acquisition of an inspection area; A determination module, used to determine a recognition result of image recognition performed on the inspection image using a target recognition model, and determine an instrument image of the instrument panel in the inspection area from the recognition result; The obtaining module is used to determine the target instrument program corresponding to the instrument type of the instrument panel from the preset instrument programs, and perform image processing on the instrument image according to the image recognition algorithm corresponding to the target instrument program to obtain the instrument panel data of the instrument panel.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images