Instrument panel data determination method and device, storage medium and electronic device

Abstract

The application discloses a kind of dashboard data determination method and device, storage medium and electronic device, the method comprises: receiving from user interactive interface inspection image, wherein the inspection image is obtained after image acquisition to inspection area;Determine that the identification result of using target identification model is carried out to the inspection image image recognition, and determine the instrument image of the dashboard of the inspection area from the identification result;From the preset instrument program, determine the target instrument program corresponding to the instrument type of the dashboard, and according to the image recognition algorithm corresponding to the target instrument program, the instrument image is processed, and the dashboard data of the dashboard is obtained.The above technical scheme is used, the problem that the accuracy of dashboard data is lower when automatically reading dashboard data is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of instrument identification, in particular to a method and device for determining instrument panel data, a storage medium and an electronic device. BACKGROUND

[0002] In modern industry and transportation, instrument panels are important components of vehicles, ships, aircraft and various equipment. Various instruments and indicator lights are integrated on the instrument panel to display key parameters such as speed, temperature, pressure, and oil level in real time, reflecting the running state of the vehicle or equipment.

[0003] Generally, the parameters displayed on the physical instrument panel or digital instrument panel are read by manual reading. For example, for a physical instrument panel, the data displayed by the pointer and the dial can be read. For a digital instrument panel, the data displayed by the liquid crystal display or organic light-emitting diode can be directly read. However, the manual reading method is inefficient, so the related art proposes a method for automatically identifying instrument panel data, such as using computer vision to identify key parameters in the instrument panel image. Or use intelligent models such as convolutional networks to identify the instrument panel position and specific elements on the instrument panel, such as speedometer, fuel gauge, etc., but the recognition result of such automatic recognition is not accurate enough and is prone to misreading data. Therefore, when automatically reading instrument panel data, there is a technical problem of low accuracy of instrument panel data.

[0004] In view of the problem of low accuracy of instrument panel data in the related art when automatically reading instrument panel data, no effective solution has been proposed so far.

[0005] Therefore, it is necessary to improve the related art to overcome the defects in the related art. SUMMARY

[0006] The embodiments of the present application provide a method and device for determining instrument panel data, a storage medium and an electronic device to at least solve the problem of low accuracy of instrument panel data when automatically reading instrument panel data.

[0007] According to an aspect of the embodiments of the present application, a method for determining instrument panel data is provided, comprising: receiving an inspection image from a user interaction interface, wherein the inspection image is obtained by image acquisition on an inspection area; determining a recognition result of image recognition of the inspection image using a target recognition model, and determining an instrument image of an instrument panel of the inspection area from the recognition result; determining a target instrument program corresponding to an instrument type of the instrument panel from a preset instrument program, and performing image processing on the instrument image according to an image recognition algorithm corresponding to the target instrument program to obtain instrument panel data of the instrument panel.

[0008] In an example embodiment, before determining the recognition result of the inspection image by using the target recognition model, the method further comprises: determining a training image from a pre-generated training data set, performing an image enhancement operation on the training image to obtain an enhanced image; performing a cropping operation on the enhanced image to obtain a sub-image, and performing a splicing operation on the sub-image to obtain a new training image, wherein the training image and the new training image have the same image label; determining a new training data set based on a plurality of new training images, and training an initial recognition model using the new training data set to obtain the target recognition model.

[0009] In an example embodiment, the image processing on the instrument image according to the image recognition algorithm corresponding to the target instrument program to obtain the instrument panel data of the instrument panel comprises: performing image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result, the recognition result comprising at least one of the following: pointer data of the instrument panel, scale data of the instrument panel, and character data of the instrument panel; and determining the instrument panel data by using the pointer data, the scale data, and the character data.

[0010] In an example embodiment, the image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result comprises: performing Hough circle detection on the instrument image by using the image recognition algorithm to identify a dial contour in the instrument image; determining a circumscribed rectangle of all dial contours, and screening a pointer contour from the all dial contours according to geometric information of the circumscribed rectangle; generating a mask image corresponding to the pointer contour, performing a morphological operation on the first mask image to obtain a first morphological image, and obtaining different Hough line segments by performing Hough line detection on the first morphological image; determining a Hough line segment with a maximum line segment length from the different Hough line segments, and determining the pointer data according to a fitting line segment of the Hough line segment.

[0011] In an example embodiment, the image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result comprises: converting an image format of the instrument image into a luminance display format, wherein a pointer of the instrument panel has a first luminance value, and a dial of the instrument panel has a second luminance value; for a first pointer with a line segment width greater than a preset width, determining a pointer contour of the first pointer, performing least square fitting on the first pointer according to an outer-approximate matrix of the pointer contour of the first pointer to obtain a fitted straight line, and determining the pointer data based on the fitted straight line; for a second pointer with a line segment width less than the preset width, determining a pointer contour of the second pointer, determining a target contour point from the pointer contour of the second pointer, and determining the pointer data based on a connecting line between the target contour point and a center of a Hough circle for Hough circle detection, wherein the target contour point is farthest from the Hough circle.

[0012] In an example embodiment, the image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result comprises: determining a character recognition model used by the image recognition algorithm, wherein the character recognition model comprises a text recognition model; inputting the instrument image into the text recognition model to obtain a text sequence output by the text recognition model, and determining the character data based on text characters in the text sequence.

[0013] In an example embodiment, the method further comprises: inputting the instrument image into the text positioning model to obtain a text position output by the text positioning model, wherein each text character in the text sequence corresponds to a text position; in a case where the text character is determined to be consistent with a preset starting character, obtaining a first text position of the text character, and determining a vector between a center point of an outer-approximate rectangle in which the first text position is located and a center of the instrument panel as a starting vector; in a case where the text character is determined to be consistent with a preset ending character, obtaining a second text position of the text character, and determining a vector between a center point of an outer-approximate rectangle in which the second text position is located and the center of the instrument panel as an ending vector; and determining a pointer starting position and a pointer ending position of the instrument panel based on the starting vector and the ending vector.

[0014] According to another aspect of the embodiments of the present application, the present application further provides a device for determining instrument panel data, comprising: a receiving module configured to receive an inspection image from a user interface, wherein the inspection image is obtained by image collection of an inspection area; a determining module configured to determine an identification result of image identification of the inspection image using a target identification model, and determine an instrument image of an instrument panel of the inspection area from the identification result; and a obtaining module configured to determine a target instrument program corresponding to an instrument type of the instrument panel from a preset instrument program, and perform image processing on the instrument image according to an image identification algorithm corresponding to the target instrument program to obtain instrument panel data of the instrument panel.

[0015] According to still another aspect of the embodiments of the present application, the present application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is configured to execute the above-mentioned method for determining instrument panel data when running.

[0016] According to still another aspect of the embodiments of the present application, the present application further provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-mentioned method for determining instrument panel data through the computer program.

[0017] According to still another aspect of the embodiments of the present application, the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps in any of the above-mentioned method embodiments.

[0018] According to the present application, by receiving an inspection image from a user interface, wherein the inspection image is obtained by image collection of an inspection area; determining an identification result of image identification of the inspection image using a target identification model, and determining an instrument image of an instrument panel of the inspection area from the identification result; determining a target instrument program corresponding to an instrument type of the instrument panel from a preset instrument program, and performing image processing on the instrument image according to an image identification algorithm corresponding to the target instrument program to obtain instrument panel data of the instrument panel. Not only the problem of low accuracy of instrument panel data in automatic reading of instrument panel data is solved, but also the accuracy of instrument panel data is improved, and more accurate monitoring, predictive maintenance and fault diagnosis of equipment or vehicles equipped with instrument panels are realized, and safety and efficiency are improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the present application.

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, for those skilled in the field, other drawings can also be obtained based on these drawings without any creative effort.

[0021] Figure 1 Fig. 1 is a hardware structure block diagram of a mobile terminal according to a dashboard data determination method of an embodiment of the present application;

[0022] Figure 2 Fig. 2 is a flow chart of a dashboard data determination method according to an embodiment of the present application;

[0023] Figure 3 Fig. 3 is a flow schematic diagram of a dashboard data determination method according to an embodiment of the present application;

[0024] Figure 4 Fig. 4 is a running flow schematic diagram of an instrument program according to an embodiment of the present application;

[0025] Figure 5 Fig. 5 is a structure block diagram of a dashboard data determination device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the person skilled in the art better understand the present application, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort should belong to the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0028] The methods and embodiments provided in this application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a method of determining dashboard data according to an embodiment of this application. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor (MCU) or a field-programmable gate array (FPGA)) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0029] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the dashboard data determination method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0031] In this embodiment, a method for determining dashboard data is provided, Figure 2 is a flowchart of a method for determining dashboard data according to an embodiment of the present application, as shown in Figure 2 , the flowchart includes the following steps:

[0032] Step S202: receiving an inspection image from a user interface, wherein the inspection image is obtained by image acquisition of an inspection area;

[0033] Optionally, PyQt can be used to design a GUI interactive interface (i.e. user interface) for instrument recognition. The specific process can include:

[0034] 1. PyQt library needs to be installed. PyQt5 can be installed through pip, with the command: pip install PyQt5.

[0035] 2. Import PyQt5 modules in the Python script, for example: from PyQt5.QtWidgets import QApplication, QWidget, QVBoxLayout, QPushButton, QLabel, QFileDialog, etc.

[0036] 3. Use PyQt layout managers (such as QVBoxLayout) to organize interface elements such as buttons, labels, and input boxes.

[0037] 4. Add interface elements as needed, such as buttons (QPushButton), labels (QLabel), and input boxes (QLineEdit). Set properties for each element, such as text, size, and position.

[0038] 5. Add event handling functions to interface elements, such as button click events. Implement instrument recognition logic in these functions, such as reading images, recognizing instruments, and displaying results.

[0039] 6. Create a QWidget as the main window and set window title, size, and other properties.

[0040] 7. Create a QApplication instance and start the event loop. Display the main window and wait for user operations.

[0041] Optionally, the user interface includes a file upload module, a type matching module, and a result display module. The file upload module supports users to upload multiple images. The type matching module can automatically match instrument types and call related detection programs (i.e. target instrument programs) to output the running results of the detection programs, which at least include the detected dashboard data.

[0042] Step S204: Determine the recognition result of image recognition of the inspection image using the target recognition model, and determine the instrument image of the instrument panel of the inspection area from the recognition result;

[0043] Step S206: Determine the target instrument program corresponding to the instrument type of the instrument panel from the preset instrument program, and perform image processing on the instrument image according to the image recognition algorithm corresponding to the target instrument program to obtain the instrument panel data of the instrument panel.

[0044] Optionally, the instrument type of the instrument panel includes a disc type and a non-disc type. For the disc type, the pointers, line segments and characters in the instrument panel are sequentially recognized. For the non-disc type, the instrument panel contour needs to be recognized from the instrument image, and then the pointers, line segments and characters in the instrument panel are sequentially recognized.

[0045] Through the above steps, the inspection image is received from the user interaction interface, wherein the inspection image is obtained by image acquisition on the inspection area; the recognition result of image recognition of the inspection image using the target recognition model is determined, and the instrument image of the instrument panel of the inspection area is determined from the recognition result; the target instrument program corresponding to the instrument type of the instrument panel is determined from the preset instrument program, and the instrument image is processed according to the image recognition algorithm corresponding to the target instrument program to obtain the instrument panel data of the instrument panel. Not only solves the problem of low accuracy of instrument panel data in automatic reading of instrument panel data, but also improves the accuracy of instrument panel data, thereby realizing more accurate monitoring, predictive maintenance and fault diagnosis of the equipment or vehicle installed with the instrument panel, and improving the safety and efficiency.

[0046] In the present application, the data set of the instrument class can be constructed first, and the data set is further trained on the basis of the existing weight of the target recognition model YOLOv7 to improve the accuracy of subsequent recognition of the model. After receiving the inspection image, the YOLOv7 is used to locate the instrument panel position from the inspection image, and the reading of the instrument panel area is performed after the instrument panel image is extracted.

[0047] In the process of image recognition, the recognition of pointers, scales and characters is mainly involved. For the pointers and scales, two methods of Hough line detection and color segmentation based recognition are proposed to improve the recognition accuracy. For the characters, the inspection image needs to be input into the pre-trained OCR model, and the position of the starting scale is located according to the image information and text sequence output by the model, wherein the OCR model can be trained by file data of multiple characters.

[0048] Optionally, the application also uses the Pyqt design instrument identification interface, which can receive user batch upload of corresponding pictures, automatically call the corresponding instrument identification program according to the instrument type, automatically identify the instrument panel data and display the results.

[0049] Optionally, according to the application scene of the instrument in the inspection image and the difference of the dial content, the instrument can be specifically divided into oil temperature gauge, oil level gauge, lightning arrester and on-load voltage regulating transformer, and the corresponding algorithm process is designed for different instruments.

[0050] In an exemplary embodiment, before determining the identification result of the image recognition of the inspection image using the target identification model, the following steps can also be implemented: determining a training image from a pre-generated training data set, performing an image enhancement operation on the training image to obtain an enhanced image; performing a cropping operation on the enhanced image to obtain a sub-image, and then performing a splicing operation on the sub-image to obtain a new training image, wherein the training image and the new training image have the same image label; determining a new training data set based on a plurality of new training images, and training an initial identification model using the new training data set to obtain the target identification model. The embodiment proposes a training method of the target identification model. In the training process, the training data set is optimized, and the model recognition accuracy can be further improved.

[0051] The image enhancement operation includes flipping, rotating, shifting and the like. The above initial identification model is, for example, a basic model such as a convolutional neural network.

[0052] In an embodiment, the training process of the identification model is specifically described by the following process. First, the data set of the instrument is constructed, and the instrument image therein is subjected to processing operations including flipping, rotating, shifting and the like, so as to realize single-sample data enhancement. Then, the pictures in the training set are subjected to picture cropping and splicing to synthesize a new training sample. At this time, the label of the new training sample is still the label of the original training sample, so as to realize multi-sample data enhancement. After using the new training set to train the model, a new weight file can be obtained, which is then used for inference of new pictures. In the inference process, the bounding box and other information of the identified target are written into a json file, facilitating subsequent image segmentation processing.

[0053] In one example embodiment, the process of image processing the instrument image according to the image recognition algorithm corresponding to the target instrument program to obtain the instrument panel data of the instrument panel specifically includes: performing image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result, the recognition result including at least one of the following: pointer data of the instrument panel, scale data of the instrument panel, and character data of the instrument panel; and determining the instrument panel data by using the pointer data, the scale data, and the character data. Through this embodiment, the accuracy of instrument panel recognition is improved by recognizing the pointer, scale, and character respectively in the process of image recognition.

[0054] In one example embodiment, further implementation schemes of performing image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result are proposed, specifically including: performing Hough circle detection on the instrument image using the image recognition algorithm to identify the dial outline in the instrument image; determining the circumscribed rectangle of all dial outlines, and screening out the pointer outline from the all dial outlines according to the geometric information of the circumscribed rectangle; generating the mask image corresponding to the pointer outline, performing morphological operation on the first mask image to obtain a first morphological image, and obtaining different Hough line segments obtained by performing Hough line detection on the first morphological image, determining the Hough line segment with the maximum line segment length from the different Hough line segments, and determining the pointer data according to the fitting line segment of the Hough line segment.

[0055] The geometric information of the circumscribed rectangle includes, for example, the aspect ratio and the center point position.

[0056] In one embodiment, specific processing steps of Hough line detection are proposed. First, the ROI region is extracted according to the image position information to obtain the instrument image, Hough circle detection is performed on the instrument image after image preprocessing, and then the center position and radius of the dial are extracted, and the mask picture corresponding to the dial is generated to ensure that the region of Hough line detection is within the dial.

[0057] The image preprocessing process specifically includes, for example, grayscale processing, Gaussian filtering, and Canny edge detection.

[0058] Then, after Canny edge detection is performed on the mask picture, a contour set is obtained, the circumscribed rectangle corresponding to the contour is drawn, and the contour is preliminarily screened according to the length and width of the circumscribed rectangle and the center point position thereof to retain the scale contour and the pointer contour.

[0059] After the corresponding contour is extracted, the mask pictures (equivalent to the mask images described above) of the scale and the pointer are generated, and further morphological operations such as dilation and corrosion are performed to fill in small holes of missing pixels in the mask picture and eliminate part of the noise.

[0060] Then, the mask pictures of the scale and the pointer are subjected to Hough line detection, and the obtained approximate Hough lines are subjected to length screening. For example, in the identification of the pointer, the fitting straight line of the pointer can be further obtained by length screening. Among them, the longest and the second longest are reserved for the double-pointer instrument, and the longest is reserved for the single-pointer instrument.

[0061] The screening process of the approximate Hough lines after the Hough line detection can include: arranging the identified Hough lines in descending order according to the length, then comparing each line segment backward in turn, fitting the straight line equation of each line segment, merging the line segments, and only reserving the line segments whose straight lines are within a certain threshold distance from the center of the circle, otherwise, it is not the position of the pointer.

[0062] Optionally, the screening by length is as follows:

[0063] 1. If the slopes of two lines are similar, and the distance between the two lines is less than a certain set value, screening is required. At this time, the reason for screening is that for some pointers with relatively wide pixels, a pointer may be identified as multiple similar straight lines, and these similar straight lines need to be merged.

[0064] 2. If two straight lines intersect, and the lengths of the line segments of the two straight lines have a significant difference, screening is required. The reason for screening is that after the morphological operation, there may still be some noise points, which cause the original pointer part to be connected with the noise points and be identified as a straight line.

[0065] After screening and merging, the remaining straight lines can be subjected to secondary screening, and the line segments that still exist are extended. When the straight line of the line segment is within a certain distance from the center of the circle, the straight line is reserved, otherwise it is discarded.

[0066] It should be noted that in the present application, the identification method for the scale can refer to the identification method for the pointer, which will not be described here.

[0067] In an example embodiment, for image recognition of the instrument image according to the image recognition algorithm, an identification result is obtained, and the process can further include the following implementation steps: converting the image format of the instrument image into a luminance display format, wherein the pointer of the instrument panel has a first luminance value, and the dial of the instrument panel has a second luminance value; for a first pointer with a line segment width greater than a preset width, determining a pointer contour of the first pointer, performing least square fitting on the pointer contour of the first pointer according to an outer matrix of the pointer contour to obtain a fitted straight line, and determining the pointer data based on the fitted straight line; for a second pointer with a line segment width less than the preset width, determining a pointer contour of the second pointer, determining a target contour point from the pointer contour of the second pointer, and determining the pointer data based on a connecting line between the target contour point and a center of a Hough circle for Hough circle detection, wherein the target contour point is farthest from the Hough circle.

[0068] In an example embodiment, the step of identifying the pointer using the color recognition method is as follows: first, according to the dial color and the pointer color, the dial picture is converted into an HSV color space, the upper and lower limits of the target color are set, and the pointer region is segmented according to the target color. The display format of the picture in the HSV color space corresponds to the luminance display format described above.

[0069] Then, for a thicker pointer, a straight line fitting is directly performed, and specifically, the thicker pointer is least square fitted according to an outer rectangle of the contour, and then the fitted straight line is taken as the pointer. For a thinner pointer, points on the pointer contour can be traversed, and a point on the contour farthest from the distance of the previous Hough circle detection is taken as a far point of the pointer, and a connecting line between the far point and the center is the pointer.

[0070] In an example embodiment, further, the process of image recognition of the instrument image according to the image recognition algorithm to obtain an identification result can further include: determining a character recognition model used by the image recognition algorithm, the character recognition model including a text recognition model; inputting the instrument image into the text recognition model to obtain a text sequence output by the text recognition model, and determining the character data based on text characters in the text sequence.

[0071] The character recognition model can specifically include a text positioning model and a text recognition model, and the character recognition model can be used to identify characters and determine that the starting point of the pointer is not on a disc with a center.

[0072] The text positioning model is, for example, a CnSTD (Chinese Scene Text Detection) model, and the text recognition model is, for example, a CnOCR (Chinese Optical Character Recognition) model. The two models are usually combined to implement a complete process of extracting and recognizing Chinese text from an image. First, the CnSTD locates the text regions in the image, and then the CnOCR recognizes the characters in these regions. This combination can be applied to various practical scenarios, such as automatic translation, information extraction, traffic sign recognition for autonomous vehicles, etc.

[0073] In an exemplary embodiment, further, the instrument image is input into the text positioning model to obtain text positions output by the text positioning model, wherein each text character in the text sequence corresponds to a text position; in a case where it is determined that the text character is consistent with a preset starting character, a first text position of the text character is obtained, and a vector between a center point of a circumscribed rectangle in which the first text position is located and a center of the instrument panel is determined as a starting vector; in a case where it is determined that the text character is consistent with a preset ending character, a second text position of the text character is obtained, and a vector between a center point of a circumscribed rectangle in which the second text position is located and the center of the instrument panel is determined as an ending vector; and the starting position and the ending position of the pointer of the instrument panel are determined based on the starting vector and the ending vector.

[0074] For the process of determining the starting vector and the ending vector, the following process can be further combined: if a character such as "0" or "MIN" (corresponding to the preset starting character) is recognized, the corresponding character position is returned, for example, a circumscribed rectangle containing its upper left, lower left, upper right, and lower right four endpoints, and the center point of the circumscribed rectangle and the center vector are taken as the starting vector. Similarly, if the largest number or "MAX" or the like is recognized, the character corresponding position is returned, for example, a circumscribed rectangle containing its upper left, lower left, upper right, and lower right four endpoints, and the center point of the circumscribed rectangle and the center vector are taken as the ending vector.

[0075] Further, for the case where the straight line where the pointer is located does not pass through the center of the dial, the center points of different scale positions recognized can be determined, and the mid-perpendiculars of the center point connecting line segments are calculated in turn. The straight lines where the mid-perpendiculars are located intersect with the straight line where the pointer is located, and a plurality of intersection points are obtained. The average of the intersection points is calculated, and the starting point of the pointer fitted is obtained.

[0076] Obviously, the above-described embodiments are only part of the embodiments of the present application, not all. In order to better understand the above method, the following describes the above process in combination with the embodiments, but is not used to limit the technical solutions of the embodiments of the present application, specifically:

[0077] In an optional embodiment, the present application combines Figure 3 The flowchart shown illustrates the process of identifying dashboard data. The specific steps are as follows:

[0078] Step 1: Run YOLOv7 to extract dial position and segmentation.

[0079] It should be noted that, unlike the two-stage detector Faster-RCNN, YOLOv7 is a single-stage target detection algorithm based on convolutional neural network, which can directly predict the class and bounding box position of the target in the output layer. The input image is divided into fixed grids, and each grid unit is responsible for detecting the target in its own region. After the detection result is output. From the picture input process, YOLOv7 network mainly includes Input (input), Backbone (backbone network), Neck (neck), Head (head) these four parts. First, the picture is preprocessed after a series of operations such as data enhancement, and then sent to the backbone network. The backbone network part extracts the features of the processed picture; then, the extracted features are processed by the Neck module feature fusion to obtain large, medium and small size features, and finally, the fused features are sent to the detection head for detection and get the result.

[0080] Among them, in the process of using YOLOv7 model to extract the image, first collect the relevant instrument data set from the network, and use Labelme to label the instruments in it. The instrument data set mainly includes the dial instrument. The data set is processed by data enhancement (including cropping, rotating, flipping, etc.), and then the data set is divided into training set, test set and validation set, the proportion is 8:1:1.

[0081] The xml file of the labeled data set is stored in the Annotations folder. The file labeled as xml is converted into a txt file, which contains the bounding box position information of the target area corresponding to the picture, and is stored under the labels folder. The picture is placed under Image, and the picture path is placed under data.

[0082] Run the model training file train.py, train on the basis of the original model, get the new weight, modify the weight in the detection program detect.py file to the new weight, import the data set that needs to be detected into the model, run the detection program, recognize and return the class, position information, confidence and other information. In the information, the index target is the instrument class, the position information is returned and written into the json file, and the instrument is segmented before the instrument image processing by using the json file.

[0083] Step 2: Determine whether the instrument panel is a round instrument panel. If yes, execute step 3, otherwise execute step 4.

[0084] Step 3: Perform Hough circle detection on the instrument image to further extract the dial position.

[0085] After segmenting and extracting the dial area, if the obtained area is a rectangular area, for a round instrument, Hough circle detection is required to extract the dial area.

[0086] The pre-step of Hough circle detection usually includes picture graying operation and Canny edge detection operation.

[0087] The principle of picture graying is to adjust the saturation of each primary color in the RGB image to zero to form an image without color distinction, which is the gray image of the original image.

[0088] The principle of Canny edge detection is as follows:

[0089] (1) Gaussian smoothing is performed on the gray picture using a Gaussian filter to make the pixel value of each point close to the true value and remove isolated noise. The two-dimensional normal distribution formula for determining the weight is:

[0090]

[0091] Where σ is the standard deviation of the Gaussian distribution, used to determine the blurring degree of the filter.

[0092] (2) Gradient calculation. The Sobel operator is used to calculate the gradient intensity direction of the image for preliminary positioning of the edge. The Sobel operator is used to calculate the gradient in the horizontal and vertical directions respectively:

[0093]

[0094] Through convolution operation, the horizontal direction gradient G x and G Y are obtained, and the gradient amplitude and gradient direction are calculated as:

[0095]

[0096] (3) Non-maximum suppression. Only keep the pixels that are local gradient maxima, and suppress along the gradient direction. If a pixel is not the maximum in its gradient direction, suppress its value to 0.

[0097] (4) Double thresholding. Determine potential edges by setting thresholds. The high threshold is used to mark strong edges, and the low threshold is used to mark weak edges.

[0098] (5) Edge linking. Strong edges determined by the high threshold are kept as final edges. Weak edges below the high threshold but above the low threshold are also kept as edge points if they are connected to strong edges.

[0099] The principle of Hough circle detection is to map edge points in the image to parameter space and determine the parameters of the circle by accumulator voting. The standard equation of a circle is:

[0100] (x-a) 2 +(y-b) 2 =r 2 .

[0101] In Hough circle detection, the edge point (x, y) is calculated by the formula

[0102]

[0103] to obtain the possible center coordinates (a, b). The accumulator is used to record the voting situation of each possible combination (a, b, r). Finally, the peak value in the accumulator represents the detected circle. As in the previous step, first perform Canny detection to obtain edge points, then perform parameter space voting to detect circular targets in the image. Thus, the dial area is extracted, and according to the position information, the dial area is further cropped from the original image to create a corresponding mask picture.

[0104] Step 4: After pre-processing the image, perform contour recognition and preliminary contour screening based on the position and aspect ratio of the contour.

[0105] Specifically, first perform grayscale and image threshold adaptive binarization preprocessing. Then perform contour recognition and draw the bounding rectangle of all recognized contours. According to the aspect ratio and center point position of the bounding rectangle, screen the contours and identify the areas that may be scales or pointers, facilitating the next step of screening.

[0106] The principle of binarization is to calculate the mean value of the gray values of all pixel points in the pixel point matrix of the gray image. In this application, the calculation of the mean value adopts the calculation method of Gaussian mean. Similar to the Gaussian filtering process, the adaptive threshold of a pixel point is equal to the weight calculated according to the two-dimensional normal distribution in the neighborhood. The part with a pixel value greater than the threshold is set to 255, and the part less than the threshold is set to 0 to generate a binary image.

[0107] Step 5: Identify the pointer and the scale line using Hough line detection.

[0108] Step 6: Perform line segment screening according to the slope, the distance between the lines where the line segments are located, and the position of the line segments.

[0109] After identifying the scale and the pointer, mask images of the two are generated respectively. For the mask image of the pointer, the lines in the gray image are detected based on HoughlinesP, and all line segments contained in the mask image are detected. After cross screening of the detected line segments, the line segments are arranged in descending order according to the length, and the longest one is generally the pointer line segment. The screening conditions of the line segments are as follows:

[0110] (1) The slopes of the two lines are similar, and the distance between the two lines is less than a certain set value. This screening is because: for some pointers with relatively wide pixels, a pointer may be identified as multiple similar straight lines, which need to be merged.

[0111] (2) The two straight lines intersect, and the lengths of the line segments have obvious differences. This screening is because: after the morphological operation, there may still be some noise points, which cause the original pointer part to be connected with the noise points and be identified as a straight line.

[0112] After screening and merging, the remaining straight lines are subjected to secondary screening, and the remaining line segments are extended. When the line where the line segment is located is within a certain distance from the center of the circle, the straight line is retained, otherwise it is discarded.

[0113] Step 7: If the identification effect is not good, the color segmentation method is used to identify the pointer area and the straight line or least square fitting is used.

[0114] For the case where the image pixels are not enough and the pre-processing effect is not good, due to the presence of many holes, morphological processing cannot complete the filling, and increasing the number of dilation may cause more noise, thus causing misidentification. Therefore, the number of times of dilating the processed image is limited, and therefore the number of line segments detected by the above Hough line detection may not be enough, so the color segmentation method is adopted.

[0115] Taking a double-pointer instrument as an example, the historical maximum indicating scale of the indicating scale is red, the upper and lower limits of the red color in the HSV value are set in advance, the original image is converted into the HSV space, and segmentation is performed according to the color range set in advance to generate a corresponding mask picture, wherein the ROI region is white and the other parts are black.

[0116] After the corresponding contour is identified, if the corresponding pointer part is thin, a Hough line detection is directly used to fit the corresponding straight line, and for the wide pointer type (i.e. nonlinear shape), a least square method is used to fit a straight line as the straight line where the direction vector of the pointer is located.

[0117] Step 8: using the CnSTD model and the CnOCR model to identify the digital reading or the dial information to determine the start and end positions of the pointer type instrument.

[0118] Specifically, the existing CnSTD model and the CnOCR model are used for positioning and identifying numbers and English characters.

[0119] Among them, the CnSTD uses a CNN as a feature extractor to extract a feature map from an input image. Through convolution operation, low-level features (such as edge, corner, etc.) are extracted, and higher-level features are formed in deeper network layers for distinguishing text and background. The convolution formula is:

[0120]

[0121] Among them, h i，j is the value of the output feature map, x i+m，j+n is the pixel value of the input image, w m，n is the convolution kernel, b is the bias term, and σ is the activation function.

[0122] After identifying the text region, the CnSTD model uses a boundary text box regression method to correct the initial predicted text box by learning the offset of the boundary box, so that the text box is closer to the real text prediction position. The formula for boundary box regression is:

[0123]

[0124] Among them, (r i , c i ) is the row and column position of the predicted text box, (h i , w i ) are the height and width respectively, and the characters with superscripts are the real boundary box parameters.

[0125] Since the model can detect multiple similar text boxes in the same area, CnSTD uses non-maximum suppression (NMS) to remove redundant detection boxes by calculating the overlap of candidate boxes, keeping the boxes with lower overlap and removing the boxes with higher overlap. At the same time, the CnSTD model can identify text regions of different sizes through multi-scale processing, improving the robustness of the model. Finally, the CnSTD model identifies the text region and returns the text box of the text region.

[0126] As described above, CnOCR is used for character recognition based on CnSTD.

[0127] The principle of CnOCR is to combine convolutional neural network (CNN) and recurrent neural network (RNN) detection, and use CTC loss function for sequence-to-sequence training. First, the input text image is converted into a set of high-dimensional feature representations through convolutional and pooling layers. For text recognition, the main role of the convolutional layer is to capture the local spatial information of the character, and the pooling layer is used for downsampling to reduce computational complexity and overfitting.

[0128] After extracting the character features, CnOCR uses recurrent neural networks, especially bidirectional long short-term memory networks, to establish sequence modeling problems to capture the context relationship of character sequences.

[0129] where the output of the recurrent neural network (RNN) is:

[0130] h t =σ(W h *h t-1 +W x *x t +b h )。

[0131] And the output of LSTM considering the information before and after is:

[0132] h t =LSTM f (x t-1 , h t )+LSTM b (x, h t+1 )。

[0133] where h t is the hidden state at time step t, W h and W t are the weight matrices of RNN, and b h is the bias.

[0134] Since the length and position of the characters are not fixed, CnOCR uses a CTC loss function to solve the alignment problem. CTC allows the character sequence output by the network to be mismatched with the target length in the world, and instead generates an intermediate probability matrix, where each character has a probability distribution. Through decoding, the character sequence can be derived from the output probability matrix.

[0135] By character recognition, the starting or ending position of different types of meters can be determined.

[0136] For example, in an oil temperature meter, the key with the text key value of '0' is indexed from the dictionary output from the result, the position information is extracted, the center point of the bounding box of the framed area is calculated, and the line connecting the point and the center of the circle is taken as the starting vector;

[0137] In the oil level meter, the keys with the text corresponding key values of "MIN" and "MAX" are indexed from the dictionary output from the result, the center points are taken, and the lines connecting the center points and the center of the circle are taken as the starting vector and the ending vector. In addition, based on the scale line recognition method described above, the scale line closest to the detected scale can also be selected as the pointing point of the starting or ending vector according to the distance measure.

[0138] In addition, there is another application scenario for character recognition, i.e., to determine that the scale line is a circular arc, but the center of the circle is not in the Hough circle, and the instrument is a single pointer, so the center of the circle cannot be found by the intersection of the pointers. For example, in the instrument type of a lightning arrester, this situation may occur, so the center of the circle is found by finding the symmetric characters, taking the center points, taking the perpendicular bisector of the connecting line, and taking the intersection point of the perpendicular bisector or the intersection point of the perpendicular bisector and the straight line on which the pointer is located as the center of the circle.

[0139] Step 9: Calculate the reading by the angle method. Based on the above recognition process, for a digital instrument, CnSTD and CnOCR can be directly used to recognize the corresponding numbers, and for a pointer instrument, after the starting vector and the direction vector on the straight line of the pointer are recognized, the included angle between the vectors is calculated, multiplied by the corresponding division value of the unit scale, and the reading of the instrument is obtained. If it is a non-circular instrument, the Hough circle detection process is skipped, and the picture obtained after the YOLOv7 positioning dial is segmented is directly processed. After the picture is preprocessed, the pointer, the scale, and the characters are detected, and the reading is obtained by the angle method.

[0140] In other embodiments, as shown in FIG. 8, the instrument panel can also be recognized by the following steps: Figure 4

[0141] Step S11, based on the pygt interface, the user uploads the pictures in batches;

[0142] Step S12, after the inspection equipment obtains the inspection image, the YOLOv7 model is used to locate and segment the instrument image.​

[0143] Step S13, automatically identify the type of instrument.

[0144] Step S14, call the instrument program of the corresponding type to identify the reading, and output the result.

[0145] Based on the YOLOv7 target detection technology, the OCR text recognition technology and the digital image processing technology, the application designs an interactive interface that allows users to upload pictures in batches, automatically identifies the type of instrument and calls the instrument program of the corresponding type to identify the reading, and finally displays the reading in the result box of the interactive interface. The application not only solves the problem of low accuracy of instrument panel data in automatic reading of instrument panel data, but also improves the accuracy of instrument panel data, thereby realizing more accurate monitoring, predictive maintenance and fault diagnosis of equipment or vehicles equipped with instrument panels, and improving safety and efficiency.

[0146] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of various embodiments of the application.

[0147] In the present embodiment, an instrument panel data determination device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware or a combination of software and hardware is also possible and is contemplated.

[0148] Figure 5 is a structural block diagram of an instrument panel data determination device according to an embodiment of the application, which comprises:

[0149] The receiving module 52 is configured to receive the inspection image from the user interactive interface, wherein the inspection image is obtained by image acquisition on the inspection area;

[0150] The determination module 54 is configured to determine the identification result of image recognition of the inspection image by using the target recognition model, and determine the instrument image of the instrument panel of the inspection area from the identification result;

[0151] The obtaining module 56 is configured to determine a target instrument program corresponding to the instrument type of the instrument panel from the preset instrument program, and perform image processing on the instrument image according to an image recognition algorithm corresponding to the target instrument program, to obtain the instrument panel data of the instrument panel.

[0152] Through the above device, by receiving an inspection image from a user interaction interface, wherein the inspection image is obtained by image acquisition on an inspection area; determining a recognition result of using a target recognition model to perform image recognition on the inspection image, and determining an instrument image of an instrument panel of the inspection area from the recognition result; determining a target instrument program corresponding to the instrument type of the instrument panel from a preset instrument program, and performing image processing on the instrument image according to an image recognition algorithm corresponding to the target instrument program, to obtain the instrument panel data of the instrument panel. Not only solves the problem of low accuracy of instrument panel data when automatically reading instrument panel data, but also improves the accuracy of instrument panel data, thereby realizing more accurate monitoring, predictive maintenance and fault diagnosis of the equipment or vehicle installed with the instrument panel, and improving safety and efficiency.

[0153] In one exemplary embodiment, the determining module, before determining the recognition result of using the target recognition model to perform image recognition on the inspection image, is further configured to: determine a training image from a pre-generated training data set, perform image enhancement operation on the training image to obtain an enhanced image; perform splicing on a sub-image obtained after performing cropping operation on the enhanced image to obtain a new training image, wherein the training image and the new training image have the same image label; determine a new training data set based on a plurality of new training images, and train an initial recognition model using the new training data set to obtain the target recognition model.

[0154] In one exemplary embodiment, the obtaining module is further configured to: perform image recognition on the instrument image according to the image recognition algorithm to obtain a recognition result, wherein the recognition result at least includes one of the following: pointer data of the instrument panel, scale data of the instrument panel, and character data of the instrument panel; and determine the instrument panel data by using the pointer data, the scale data, and the character data.

[0155] In an exemplary embodiment, the module is further configured to: perform Hough circle detection on the instrument image using the image recognition algorithm to identify the dial outline in the instrument image; determine the circumscribed rectangles of all dial outlines, and filter out the pointer outline from all dial outlines based on the geometric information of the circumscribed rectangles; generate a mask image corresponding to the pointer outline; perform morphological operations on the first mask image to obtain a first morphological image; obtain different Hough line segments obtained by performing Hough line detection on the first morphological image; determine the Hough line segment with the maximum line segment length from the different Hough line segments; and determine the pointer data based on the fitted line segment of the Hough line segment.

[0156] In an exemplary embodiment, the obtaining module is further configured to: convert the image format of the instrument image into a brightness display format, wherein the pointer of the instrument panel has a first brightness value and the dial of the instrument panel has a second brightness value; for a first pointer whose line segment width is greater than a preset width, determine the pointer outline of the first pointer, perform least-squares fitting on the first pointer according to the outer matrix of the pointer outline of the first pointer to obtain a fitted straight line, and determine the pointer data based on the fitted straight line; for a second pointer whose line segment width is less than a preset width, determine the pointer outline of the second pointer, determine a target outline point from the pointer outline of the second pointer, and determine the pointer data based on the line connecting the target outline point and the center of the Hough circle for Hough circle detection, wherein the target outline point is farthest from the Hough circle.

[0157] In an exemplary embodiment, the obtaining module is further configured to: determine the character recognition model used by the image recognition algorithm, the character recognition model including a text recognition model; input the instrument image into the text recognition model to obtain a text sequence output by the text recognition model, and determine the character data based on the text characters in the text sequence.

[0158] In an exemplary embodiment, the obtaining module is further configured to: input the instrument image into the text positioning model to obtain the text position output by the text positioning model, wherein each text character in the text sequence corresponds to a text position; if the text character is determined to be consistent with a preset starting character, obtain the first text position of the text character, and determine the vector between the center point of the outer rectangle where the first text position is located and the center of the instrument panel as the starting vector; if the text character is determined to be consistent with a preset ending character, obtain the second text position of the text character, and determine the vector between the center point of the outer rectangle where the second text position is located and the center of the instrument panel as the ending vector; and determine the pointer start position and pointer end position of the instrument panel based on the starting vector and the ending vector.

[0159] The embodiment of the present application further provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program is configured to execute the steps in any of the method embodiments when running.

[0160] Optionally, in the embodiment, the storage medium is configured to store the computer program for executing the following steps.

[0161] S1, receiving an inspection image from a user interaction interface, wherein the inspection image is obtained by image acquisition on an inspection area;

[0162] S2, determining an identification result of image identification of the inspection image by using a target identification model, and determining an instrument image of an instrument panel of the inspection area from the identification result;

[0163] S3, determining a target instrument program corresponding to an instrument type of the instrument panel from a preset instrument program, and performing image processing on the instrument image according to an image identification algorithm corresponding to the target instrument program, to obtain instrument panel data of the instrument panel.

[0164] In an example embodiment, the computer readable storage medium can include but is not limited to a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0165] The specific examples in the embodiment can refer to the examples described in the above embodiments and example embodiments, and the embodiment will not be described here.

[0166] The embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to execute the steps in any of the method embodiments.

[0167] Optionally, in the embodiment, the processor is configured to execute the following steps by the computer program.

[0168] S1, receiving an inspection image from a user interaction interface, wherein the inspection image is obtained by image acquisition on an inspection area;

[0169] S2, determining an identification result of image identification of the inspection image by using a target identification model, and determining an instrument image of an instrument panel of the inspection area from the identification result;

[0170] S3, determine the target instrument program corresponding to the instrument type of the instrument panel from the preset instrument program, and perform image processing on the instrument image according to the image recognition algorithm corresponding to the target instrument program to obtain instrument panel data of the instrument panel.

[0171] In one example embodiment, the electronic device described above can further include a transmission device connected to the processor, and an input / output device connected to the processor.

[0172] Embodiments of the present application also provide a computer program product, which includes a computer program, and the computer program implements the steps in any of the method embodiments described above when executed by a processor.

[0173] Embodiments of the present application also provide another computer program product, which includes a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program implements the steps in any of the method embodiments described above when executed by a processor.

[0174] Embodiments of the present application also provide a computer program, which includes computer instructions stored in a computer readable storage medium; a processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to perform the steps in any of the method embodiments described above.

[0175] The specific examples in the present embodiment can refer to the examples described in the above embodiments and example embodiments, which will not be described here again.

[0176] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be realized by general computing devices, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, and they can be realized by program codes executable by computing devices, so that they can be stored in storage devices and executed by computing devices, and in some cases, the steps shown or described can be executed in different order, or they can be manufactured into individual integrated circuit modules, or multiple modules or steps can be manufactured into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.

[0177] The above only describes the preferred embodiments of the present application, and it should be noted that those skilled in the art can make several improvements and refinements without departing from the principles of the present application, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A method of determining dashboard data, characterized by, The method comprises the following steps: receiving an inspection image from a user interaction interface, wherein the inspection image is obtained by image acquisition on an inspection area; determining an identification result of image identification of the inspection image using a target identification model, and determining an instrument image of an instrument panel of the inspection area from the identification result; determining a target instrument program corresponding to an instrument type of the instrument panel from a preset instrument program, and performing image processing on the instrument image according to an image recognition algorithm corresponding to the target instrument program to obtain instrument panel data of the instrument panel; performing image processing on the instrument image according to the image recognition algorithm corresponding to the target instrument program to obtain instrument panel data of the instrument panel, comprising: performing image recognition on the instrument image according to the image recognition algorithm to obtain an identification result, wherein the identification result at least includes one of the following: pointer data of the instrument panel, scale data of the instrument panel, and character data of the instrument panel; determining the instrument panel data by using the pointer data, the scale data, and the character data; performing image recognition on the instrument image according to the image recognition algorithm to obtain an identification result, comprising: converting the image format of the instrument image into a brightness display format, wherein the pointer of the instrument panel has a first brightness value, and the dial of the instrument panel has a second brightness value; for a first pointer with a line segment width greater than a preset width, determining a pointer contour of the first pointer, performing least square fitting on the first pointer according to the circumscribed matrix of the pointer contour of the first pointer to obtain a fitted straight line, and determining the pointer data based on the fitted straight line; for a second pointer with a line segment width less than a preset width, determining a pointer contour of the second pointer, determining a target contour point from the pointer contour of the second pointer, and determining the pointer data based on the connection line between the target contour point and the center of a Hough circle for Hough circle detection, wherein the target contour point is farthest from the Hough circle.

2. The method of claim 1, wherein, Before determining the identification result of image identification of the inspection image using the target identification model, the method further comprises: determining a training image from a pre-generated training data set, performing an image enhancement operation on the training image to obtain an enhanced image; performing splicing on a sub-image obtained after performing a cropping operation on the enhanced image to obtain a new training image, wherein the training image and the new training image have the same image label; determining a new training data set based on a plurality of new training images, and training an initial identification model using the new training data set to obtain the target identification model.

3. The method of claim 1, wherein, performing image recognition on the instrument image according to the image recognition algorithm to obtain an identification result, comprising: performing Hough circle detection on the instrument image using the image recognition algorithm to identify a dial contour in the instrument image; determining the circumscribed rectangle of all dial contours, and screening out a pointer contour from the all dial contours according to the geometric information of the circumscribed rectangle; A mask image corresponding to the pointer contour is generated, morphological operations are performed on the mask image to obtain a first morphological image, different Hough line segments obtained by performing Hough line detection on the first morphological image are acquired, a Hough line segment with a maximum line segment length is determined from the different Hough line segments, and the pointer data is determined according to a fitting line segment of the Hough line segment.

4. The method of claim 1, wherein, The image recognition algorithm is used to perform image recognition on the instrument image to obtain a recognition result, including: A character recognition model used by the image recognition algorithm is determined, and the character recognition model includes a text recognition model; The instrument image is input into the text recognition model to obtain a text sequence output by the text recognition model, and the character data is determined based on text characters in the text sequence.

5. The method of claim 4, wherein, The method further includes: The instrument image is input into a text positioning model to obtain a text position output by the text positioning model, wherein each text character in the text sequence corresponds to a text position; In a case where the text character is determined to be consistent with a preset starting character, a first text position of the text character is acquired, and a vector between a center point of a circumscribed rectangle in which the first text position is located and a center of the instrument panel is determined as a starting vector; In a case where the text character is determined to be consistent with a preset ending character, a second text position of the text character is acquired, and a vector between a center point of a circumscribed rectangle in which the second text position is located and the center of the instrument panel is determined as an ending vector; The pointer starting position and the pointer ending position of the instrument panel are determined based on the starting vector and the ending vector.

6. An apparatus for determining dashboard data, characterized by It includes: The receiving module is configured to receive a patrol image from a user interaction interface, wherein the patrol image is obtained by image acquisition on a patrol region; The determining module is configured to determine an identification result of image recognition on the patrol image using a target recognition model, and determine an instrument image of an instrument panel of the patrol region from the identification result; The obtaining module is configured to determine a target instrument program corresponding to an instrument type of the instrument panel from a preset instrument program, and perform image processing on the instrument image according to an image recognition algorithm corresponding to the target instrument program to obtain instrument panel data of the instrument panel. The obtaining module is further configured to perform image recognition on the instrument image according to the image recognition algorithm to obtain an identification result, the identification result including at least one of the following: pointer data of the instrument panel, scale data of the instrument panel, and character data of the instrument panel; and determine the instrument panel data by using the pointer data, the scale data, and the character data. The obtaining module is further configured to: convert an image format of the instrument image into a luminance display format, wherein a pointer of the instrument panel has a first luminance value, and a dial of the instrument panel has a second luminance value; for a first pointer with a line segment width greater than a preset width, determine a pointer contour of the first pointer, perform least square fitting on the first pointer according to an outer-approximate matrix of the pointer contour of the first pointer, obtain a fitted straight line, and determine the pointer data based on the fitted straight line; and for a second pointer with a line segment width less than the preset width, determine a pointer contour of the second pointer, determine a target contour point from the pointer contour of the second pointer, and determine the pointer data based on a connection line between the target contour point and a center of a Hough circle for Hough circle detection, wherein the target contour point is farthest from the Hough circle.

7. A computer readable storage medium, characterized in that, The computer readable storage medium comprises a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 5. 8.An electronic device comprising a memory and a processor, the electronic device comprising: The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 5 by using the computer program.

Images