Vehicle inkjet character detection and recognition method and system and electronic equipment

Abstract

The invention discloses a vehicle inkjet character detection and identification method. The method comprises the following steps: acquiring a side door picture shot by a camera after a to-be-detected vehicle drives into a specified parking area; calling a vehicle door deviation recognition model to detect the side vehicle door picture to obtain a current vehicle door deviation condition; on the basis of the current vehicle door deviation condition, whether the current camera position is in an ideal state or not is judged, a corresponding operation instruction is sent out when the camera is not in the ideal state, and the camera position is adjusted till the camera position reaches the ideal state; when it is judged that the position of the camera reaches the ideal state, a text detection model is called to detect a text area on the vehicle door in the frame picture, and a text recognition model is called to recognize text content in the text area. According to the invention, automatic positioning detection and identification can be carried out on the inkjet characters in the external fixed area of the vehicle, so that the whole vehicle inspection efficiency is improved and the possibility of human cheating is avoided.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent vehicle inspection, and specifically relates to a method, system and electronic device for detecting and recognizing vehicle spray-painted text. Background Art

[0002] With the continuous development of the vehicle inspection field, it is crucial for vehicle safety supervision and traffic accident prevention. At the same time, there are problems such as heavy review pressure on review personnel in vehicle inspection stations, inconsistent vehicle review standards, and high costs invested by vehicle management offices in supervision. Therefore, under the urgent needs of the public security traffic management department and the vehicle inspection market, intelligent vehicle inspection technology has emerged.

[0003] According to the requirements of the national standard "Motor Vehicle Inspection Work Procedure" GA801-2019, for some vehicle models, it is necessary to inspect relevant information recorded in the spray-painted text in the fixed external area of the vehicle, such as parameters like the number of passengers, total mass, and maximum towing mass. These inspection items are an essential part of the whole vehicle inspection, but the current measurement method mainly relies on manual visual inspection, which not only wastes manpower and increases costs, but more importantly, there is a risk of human cheating during the inspection process. Summary of the Invention

[0004] . To solve the above problems, the present invention provides a method for detecting and recognizing vehicle spray-painted text, which can realize full-automatic positioning detection and recognition of the spray-painted text in the fixed external area of the vehicle, and improve the efficiency of the whole vehicle inspection. Further, a corresponding system, electronic device and computer-readable storage medium for detecting and recognizing vehicle spray-painted text are also disclosed.

[0005] The technical solution of the present invention includes the following aspects:

[0006] The first aspect discloses a method for detecting and recognizing vehicle spray-painted text, including the following steps:

[0007] Obtain a side door picture taken by a camera after the vehicle to be inspected drives into the specified parking area;

[0008] Call a door offset recognition model to detect the side door picture, and obtain the current door offset situation;

[0009] Based on the current door offset situation, judge whether the current camera position is in an ideal state, and when the camera is not in an ideal state, issue a corresponding operation instruction to adjust the position of the camera until the camera position reaches the ideal state;

[0010] When it is judged that the camera position reaches the ideal state, call a text detection model to detect the text area on the door in the frame picture, and call a text recognition model to recognize the text content in the text area;

[0011] The current door offset situation includes the direction and amount of door offset; among them, the direction of door offset includes offsets in the four directions of up, down, left, and right, which is judged by whether the side of the door in the corresponding direction is missing; the amount of door offset is characterized by the missing ratio of the door in the offset direction area.

[0012] As an alternative, based on the current door offset situation, it is judged whether the current camera position is in an ideal state, and corresponding operation instructions are issued when the camera is not in an ideal state, specifically including:

[0013] If there is no missing side of the door, it means that the door has no offset, the camera position is in an ideal state, and no adjustment is required;

[0014] If there is a missing side of the current door, it means that the door has an offset, and an operation instruction to translate the camera in the opposite direction of the missing side is issued;

[0015] If there are multiple missing sides of the current door at the same time, it means that the door has an offset, and an operation instruction to translate the camera backward is issued to expand the receptive field captured by the camera.

[0016] As an alternative, the door offset recognition model includes two parts: a Backbone module and a classifier module; the Backbone module uses a ViT network based on the Transformer architecture to extract features from the side door pictures; the classifier module uses a multi-layer perceptron network to classify the extracted high-order semantic features to obtain the current door offset situation.

[0017] As an alternative, in the ViT network, first divide the side door picture into image patches of a fixed size, then flatten the image patches and map them to vectors of a fixed length through a linear transformation; then splice the vector sequence behind a learnable classification token; then add the learnable position encoding at the corresponding position and input it into the Transformer encoder; finally, extract the features corresponding to the classification token and input them into the classifier module for learning.

[0018] As an alternative, the output dimension of the classifier module is 4, corresponding to the missing ratio values of the door areas in the four directions of up, down, left, and right of the current side door respectively, and the missing ratio values have preset categories; the missing ratio value greater than 0 indicates that the side of the door in that direction is missing and the door is offset in that direction; the missing ratio value equal to 0 indicates normal and the door is not offset.

[0019] As an alternative, the text detection model uses the open-source DBNet network model; the DBNet network model includes a backbone network, a feature pyramid network, and a differentiable binarization module. Among them, the backbone network is a ResNet network, which is used to extract the features of the image and obtain a series of feature maps; the feature pyramid network is used to fuse the feature maps of different levels to obtain fused features; the differentiable binarization module adaptively learns through the network to obtain a probability map representing the probability of each pixel belonging to the text area and a threshold map representing the binarization threshold of each pixel, and compares the probability map and the threshold map pixel by pixel to obtain the final binarization result.

[0020] As an alternative, the text recognition model uses the open-source CRNN network model; the CRNN network model includes a convolutional layer for feature extraction and a recurrent layer and a transcription layer for decoding the extracted features; among them, the convolutional layer uses a ResNet network to extract the image features of the text area and outputs a high-level abstract feature map; the recurrent layer is used to convert the high-level abstract feature map into a feature sequence; the transcription layer is used to convert the feature sequence into a final character sequence and then output it.

[0021] In a second aspect, an electronic device is disclosed, including a processor and a memory; the processor is used to call a computer program stored in the memory and execute the vehicle spray painting text detection and recognition method described in the first aspect and any one of the alternative solutions.

[0022] In a third aspect, a computer-readable storage medium is disclosed, and the computer-readable storage medium is used to store program code, and the program code is used to execute the vehicle spray painting text detection and recognition method described in the first aspect and any one of the alternative solutions.

[0023] The fourth aspect discloses a vehicle spray painting text detection and recognition system, including a data acquisition device, a mechanical motion device, and a data processing module; the data acquisition device is a camera for obtaining side door pictures of a vehicle to be inspected driving into a specified parking area; the mechanical motion device includes a motion controller and a robotic arm electrically connected to the motion controller; the motion controller is communicatively connected to the data processing module, and performs motion control on the robotic arm according to the operation instructions sent by the data processing module, so as to realize the up, down, left, right, front, and back movement of the camera installed on the robotic arm in three-dimensional space; the data processing module includes: a door offset recognition unit, configured to receive the side door pictures uploaded by the data acquisition device, detect the input pictures according to the built-in door offset recognition model, and obtain the door offset situation; the camera state determination module is configured to: based on the current door offset situation, judge whether the current camera position reaches the ideal state, and when it does not reach, send corresponding operation instructions to the motion controller to adjust the camera position through the robotic arm until it is judged that the camera position reaches the ideal state; a text detection unit, configured to call the pre-stored text detection model to detect the text area of the input picture and output a text picture containing the text area; a text recognition unit, configured to call the pre-stored text recognition model to recognize the text content in the text picture.

[0024] The present invention has the following beneficial effects:

[0025] The present invention can realize full-automatic spray painting text detection and recognition, which can not only improve the vehicle inspection efficiency, but also eliminate the possibility of human cheating.

[0026] The designed door offset recognition algorithm of the present invention can assist the robotic arm to achieve precise position adjustment. For the text detection model and the text recognition model, the DBNet+CRNN network can achieve a good balance between the detection and recognition accuracy and the inference speed. Description of the Drawings

[0027] Figure 1 It is a schematic diagram of the network architecture of the door offset recognition model.

[0028] Figure 2 It is a schematic diagram of the ViT network architecture based on the Transformer architecture.

[0029] Figure 3 It is a schematic diagram of the relevant logic for camera position judgment.

[0030] Figure 4 It is a schematic diagram of the network architecture of the DBNet network model.

[0031] Figure 5 It is a schematic diagram of the network architecture of the CRNN network model.

[0032] Figure 6 It is an example diagram of door offset. Among them, (a), (b), and (c) respectively represent three offset situations. Specific implementation manners

[0033] To better understand the present invention, the technical solution of the present invention will be further introduced below in conjunction with the accompanying drawings and specific embodiments.

[0034] In the inspection of large vehicles, it is usually necessary to detect and recognize the painted text on the doors. Combining Figures 1 to 5 As shown, an embodiment of the present invention proposes a method for detecting and recognizing vehicle painted text, which mainly includes the following steps:

[0035] Step 1: Obtain a side door picture taken by a camera after the vehicle to be inspected drives into the specified parking area.

[0036] The camera used can be a binocular camera. The camera is installed at the front end of the robotic arm and is used to obtain pictures related to the vehicle to be inspected. It can be understood that the robotic arm is installed at a specified position of the inspection station and is electrically connected to the motion controller. The motion controller is communicatively connected to the server and is used to receive the operation instructions issued by the controller to flexibly adjust the position of the camera through the robotic arm.

[0037] At the specified inspection station, the camera is in a silent state, waiting for the arrival of the vehicle to be inspected. When the vehicle to be inspected slowly drives into the parking area of the current inspection station, the vehicle brakes and parks. Under normal conditions, the camera faces the door area on the side of the front of the vehicle to be inspected. The side of the front of the vehicle and the door area of the driver's seat or the passenger seat appear in the camera screen. The camera takes a frame picture at the current moment and transmits the captured frame picture to the server in the form of a picture. The server obtains the picture containing the side door of the vehicle to be inspected uploaded by the camera (abbreviated as "side door picture") for subsequent text detection and recognition.

[0038] Step 2: Call the door offset recognition model to detect the input picture and obtain the current door offset situation.

[0039] The server starts the pre-trained door offset recognition model, performs inference and analysis on the transmitted side door picture, and obtains the current door offset situation. The door offset situation mainly includes the direction of door offset and the amount of offset. Among them, the direction of door offset includes offsets in the up, down, left, and right directions, which are judged by whether the side of the door in the corresponding direction is missing; the amount of offset is characterized by the missing ratio of the area of the door in the offset direction.

[0040] The trained door offset recognition model is mainly used to detect the input side door images, and determine whether there are any missing areas in the four directions of up, down, left, and right of the target door, that is, the side door. If there are missing areas, what is the corresponding missing ratio value, so as to reflect the current door offset situation.

[0041] As Figure 1 shown, the overall architecture of the door offset recognition model mainly includes two parts: the Backbone module and the classifier module. The Backbone module uses the ViT network based on the Transformer architecture to effectively extract features from the input side door images. The ViT network has the characteristics of rich semantic information extraction, high classification accuracy, and relatively accurate detection of the door offset amount.

[0042] Specifically, in the ViT network, first, the input image (side door image) is divided into image patches of a fixed size, and then these image patches are flattened and mapped into vectors of a fixed length through a linear transformation; then these vector sequences are concatenated behind a learnable classification token ([CLS] token); then, after adding the learnable position encoding corresponding to the position, it is input into the Transformer Encoder; finally, the features corresponding to the [CLS] token are extracted and input into the classifier module for learning. As Figure 2 shown, the description of each step is as follows:

[0043] Image divided into patches: The input image X ∈ R H×W×C is divided into image patches of size P×P, forming a sequence of length image patches, and each image patch is represented as

[0044] Linear projection of image patches: Each image patch is flattened into a vector, and then mapped into a vector of dimension D through a learnable linear projection

[0045] Classification token ([CLS] Token): A learnable embedding vector X class ∈ R D is used to represent the classification token, which is used to aggregate the information of the entire image and output the probability of each category by combining the information of the entire image.

[0046] Positional encoding embedding: In order to record the position information of each image patch, a learnable position encoding is added to the embedding of each image patch. Since there are N + 1 sequences after adding the classification token, the learnable position encoding is represented as E pos ​∈R (N+1)×D 。

[0047] Transformer Encoder: The encoder consists of L layers, and each layer includes: a multi-head self-attention (MSA) layer, a multi-layer perceptron (MLP) block. There is layer normalization (LN) before each block and a residual connection after each block. The specific formula is as follows:

[0048] z l ′ = MSA(LN(z l-1 )) + z l-1 (1)

[0049] z l = MLP(LN(z l ′)) + z l ′(2)

[0050] In the formula, z l-1 represents the output of the (l - 1)-th layer encoder, and z l ′ represents the result obtained by performing layer normalization and multi-head self-attention calculations on the output of the (l - 1)-th layer encoder in sequence. Further, layer normalization and multi-layer perceptron calculations are performed on z l ′ to obtain the output result z l of the current encoding layer l.

[0051] The classifier module uses a designed multi-layer perceptron (MLP) network to classify the high-order semantic features extracted by the Transformer encoder. The output dimension of the classifier module is 4, which is used to classify the offset conditions of the car door in the up, down, left, and right directions and the missing ratio values of the car door in the offset direction. The output values include [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9], a total of 10 categories, corresponding to 10 missing ratio values, and the offset amounts of each car door in each direction are reflected by the missing ratio values.

[0052] It can be understood that the car door is usually rectangular. Based on the center point of the car door, the car door is divided into four regions: up, down, left, and right. The upper region represents the region above the center point (occupying half of the car door area), the lower region represents the region below the center point (occupying half of the car door area), the left region represents the region to the left of the center point (occupying half of the car door area), and the right region represents the region to the right of the center point (occupying half of the car door area).

[0053] For example, two sides, namely the upper side and the right side of the car door, are not within the camera's field of view. The car door is shifted both upward and leftward, and there are missing areas in the upper and left directions of the car door. The output value of the classifier is [0.3, 0, 0.5, 0], which represents the missing details of the upper, lower, left, and right sides of the car door from left to right in sequence. A value greater than 0 indicates a missing area, and a value equal to 0 indicates normal. Therefore, the results corresponding to the output values are: the missing ratio of the upper area is 0.3, the missing ratio of the left area is 0.5, and the other sides are normal.

[0054] Step 3: Based on the current shift situation of the car door, determine whether the current camera position is in an ideal state. When the camera is not in an ideal state, issue corresponding operation instructions to adjust the camera position through the robotic arm until the camera position reaches the ideal state.

[0055] According to the current shift situation of the car door output by the car door shift recognition model, the server analyzes and judges it to determine whether the current camera position reaches the ideal state. If not, the server decides how to adjust the position of the robotic arm connected to the camera in the next step and sends specific operation instructions to the robotic arm. The robotic arm receives and executes the operation instructions to make corresponding adjustments to the camera position.

[0056] The processing logic of the server is as Figure 3 shown. It mainly replaces manual judgment through a series of logics. The robotic arm makes corresponding adjustments by receiving the logical judgment results until the camera position reaches the ideal state. At this time, the quality of the pictures captured by the camera is also the highest. The ideal state can be specifically set as no shift on the upper, lower, left, and right sides of the car door. The specific process includes the following situations:

[0057] The first case: If there is no car door shift, that is, no side of the car door is missing, it means the camera position is in the ideal state, and directly proceed to Step 4.

[0058] The second case: If the car door shift detection result shows that one side is missing, it is necessary to command the robotic arm to drive the camera to translate in the opposite direction of the missing side. For example, if the output result shows that the missing ratio of the left area of the car door is 0.2, give the operation instruction to the robotic arm to translate to the right, and the specific moving distance is W * 0.2; another example, if the output result shows that the missing ratio of the upper area of the car door is 0.6, give the operation instruction to translate downward, and the moving distance is H * 0.6, where H and W represent the height and width of the camera's field of view respectively.

[0059] The third case: If the prediction result of the door offset shows that there are multiple side missing areas simultaneously, it is very likely that the camera is too close to the door and the receptive field is small. Therefore, it is necessary to command the robotic arm to drive the camera to move backward horizontally to increase the receptive field captured by the camera. For example, if there are missing areas on both the upper and lower sides of the door, the camera moves backward by a fixed unit distance, which is determined according to the on-site environment, such as 30 cm. It should be noted that when there are multiple side missing areas, whether they are adjacent or opposite sides, the camera is first moved backward by one unit to expand the receptive field, and then the next round of judgment and adjustment is carried out. Usually, the translation distance of one unit is not large, such as 30 cm. Translating once has little impact on the text size, but more of the door area can be seen, which is beneficial for algorithm recognition.

[0060] Combined with Figure 6 as shown in Figure 6 (a), the left side of the door does not appear in the camera image, indicating that there is a missing area on the left side of the door. The classifier output value is [0, 0, 0.3, 0], indicating that the missing ratio of the left side area of the door is 0.3, and the other sides are normal. In this case, usually, the camera can be translated to the left. Figure 6 (b), neither the left side nor the right side of the door appears in the camera image, indicating that there are missing areas on both the upper and lower sides of the door. The classifier output value is [0.1, 0.1, 0, 0], indicating that the missing ratios of both the left and right side areas of the door are 0.3. This situation indicates that the door is too close to the camera, and the distance between the camera and the door needs to be adjusted by moving the camera backward by a fixed unit distance. Figure 6 (c), neither the left side nor the upper side of the door appears in the camera image, indicating that there are missing areas on both the left and upper sides of the door. The classifier output value is [0.2, 0, 0.1, 0], indicating that the missing ratios of the left and upper side areas of the door are 0.2 and 0.1 respectively, and the other sides are normal. In this case, the camera is also first moved backward by a fixed unit distance.

[0061] After the camera distance is adjusted, return to step 1 and repeat the above process, that is: re-capture the frame image at the current moment, perform the detection of the offset of each side of the door on the server side, obtain the door offset situation, issue an operation instruction, and adjust the position of the camera through the robotic arm until the camera position reaches the ideal state.

[0062] Step 4: After the camera position reaches the ideal state, call the text detection model to detect the text area on the door in the frame image, and call the text recognition model to recognize the text content in the text area.

[0063] After the camera position is adjusted to the ideal state required by the system, the server calls the trained text detection model to detect the text area on the car door through the text detection model, and obtains the text picture after extracting the text box. Since the inspection items need to review parameter information such as the number of passengers, net weight, and load capacity, and then determine whether the vehicle to be inspected meets the relevant standard requirements, after detecting the text box, it is also necessary to further call the trained text recognition algorithm model to recognize the text content in the text picture.

[0064] Specifically, the open-source DBNet network model can be used as the text detection model. The DBNet network model realizes the close combination of the binarization process and network training by introducing a differentiable binarization function, improving the detection accuracy and flexibility. As Figure 4 shown, the DBNet network structure mainly consists of three parts, namely: the backbone network, the Feature Pyramid Network (FPN), and the Differentiable Binarization Module (DB Module). The network structure is relatively simple and has high computational efficiency, making it suitable for scenarios with high real-time requirements. The components of the DBNet network model are introduced as follows:

[0065] Backbone network: The DBNet network model uses ResNet as the backbone network to extract the features of the image and obtain a series of feature maps. By introducing residual connections, ResNet can effectively solve the problems of gradient disappearance and representation bottleneck in deep neural networks, enabling the network to learn the features of the image more deeply.

[0066] Feature Pyramid Network (FPN): FPN is a method for multi-scale feature fusion. By fusing feature maps at different levels, the network can capture text information at different scales simultaneously. In the DBNet network model, the fused features output by FPN are used as the input for subsequent segmentation.

[0067] Differentiable Binarization Module (DB Module): This is the core of the DBNet network model. Traditional text detection methods usually perform binarization as a post-processing step, separate from network training. The DB Module integrates the binarization process into network training, enabling the network to adaptively learn a more suitable binarization threshold. Specifically, after the fused features output by FPN are input into the DB Module, it contains two output branches: the Probability Map and the Threshold Map. The Probability Map represents the probability that each pixel belongs to the text region, while the Threshold Map represents the binarization threshold for each pixel. In the inference stage, by comparing the Probability Map and the Threshold Map pixel by pixel, the final binarization result can be obtained. Based on the obtained binarization result, the position of the text region in the original image can be obtained, and the text region in the original image can be extracted in the form of a text box to obtain a text image containing the text box for subsequent text recognition.

[0068] The text recognition model uses the open-source CRNN network model. The CRNN network combines the advantages of convolutional neural networks (CNNs) and recurrent neural networks (RNNs), and can effectively process sequential data in images, suitable for the recognition of text sequences, such as handwritten recognition, scene text recognition, etc. As Figure 5 shown, the CRNN network mainly includes a convolutional layer (CNN) for feature extraction, a recurrent layer (RNN), and a transcription layer (CTC loss) for decoding the extracted features. The specific description is as follows:

[0069] Convolutional layer (CNN): The ResNet network is used, which is mainly responsible for extracting features from the input image (i.e., the text image). Through multiple convolutional and pooling operations, local features in the image, such as edges and corners, can be captured and represented as high-level abstract feature maps. The convolutional layer specifically includes the ReLU activation function, batch normalization (BatchNormalization), and pooling layers (such as max pooling).

[0070] Recurrent layer (RNN): Receives the high-level abstract feature map of a specific size output by the convolutional layer and converts it into a feature sequence. The recurrent layer uses a bidirectional long short-term memory network (BiLSTM) to handle long-term dependencies in sequential data. For each time step (corresponding to a column or a row of the feature map), the RNN outputs a state vector that contains information about the current position and its context.

[0071] It is understandable that during the process of feature extraction by the convolutional layer, downsampling is performed. Eventually, a value on the feature map corresponds to a receptive field area of the input image. When the recurrent layer processes the feature map, it outputs state vectors sequentially from left to right. The state vector at the current position corresponds to the receptive field of the corresponding area of the original image from left to right, and the context refers to the state vectors corresponding to all the receptive fields up to and including the current position.

[0072] In addition, when performing sequence classification in the recurrent layer, a character may be recognized multiple times, and a redundancy removal mechanism is required. Specifically, a space '-' can be added between repeated characters, consecutive repeated characters can be deleted, and then the '-' characters on the left and right in the path can be removed.

[0073] Transcription layer (CTC loss): Responsible for converting the feature sequence output by the recurrent layer into the final character sequence. By connecting the Connectionist Temporal Classification (CTC) layer, the alignment between the label sequence and the input data can be directly learned from the sequence data without the need to pre-annotate the separators between characters. The output of the CTC layer is usually a probability distribution representing the probability of each character corresponding to each time step. Finally, through a decoding algorithm (greedy search), the final character sequence can be recovered from this probability distribution.

[0074] In a specific embodiment, the convolutional layer extracts features from the input text image to obtain a feature map of size 1*26*512. The height and width of the feature map are 1 and 26 respectively, and the number of feature channels is 512. The height and width dimensions are merged, and the merged dimension becomes the time step input to the RNN layer. Here, the time step is 26, corresponding to a point on the feature map, which is also a receptive field on the text image. Each sequence is a point on the feature map, and a point corresponds to a 512-channel vector, so its length is the number of channels 512 of the original feature layer. Each sequence from left to right corresponds to a rectangular area in the original image, and the order of the sequences corresponds to the position of the image. The feature sequences that make up the feature map are input into the recurrent layer according to the time step. The number of neurons in the hidden layer of the recurrent layer is 256, and a multi-layer BiLSTM is used for processing. The input time step is 26, and 26 feature vectors are obtained after passing through the recurrent layer. The classification result of the output feature vectors, that is, the probability of each character, is obtained. According to the input feature vectors, the softmax probability distribution of all characters is obtained. This is a vector with a length equal to the total number of character categories to be recognized. The feature vectors in the feature sequence output by the recurrent layer are used as the input to the transcription layer. The transcription layer converts the predictions made by the recurrent layer for each feature vector into a label sequence, which is used to represent the text information contained in each receptive field, and outputs the final character sequence.

[0075] Furthermore, another embodiment of the present invention provides a vehicle spray painting text detection and recognition system, which mainly includes a data acquisition device, a mechanical motion device, and a data processing module.

[0076] The data acquisition device is mainly used to obtain the side door pictures of the vehicle to be inspected driving into the specified parking area, and transmit the pictures to the data processing module, which can be specifically implemented by a binocular camera.

[0077] The mechanical motion device is mainly used to carry the data acquisition device and adjust the position of the data acquisition device based on the instructions issued by the data processing module. The mechanical motion device mainly includes a motion controller and a robotic arm electrically connected to the motion controller. The camera is installed at the front end of the robotic arm. The motion controller is communicatively connected to the data processing module, receives the operation instructions issued by the data processing module, and performs corresponding motion control on the robotic arm; the robotic arm drives the camera to move up, down, left, right, forward, and backward in three-dimensional space.

[0078] The data processing module can be a server, or other electronic devices such as a computer or a mobile terminal, and is used to implement the corresponding process in the vehicle spray painting text detection and recognition method in the previous embodiment. Specifically, the data processing module mainly includes the following functional units: a door offset recognition unit, which is configured to receive the side door pictures uploaded by the data acquisition device, detect the input pictures according to the built-in door offset recognition model, and obtain the door offset situation; the camera state determination module is configured to: based on the current door offset situation, judge whether the current camera position reaches the ideal state, and when it does not reach, send corresponding operation instructions to the motion controller to adjust the camera position through the robotic arm until it is judged that the camera position reaches the ideal state; a text detection unit, which is configured to call the pre-stored text detection model to detect the text area of the input picture and output a text picture containing the text area; a text recognition unit, which is configured to call the pre-stored text recognition model to recognize the text content in the text picture. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific processes of the above-mentioned door offset recognition model, text detection model, and text recognition model can refer to the corresponding processes in the vehicle spray painting text detection and recognition method in the previous embodiment, which will not be elaborated here.

[0079] Furthermore, another embodiment of the present invention also discloses an electronic device, which mainly includes a processor and a memory. The processor is mainly used to call and execute the computer program stored in the memory to implement the vehicle spray painting text detection and recognition method in the foregoing embodiment.

[0080] Further, another embodiment of the present invention further provides a computer-readable storage medium, which is used to store program codes for executing the vehicle spray-painted text detection and recognition method in the foregoing embodiments. Optional computer-readable storage media may include: flash drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, etc.

[0081] Finally, it should be noted that although the embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the above specific embodiments and application fields. The above specific embodiments are merely illustrative and guiding, rather than restrictive. Under the inspiration of this specification, those of ordinary skill in the art can also make many forms without departing from the scope protected by the claims of the present invention, and these all fall within the scope of protection of the present invention.

Claims

1. A method for detecting and identifying vehicle spray-painted text, characterized in that: The following steps are involved: Obtain the side door image taken by the camera after the vehicle to be inspected enters the specified parking area; Calling a door offset recognition model to detect the side door image to obtain the current door offset; Based on the current door offset, determine whether the current camera position is in an ideal state, and issue corresponding operation instructions if the camera is not in an ideal state to adjust the camera position until the camera position reaches the ideal state; When it is determined that the camera position reaches an ideal state, a text detection model is called to detect a text area on the door in the frame image, and a text recognition model is called to recognize text content in the text area; The current door offset condition includes the direction of the door offset and the amount of the door offset; wherein the direction of the door offset includes the offset in four directions of up, down, left, and right, and is judged by whether the door side in the corresponding direction is missing; the amount of the door offset is represented by the missing ratio of the door in the offset direction area.

2. The vehicle spray-painted text detection and recognition method according to claim 1, characterized in that: The method of judging whether the current camera position is in an ideal state based on the current door offset and issuing corresponding operation instructions when the camera is not in an ideal state specifically includes: If there is no missing door side, it means the door is not offset and the camera position is ideal, no adjustment is needed; If one side of the current door is missing, it means that the door is offset, and an operation command is issued to move the camera in the opposite direction of the missing side; If the current door has multiple missing sides at the same time, it means that the door is offset, and an operation command is issued to move the camera backward to expand the receptive field captured by the camera.

3. The vehicle spray-painted text detection and recognition method according to claim 1 or 2, characterized in that: The door offset recognition model includes two parts: a backbone module and a classifier module; the backbone module adopts a ViT network based on a Transformer architecture to extract features from the side door image; the classifier module adopts a multi-layer perceptron network to classify the extracted high-order semantic features to obtain the current door offset situation.

4. The vehicle spray-painted text detection and recognition method according to claim 3, characterized in that: In the ViT network, the side door image is first divided into image blocks of fixed size, and then the image blocks are flattened and mapped into vectors of fixed length through linear transformation; then the vector sequence is concatenated behind a learnable classification tag; then the learnable position code of the corresponding position is added and input into the Transformer encoder; finally, the features corresponding to the classification tag are extracted and input into the classifier module for learning.

5. The vehicle spray-painted text detection and recognition method according to claim 3, characterized in that: The output dimension of the classifier module is 4, which corresponds to the missing ratio values ​​of the door area in the four directions of up, down, left and right of the current side door, and the missing ratio values ​​have preset categories; the missing ratio value greater than 0 indicates that the side of the door in this direction is missing and the door is offset in this direction; the missing ratio value equal to 0 indicates that it is normal and the door is not offset.

6. The vehicle spray-painted text detection and recognition method according to claim 1 or 2, characterized in that: The text detection model adopts the open source DBNet network model; The DBNet network model includes a backbone network, a feature pyramid network and a differentiable binarization module, wherein the backbone network is a ResNet network, which is used to extract image features and obtain a series of feature maps; the feature pyramid network is used to fuse feature maps of different levels to obtain fused features; the differentiable binarization module obtains a probability map for representing the probability that each pixel belongs to a text area and a threshold map for representing the binarization threshold of each pixel through network adaptive learning, and compares the probability map and the threshold map pixel by pixel to obtain the final binarization result.

7. The vehicle spray-painted text detection and recognition method according to claim 1 or 2, characterized in that: The text recognition model adopts an open source CRNN network model; The CRNN network model includes a convolutional layer for feature extraction and a recurrent layer and a transcription layer for decoding the extracted features; wherein the convolutional layer adopts a ResNet network to extract image features of the text area and output a high-level abstract feature map; the recurrent layer is used to convert the high-level abstract feature map into a feature sequence; The transcription layer is used to convert the feature sequence into a final character sequence and then output it.

8. An electronic device, characterized in that: It comprises a processor and a memory; the processor is used to call a computer program stored in the memory to execute the vehicle spray-painted text detection and recognition method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the vehicle spray-painted text detection and recognition method according to any one of claims 1 to 7.

10. A vehicle spray-painted text detection and recognition system, characterized in that: It includes a data acquisition device, a mechanical motion device and a data processing module; The data acquisition device is a camera for obtaining images of the side doors of vehicles to be inspected that enter the specified parking area; the mechanical motion device includes a motion controller and a mechanical arm electrically connected to the motion controller; the motion controller is in communication with the data processing module, and controls the motion of the mechanical arm according to the operation instructions sent by the data processing module, so as to realize the movement of the camera installed on the mechanical arm in three-dimensional space. The data processing module comprises: The door offset recognition unit is configured to receive the side door image uploaded by the data acquisition device, detect the input image according to the built-in door offset recognition model, and obtain the door offset situation; The camera state determination module is configured to: determine whether the current camera position reaches an ideal state based on the current door offset, and if not, send a corresponding operation instruction to the motion controller to adjust the camera position through the robotic arm until the camera position reaches an ideal state; A text detection unit is configured to call a pre-stored text detection model to detect the text area of ​​the input image and output a text image containing the text area; The text recognition unit is configured to call a pre-stored text recognition model to recognize the text content in the text image.

Images