Character string recognition method and device, equipment and storage medium

Abstract

The invention discloses a character string recognition method and device, equipment and a storage medium, and relates to the technical field of computers and the Internet. The embodiment of the invention can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, auxiliary driving and the like. The method comprises the following steps: acquiring a plurality of to-be-recognized images, wherein each to-be-recognized image comprises at least one to-be-recognized character string; character string recognition is carried out on the multiple to-be-recognized images, N recognition results are obtained, and N is an integer larger than 1; calculating the similarity among the N identification results to obtain similar tags corresponding to the N identification results; and obtaining a character string recognition result based on the similar tags corresponding to the N recognition results. Through the method, the similar labels corresponding to the real result can be improved, so that the accuracy of the finally obtained character string recognition result is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer and Internet technology, and in particular to a string recognition method, device, equipment and storage medium. Background Art

[0002] Taking the on-board car search system as an example, during actual operation, the vehicle will continue to take photos during the reversing process in the parking state, and perform string recognition on the photos taken to determine the parking space number of the vehicle.

[0003] In the related art, a string recognition method is provided. After obtaining multiple recognition results through a string recognition model, a recognition result group is determined based on the character overlap rate between the multiple recognition results. For the i-th character in the string to be recognized, the character with the most occurrences at the i-th character position in the above recognition result group is determined as the i-th character in the string to be recognized, where i is a positive integer.

[0004] However, the above method requires a high character overlap rate between multiple recognition results, which requires the recognition results to have a certain length. Some photos that do not capture the complete character string are difficult to use. Summary of the invention

[0005] The embodiment of the present application provides a method, device, equipment and storage medium for character string recognition. The technical solution provided by the embodiment of the present application is as follows.

[0006] According to one aspect of an embodiment of the present application, a method for character string recognition is provided, the method comprising:

[0007] Acquire multiple images to be identified, each of which includes at least one character string to be identified;

[0008] Performing character string recognition on the plurality of images to be recognized to obtain N recognition results, where N is an integer greater than 1;

[0009] Calculate the similarity between the N recognition results to obtain similarity labels corresponding to the N recognition results respectively; wherein the similarity label corresponding to the i-th recognition result among the N recognition results is used to characterize the similarity between the i-th recognition result and the other N-1 recognition results among the N recognition results except the i-th recognition result, where i is a positive integer less than or equal to N;

[0010] Based on the similar labels respectively corresponding to the N recognition results, a character string recognition result is obtained.

[0011] According to one aspect of an embodiment of the present application, a string recognition device is provided, the device comprising:

[0012] An acquisition module is used to acquire a plurality of images to be recognized, each of which includes at least one character string to be recognized;

[0013] A recognition module, used for performing character string recognition on the plurality of images to be recognized to obtain N recognition results, where N is an integer greater than 1;

[0014] A calculation module, used to calculate the similarity between the N recognition results, and obtain similarity labels corresponding to the N recognition results respectively; wherein the similarity label corresponding to the i-th recognition result among the N recognition results is used to characterize the similarity between the i-th recognition result and the other N-1 recognition results among the N recognition results except the i-th recognition result, i being a positive integer less than or equal to N;

[0015] The obtaining module is used to obtain a character string recognition result based on similar tags corresponding to the N recognition results.

[0016] According to one aspect of an embodiment of the present application, a computer device is provided, the computer device comprising a processor and a memory, the memory storing a computer program, and the processor being configured to execute the computer program to implement the above-mentioned string recognition method.

[0017] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the above-mentioned string recognition method.

[0018] According to one aspect of an embodiment of the present application, a computer program product is provided. The computer program product includes a computer program. The computer program is loaded and executed by a processor to implement the above-mentioned character string recognition method.

[0019] The technical solution provided in the embodiments of the present application can bring the following beneficial effects:

[0020] After performing string recognition on the image to be recognized including the string to be recognized, N recognition results are obtained, similarity labels corresponding to the N recognition results are determined based on the similarity between the N recognition results, and the string recognition result is determined based on the similarity labels corresponding to the N recognition results. Since the recognition result corresponding to the image to be recognized that fails to obtain complete string information has a shorter string length, but the similarity between it and the real result corresponding to the string to be recognized is still high, the recognition result with a shorter string length can also be used in the embodiment of the present application, which can increase the similarity label corresponding to the real result, so as to improve the accuracy of the string recognition result finally obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of an implementation environment of a solution provided by an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of a parking space image to be identified provided by an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of a parking space image to be identified provided by another embodiment of the present application;

[0024] Figure 4 is a schematic diagram of a parking space image to be identified provided by another embodiment of the present application;

[0025] Figure 5 is a schematic diagram of a parking space image to be identified provided by another embodiment of the present application;

[0026] Figure 6 is a schematic diagram of a parking space image to be identified provided by another embodiment of the present application;

[0027] Figure 7 is a schematic diagram of a license plate recognition scenario provided by an embodiment of the present application;

[0028] Figure 8 is a schematic diagram of a license plate recognition scenario provided by another embodiment of the present application;

[0029] Fig. 9 is a schematic diagram of a pipeline character recognition scenario provided by an embodiment of the present application;

[0030] Fig.10 is a flowchart of a string recognition method provided by an embodiment of the present application;

[0031] Fig.11 is a flowchart of a string recognition method provided by another embodiment of the present application;

[0032] Fig.12 is a schematic diagram of a user interface of a target application provided by an embodiment of the present application;

[0033] Fig.13 is a schematic diagram of a user interface of a target application provided by another embodiment of the present application;

[0034] Fig.14 is a block diagram of a string recognition device provided by an embodiment of the present application;

[0035] Fig.15 is a block diagram of a string recognition device provided by another embodiment of the present application;

[0036] Fig.16It is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0038] Intelligent Traffic System (ITS), also known as Intelligent Transportation System, effectively and comprehensively applies advanced science and technology (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operations research, artificial intelligence, etc.) to transportation, service control and vehicle manufacturing, strengthens the connection between vehicles, roads and users, and thus forms a comprehensive transportation system that ensures safety, improves efficiency, improves the environment and saves energy.

[0039] Intelligent Vehicle Infrastructure Cooperative Systems (IVICS), referred to as IVICS, is a development direction of Intelligent Transportation Systems (ITS). IVICS uses advanced wireless communications and new generation Internet technologies to implement all-round dynamic real-time information interaction between vehicles and roads, and conducts active vehicle safety control and road cooperative management based on the collection and integration of dynamic traffic information in all time and space, fully realizing the effective coordination of people, vehicles and roads, ensuring traffic safety, and improving traffic efficiency, thus forming a safe, efficient and environmentally friendly road traffic system.

[0040] Before introducing the technical solution of the present application, some technical terms involved in the present application are first introduced and explained.

[0041] On-board parking lot car search technology: On-board parking lot car search technology refers to a system and method that automatically analyzes the data generated in real time by on-board sensors such as cameras, GNSS (Global Navigation Satellite System), IMU (Inertial Measurement Unit) according to set procedures and algorithms during the operation and parking of the vehicle, and uses this analysis to obtain information such as the parking space number of the vehicle, the corresponding GNSS position of the vehicle, the specific floor where the parking is located, the complete video of the parking process, and the parking time. In the embodiments of the present application, the on-board parking lot car search technology can also be referred to as the on-board car search technology, or the on-board parking system.

[0042] Optical Character Recognition (OCR) is a technology that converts printed or handwritten text into editable and searchable electronic text. It uses image processing, pattern recognition, and machine learning technologies to extract text from scanned documents, photos, or other images and convert them into a text format that can be recognized by computers. The application of OCR technology is very wide. It can be used in digitized documents, automated data entry, text search, automated form processing, ID card recognition, bill recognition, and other fields. Through OCR, the efficiency and accuracy of document processing can be greatly improved, and the workload of manual input can be reduced.

[0043] Please refer to Figure 1 , which shows a schematic diagram of a solution implementation environment provided by an embodiment of the present application. The solution implementation environment can be implemented as a system architecture for string recognition. The solution implementation environment may include: a terminal device 100 and a server 200.

[0044] The terminal device 100 includes but is not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, PCs (Personal Computers), tablet computers, wearable devices, VR (Virtual Reality) devices, AR (Augmented Reality) devices, MR (Mixed Reality) devices and other electronic devices. The terminal device 100 can be installed with a client that runs a target application, and the target application can provide a string recognition function for the user. Exemplarily, the target application can be a retrieval application, or a payment application, a shopping application, a camera application, etc., which is not limited in this application. Exemplarily, the target application can obtain a plurality of images to be recognized by shooting the string to be recognized, and after performing string recognition on the plurality of images to be recognized, a plurality of recognition results are obtained, and the string recognition result corresponding to the string to be recognized is determined in the plurality of string recognition results. In addition, the present application does not limit the form of the target application, including but not limited to App (Application, application), applet, etc. installed in the terminal device 100, and can also be in the form of a web page.

[0045] The server 200 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The server 200 may be a background server of the target application, used to provide background services for the client of the target application.

[0046] The terminal device 100 and the server 200 may communicate via a network, such as a wired or wireless network.

[0047] The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc.

[0048] For ease of understanding, before introducing the specific technical solutions of the embodiments of the present application, the technical problems that the embodiments of the present application are intended to solve are first introduced and explained.

[0049] Taking the on-board car search system as an example, in actual operation, the vehicle will continue to take photos during the reverse process of the parking state, and obtain the parking space number through OCR technology. During the entire parking process, multiple photos will be taken to obtain multiple pictures containing parking space numbers, and the corresponding parking space number will be determined based on the string recognition results obtained by performing string recognition on the multiple pictures containing parking space numbers.

[0050] However, in actual scenarios, we may face the following problems:

[0051] 1. As Figure 2 As shown in the figure, the parking space numbers of different parking spaces may be captured in the same picture. Figure 2 In the example of parking space 97 and parking space 98, the parking space numbers obtained by performing OCR recognition based on the image 210 will include number 97 and number 98, and it is difficult to determine a correct parking space number between the two numbers.

[0052] 2. If Figure 3 As shown in the figure, in some pictures, the parking space number may be obscured. Figure 3 In the parking space 310, due to the shooting angle, light and other issues, the parking space number is blocked, and the parking space number of the parking space cannot be obtained based on OCR recognition.

[0053] 3. Such as Figure 4 , Figure 5 As shown in the figure, in some pictures, the parking space number may only be partially captured. Figure 4 In the picture, only the "9" part of parking space 410 is captured, and the actual number should be 99. Figure 5 The parking space 510 in the picture only shows part 83, and the actual area should be 283.

[0054] 4. Such as Figure 6 As shown in the figure, the captured image is blurry and the accuracy of OCR recognition is low. Figure 6 The parking space 610 in the image has a corresponding parking space number of 99. Since the number of the photographed parking space is relatively blurry and limited by the clarity of the image, the accuracy of the parking space number obtained by OCR recognition will be relatively low.

[0055] In addition, similar problems may also occur in scenarios that require string recognition, such as license plate recognition and pipeline character recognition. Figure 7 As shown, the traffic probe on the road takes a picture of the vehicle 710 and identifies the license plate of the vehicle. If the vehicle is moving at a high speed, the picture taken may not be clear and the license plate cannot be accurately identified. Figure 8 As shown, the parking lot entrance and exit takes a photo of the license plate of vehicle 810 and charges the vehicle based on the time interval between vehicles with the same license plate entering and exiting the parking lot. During the photo shooting process, the license plate may be blocked by the horizontal bar, resulting in the license plate not being captured in the photo. Fig. 9 As shown, the characters of the products on the production line are generally small. For example, the character string on the product 910 is difficult to see clearly without magnification. It is difficult to obtain accurate character string recognition results through character string recognition technology.

[0056] In the related art, a string recognition method is provided. After obtaining multiple recognition results through a string recognition model, a recognition result group is determined based on the character overlap rate between the multiple recognition results. For the i-th character in the string to be recognized, the character that appears most frequently at the i-th character position in the above recognition result group is determined as the i-th character in the string to be recognized. However, the above method requires a high character overlap rate between multiple recognition results, which requires the recognition results to ensure a certain length, and some photos that do not capture the complete string are difficult to use. Usually, the images to be recognized in the above four situations are discarded, or after character recognition is performed on the image to be recognized, the recognition results with a small number of characters in the obtained recognition results are discarded. However, this method cannot utilize the information of those images that fail to obtain the complete string to be recognized. For example, in the above 2 and 3, although the parking space number is not fully captured, part of the parking space number information can still be obtained in the image, and this part of the information can also be used to improve the accuracy of the final string recognition result.

[0057] Please refer to Fig.10 , which shows a flow chart of a string recognition result provided by an embodiment of the present application. The method can be executed by a computer device. The method can include at least one of the following steps 1010 to 1040.

[0058] Step 1010: Acquire multiple images to be recognized, each of which includes at least one character string to be recognized.

[0059] The image to be identified refers to an image obtained by photographing the target object. The target object refers to an object carrying a string to be identified. For example, in a vehicle-mounted car search system, the target object is a parking space, and the string to be identified is the parking space number; in a license plate recognition scenario, the target object is a vehicle, and the string to be identified is the license plate number; in an assembly line string recognition scenario, the target object is the goods on the assembly line, and the string to be identified is the code of the target object.

[0060] In some embodiments, the number of character strings to be recognized on a target object may be one or more, and the image obtained by photographing the target object includes at least one character string to be recognized.

[0061] For example, if the target object is a vehicle and its license plate needs to be identified, the image to be identified may only include one character string to be identified. For another example, if the target object is a product on an assembly line and the character strings to be identified include the product number, production date, and product license number, the image to be identified may include multiple character strings to be identified.

[0062] In some embodiments, the character string to be recognized may be composed of any characters, for example, the character string to be recognized may include Chinese characters, English characters, Western letters, fifty-note musical notation, Arabic numerals, Chinese symbols, English symbols, etc. Chinese symbols refer to symbols such as book title marks and dashes that appear in the Chinese context, and English symbols refer to symbols such as underscores and single quotation marks that appear in the English context.

[0063] Step 1020 , character string recognition is performed on the multiple images to be recognized to obtain N recognition results, where N is an integer greater than 1.

[0064] In some embodiments, character string recognition is performed on a plurality of images to be recognized, and a recognition result corresponding to each image to be recognized is obtained.

[0065] In some embodiments, if no character string is captured in the image to be recognized, the recognition result corresponding to the image to be recognized may not be determined.

[0066] In some embodiments, after character string recognition is performed on multiple images to be recognized, a recognition result corresponding to each image to be recognized is obtained, and the recognition result corresponding to each image to be recognized is screened to obtain N recognition results.

[0067] In some embodiments, a neural network model is used to perform string recognition on the image to be recognized.

[0068] In some embodiments, a string recognition model is used to perform string recognition on an image to be recognized. In some embodiments, the string recognition model can determine the position of a detection frame in the image to be recognized, and perform string recognition on the content in the detection frame to obtain a recognition result corresponding to the image to be recognized. The detection frame refers to the area range defined in the image to be recognized where a string recognition operation is required. The detection frame position refers to the position of the detection frame on the image to be recognized.

[0069] In some embodiments, when outputting a recognition result corresponding to an image to be recognized, the character string recognition model may output a detection box position in the image to be recognized and a confidence level of the recognition result corresponding to the image to be recognized.

[0070] In some embodiments, the string recognition model includes two parts, one is a detection model and the other is a recognition model. The detection model is used to detect the image to be recognized and determine the position of the detection frame, and the recognition model is used to perform string recognition on the content in the detection frame and determine the recognition result corresponding to the image to be recognized. In some embodiments, the detection model is also used to output the position of the detection frame, and the recognition model is also used to output the confidence of the recognition result corresponding to the recognized image.

[0071] In some embodiments, the detection model and the recognition model may be neural network models. For example, the detection model may be DBNet (Differentiable Binarization Network), and the recognition model may be CRNN (Convolutional Recurrent Neural Network).

[0072] Step 1030, calculating the similarities between the N recognition results, and obtaining similar labels corresponding to the N recognition results.

[0073] Based on the technical problem we want to solve, we hope to find the best one from several strings that may be correct, wrong or truncated. Under this premise, assuming that the string recognition technology we use is qualified and the method of obtaining multiple images to be recognized is reasonable (for example, the camera angle is normal), then there is reason to believe that the correct recognition result will occupy a considerable proportion of the N recognition results. If we calculate the similarity of all recognition results, we can infer that the correct recognition result has a higher similarity with all other strings. Therefore, it is reasonable to use the similarity labels corresponding to N recognition results to determine the final string recognition result.

[0074] The similarity label corresponding to the i-th recognition result among the N recognition results is used to characterize the similarity between the i-th recognition result and the other N-1 recognition results among the N recognition results except the i-th recognition result, where i is a positive integer less than or equal to N.

[0075] In some embodiments, for any two recognition results among the N recognition results, the similarity between the two recognition results is calculated.

[0076] In some embodiments, for any one of the N recognition results, the sum of similarities between the recognition result and other N-1 recognition results of the N recognition results is determined as the similarity label of the recognition result.

[0077] In some embodiments, for any one of the N recognition results, a weighted sum of similarities between the recognition result and other N-1 recognition results of the N recognition results is performed to obtain a similarity label of the recognition result.

[0078] In some embodiments, the weights corresponding to the similarities between the recognition result and the other N-1 recognition results in the N recognition results can be calculated based on the confidences corresponding to the N recognition results. In some embodiments, the weights can also be determined based on the detection frame positions and detection frame sizes corresponding to the images to be recognized, the lengths of the character strings corresponding to the N recognition results, etc., which are not limited in this application.

[0079] In some embodiments, the present application does not limit the similarity algorithm used to calculate the similarity between the N recognition results. For example, a cosine similarity algorithm, a Euclidean distance, and the like may be used.

[0080] Step 1040 , obtaining a character string recognition result based on similar tags corresponding to the N recognition results.

[0081] The string recognition result refers to the string recognition result corresponding to the string to be recognized that is finally obtained. In some embodiments, if there are multiple strings to be recognized, there are also multiple string recognition results.

[0082] In some embodiments, the number of character strings to be recognized is the same as the number of character string recognition results.

[0083] In some embodiments, there is one character string to be recognized, and among the similar tags corresponding to the N recognition results, a recognition result with the largest similar tag is selected and determined as the character string recognition result.

[0084] In some embodiments, if there are multiple character strings to be recognized, then among the similar tags corresponding to the N recognition results, multiple recognition results with the largest similar tags are selected and determined as the character string recognition results.

[0085] The technical solution provided in the embodiment of the present application obtains N recognition results by performing string recognition on the image to be recognized including the string to be recognized, determines similarity labels corresponding to the N recognition results based on the similarity between the N recognition results, and determines the string recognition result based on the similarity labels corresponding to the N recognition results. Since the recognition result corresponding to the image to be recognized that fails to obtain complete string information has a shorter string length, but the similarity between it and the real result corresponding to the string to be recognized is still high, the recognition result with a shorter string length can also be used in the embodiment of the present application, which can increase the similarity labels corresponding to the real result, so as to improve the accuracy of the string recognition result finally obtained.

[0086] In some embodiments, the above step 1030 may be implemented as at least one of the following steps 1031 to 1032 .

[0087] Step 1031 , for the ith recognition result, respectively calculate the similarities between the ith recognition result and the other N-1 recognition results among the N recognition results except the ith recognition result, to obtain N-1 similarities.

[0088] In some embodiments, the present application does not limit the similarity algorithm used to calculate the similarity between the i-th recognition result and the other N-1 recognition results. For ease of understanding, in the embodiments of the present application, the cosine similarity algorithm is used as an example for exemplary description.

[0089] In some embodiments, step 1031 may be implemented as at least one of the following steps a to b.

[0090] Step a, calculating a first similarity parameter between the i-th recognition result and the j-th recognition result among N recognition results, wherein the first similarity parameter is used to characterize the similarity between the i-th recognition result and the j-th recognition result in terms of the string content, where j is a positive integer less than or equal to N, and i is not equal to j.

[0091] In some embodiments, for each character in the i-th recognition result, a first similarity parameter is calculated based on the word frequency of the character in the i-th recognition result and the j-th recognition result.

[0092] In some embodiments, for each character in the i-th recognition result, the first word frequency and the second word frequency of the character are determined, the first word frequency refers to the frequency of the character appearing in the i-th recognition result, and the second word frequency refers to the frequency of the character appearing in the j-th recognition result; based on the first word frequency and the second word frequency of each different character in the i-th recognition result, the first similarity parameter is calculated.

[0093] In some embodiments, for any character in the i-th recognition result, the product of the first word frequency and the second word frequency of the character is calculated. The first similarity parameter is calculated based on the product of the first word frequency and the second word frequency corresponding to each character in the i-th recognition result.

[0094] Exemplarily, the first similarity parameter may be calculated using the following formula (1):

[0095]

[0096] Among them, x refers to the i-th recognition result, y refers to the j-th recognition result, and x i refers to the frequency of the i-th character in the i-th recognition result (i.e., the first word frequency corresponding to the i-th character in the i-th recognition result), y i It refers to the frequency of the i-th character in the i-th recognition result appearing in the j-th recognition result (i.e., the second word frequency corresponding to the i-th character in the i-th recognition result), n is the number of characters included in the i-th recognition result, and S refers to the first similarity parameter.

[0097] Step b: determining the similarity between the i-th recognition result and the j-th recognition result based on the first similarity parameter.

[0098] In some embodiments, the first similarity parameter is determined as the similarity between the i-th recognition result and the j-th recognition result.

[0099] For example, formula (1) can be written as:

[0100]

[0101] Among them, x refers to the i-th recognition result, y refers to the j-th recognition result, and x i refers to the frequency of the i-th character in the i-th recognition result (i.e., the first word frequency corresponding to the i-th character in the i-th recognition result), y i It refers to the frequency of the i-th character in the i-th recognition result appearing in the j-th recognition result (i.e., the second word frequency corresponding to the i-th character in the i-th recognition result), n is the number of characters included in the i-th recognition result, S xy It refers to the similarity between the i-th recognition result and the j-th recognition result.

[0102] In some embodiments, in addition to the first similarity parameter, the first similarity parameter and the second similarity parameter may be used to jointly determine the similarity between the i-th recognition result and the j-th recognition result.

[0103] In some embodiments, a second similarity parameter is calculated between the i-th recognition result and the j-th recognition result, and the second similarity parameter is used to characterize the similarity between the i-th recognition result and the j-th recognition result in terms of string features; based on the first similarity parameter and the second similarity parameter, the similarity between the i-th recognition result and the j-th recognition result is determined.

[0104] In some embodiments, the character string feature may include at least one of the following: character string length, detection box size corresponding to the character string, detection box position, and confidence level corresponding to the character string.

[0105] In some embodiments, a first feature parameter is calculated based on the length of the string corresponding to the i-th recognition result and the length of the string corresponding to the j-th recognition result, and the first feature parameter is used to distinguish the reliability of the i-th recognition result and the j-th recognition result based on the string length.

[0106] In some embodiments, the first feature parameter is calculated based on the ratio of the length of the character string corresponding to the ith recognition result to the length of the character string corresponding to the ith recognition result.

[0107] In some embodiments, the first feature parameter is calculated based on the difference between the length of the character string corresponding to the ith recognition result and the expected length of the character string, and the difference between the length of the character string corresponding to the ith recognition result and the expected length of the character string.

[0108] In some embodiments, the expected length of the string refers to the length of the preset correct string recognition result. The method is applicable to recognition scenarios where the length of the string is known. For example, in the license plate recognition scenario, the length of the license plate is generally fixed, so the expected length of the string is known.

[0109] In some embodiments, a second feature parameter is calculated based on a detection box position corresponding to the i-th recognition result and a detection box position corresponding to the j-th recognition result, and the second feature parameter is used to distinguish the reliability of the i-th recognition result and the j-th recognition result based on the detection box position.

[0110] In some embodiments, a second feature parameter is calculated based on the distance between the detection box position corresponding to the i-th recognition result and the preset position in the image to be recognized corresponding to the i-th recognition result, and the distance between the detection box position corresponding to the j-th recognition result and the preset position in the image with recognition corresponding to the j-th recognition result.

[0111] In some embodiments, the preset position in the image to be recognized refers to a desired detection frame position in the image to be recognized. In other words, the preset position in the image to be recognized refers to a detection frame position corresponding to a correct character string recognition result.

[0112] In some embodiments, the preset positions in different images to be recognized have a corresponding relationship. For example, the preset positions in different images to be recognized are all the center positions of the images to be recognized.

[0113] In some embodiments, the preset position in the image to be identified is related to the acquisition method of the image to be identified. Since the preset position in the image to be identified needs to be determined based on the distance and angle between the position of the camera that captures the image to be identified and the target object, this method is more suitable for scenes where a fixed camera is used for shooting, such as pipeline string recognition scenes.

[0114] In some embodiments, a third feature parameter is calculated based on the detection frame size corresponding to the i-th recognition result and the detection frame size corresponding to the j-th recognition result, and the third feature parameter is used to distinguish the reliability of the i-th recognition result and the j-th recognition result based on the detection frame size.

[0115] In some embodiments, the size of the detection box can reflect the length of the character string to a certain extent. Based on the size of the detection box and the length of the character string of the recognition result, the accuracy of the character string recognition model can also be estimated to a certain extent.

[0116] In some embodiments, the third feature parameter is calculated based on the detection box area corresponding to the i-th recognition result and the detection box area corresponding to the j-th recognition result.

[0117] In some embodiments, in some embodiments, the third feature parameter is calculated based on the ratio between the area of ​​the detection box corresponding to the i-th recognition result and the area of ​​the detection box corresponding to the j-th recognition result.

[0118] In some embodiments, a fourth characteristic parameter is calculated based on the confidence corresponding to the i-th recognition result and the confidence corresponding to the j-th recognition result, and the fourth characteristic parameter is used to distinguish the reliability of the i-th recognition result and the j-th recognition result based on the confidence.

[0119] In some embodiments, the fourth characteristic parameter is calculated based on the ratio between the confidence level corresponding to the i-th recognition result and the confidence level corresponding to the j-th recognition result.

[0120] In some embodiments, the second similarity parameter is calculated based on any one or more of the first characteristic parameter, the second characteristic parameter, the third characteristic parameter, and the fourth characteristic parameter.

[0121] In some embodiments, a second similarity parameter is calculated based on the first characteristic parameter, the second characteristic parameter, the third characteristic parameter and the fourth characteristic parameter.

[0122] In some embodiments, the second similarity parameter is calculated based on a weighted sum of the first feature parameter, the second feature parameter, the third feature parameter, and the fourth feature parameter.

[0123] In some embodiments, since the relationship between different string features and the accuracy of the recognition result is different, different weights can be assigned to different feature parameters to perform weighted summation.

[0124] In some embodiments, the similarity between the ith recognition result and the jth recognition result is determined based on the product of the first similarity parameter and the second similarity parameter.

[0125] In one example, the following formula (2) can be used to calculate the similarity between the i-th recognition result and the j-th recognition result:

[0126]

[0127] Among them, S xy It refers to the similarity between the i-th recognition result and the j-th recognition result. is the first similarity parameter, refers to the second similarity parameter, L x refers to the length of the string of the i-th recognition result, L y refers to the length of the string between the jth recognition results, L refers to the expected string length, α refers to the weight corresponding to the first feature parameter, and D x It refers to the distance between the detection frame position corresponding to the i-th recognition result and the preset position of the image to be recognized corresponding to the i-th recognition result. y It refers to the distance between the detection box position corresponding to the jth recognition result and the preset position of the image to be recognized corresponding to the jth recognition result, β refers to the weight corresponding to the second feature parameter, A x Refers to the detection box area corresponding to the i-th recognition result, A y refers to the detection box area corresponding to the jth recognition result, γ refers to the weight corresponding to the third feature parameter, C x refers to the confidence corresponding to the i-th recognition result, C y refers to the confidence level corresponding to the jth recognition result, δ refers to the weight corresponding to the fourth feature parameter. x refers to the i-th recognition result, y refers to the j-th recognition result, x i refers to the frequency of the i-th character in the i-th recognition result (i.e., the first word frequency corresponding to the i-th character in the i-th recognition result), y i It refers to the frequency of the i-th character in the i-th recognition result appearing in the j-th recognition result (ie, the second word frequency corresponding to the i-th character in the i-th recognition result), and n is the number of characters included in the i-th recognition result.

[0128] In another example, the following formula (3) can be used to calculate the similarity between the i-th recognition result and the j-th recognition result:

[0129]

[0130] Among them, S xy It refers to the similarity between the i-th recognition result and the j-th recognition result. is the first similarity parameter, L x refers to the length of the string of the i-th recognition result, L y refers to the length of the string between the jth recognition results. x refers to the i-th recognition result, y refers to the j-th recognition result, x i refers to the frequency of the i-th character in the i-th recognition result (i.e., the first word frequency corresponding to the i-th character in the i-th recognition result), y i It refers to the frequency of the i-th character in the i-th recognition result appearing in the j-th recognition result (ie, the second word frequency corresponding to the i-th character in the i-th recognition result), and n is the number of characters included in the i-th recognition result.

[0131] In some embodiments, based on actual needs, one formula from the above formulas (1) to (3) can be selected to calculate the similarity between the i-th recognition result and the j-th recognition result.

[0132] Step 1032, summing up the N-1 similarities to obtain a similar label corresponding to the i-th recognition result.

[0133] In some embodiments, the similarity tag corresponding to the i-th recognition result may be calculated using the following formula (4):

[0134]

[0135] Among them, F x refers to the similarity label corresponding to the i-th recognition result, x refers to the i-th recognition result, y refers to the j-th recognition result, S xy It refers to the similarity between the i-th recognition result and the j-th recognition result, and N refers to the N recognition results.

[0136] Take N recognition results, cat, cat, dog, cat, c, at, cae, door, as an example. Formula (1) and formula (4) are used to calculate the similarity labels corresponding to cat and dog respectively. Then, the similarity label corresponding to cat is 1+0+1+0.53+0.82+0.67+0+=4.02, and the similarity label corresponding to dog is 0+0.71=0.71. It can be seen that the similarity label of cat is much larger than that of dog. It can be seen that under normal detection and recognition capabilities, we can use this method to select the one that is "most similar" to all other recognition results from all recognition results, so as to estimate the correct string recognition result with the greatest possibility.

[0137] Taking N recognition results as cat, cat, dat, dat, c, at as an example, using formula (1) and formula (4) to calculate the similarity labels corresponding to cat and dat respectively, we can get the similarity label corresponding to cat as 1+0.67+0.67+0.58+0.82=3.74, and the similarity label corresponding to dat as 0.67+0.67+1+0+0.82=3.16. It can be seen that even when the number of cat and dat is the same, cat can still be identified as the correct string recognition result. The reason is that c and at, which may be truncated due to taking pictures, can still contribute more points to the correct string recognition result cat in the similarity calculation. In other words, through the calculation of similarity, we can not only take advantage of the fact that the correct string recognition results are more when the accuracy of the string recognition model is acceptable, but also use the partially truncated but correctly recognized recognition results to increase their scores, thereby effectively utilizing a variety of information and ultimately providing a more reliable string recognition result.

[0138] In some embodiments, the above method can be written in JAVA programming language as the following algorithm:

[0139]

[0140]

[0141] In the above embodiment, taking the cosine similarity algorithm as an example, the method of determining a string recognition result from N recognition results in the embodiment of the present application is exemplified. It can take advantage of the fact that the correct recognition result accounts for a larger proportion of the N recognition results, and can also use truncated or incompletely recognized recognition results to increase the score of the correct recognition result. It can effectively utilize a variety of information and improve the accuracy of the obtained string recognition result.

[0142] In some embodiments, after the recognition results corresponding to the images to be recognized are screened, the subsequent step of determining the character string recognition result is performed, which can reduce the number of recognition results that need to be calculated and the amount of data that needs to be processed, thereby improving the processing speed.

[0143] In some embodiments, the above step 1020 may be implemented as at least one of the following steps 1021 to 1022 .

[0144] Step 1021 , character string recognition is performed on the multiple images to be recognized to obtain recognition results corresponding to the multiple images to be recognized.

[0145] Step 1022, screening the recognition results corresponding to the multiple images to be recognized to obtain N recognition results.

[0146] In some embodiments, a recognition result satisfying the first condition is selected from recognition results corresponding to a plurality of images to be recognized, to obtain N recognition results.

[0147] In some embodiments, the first condition includes at least one of the following: the detection box size corresponding to the recognition result is greater than or equal to the first threshold, the confidence level corresponding to the recognition result is greater than or equal to the second threshold, and the recognition result satisfies the known features of the character string to be recognized.

[0148] In some embodiments, the first threshold, the second threshold, and the known features of the character string to be identified may be determined based on an actual application scenario.

[0149] In some embodiments, the first threshold may be a threshold corresponding to the detection frame area. For example, the first condition is that the detection frame area is greater than or equal to 20 cm 2 In some embodiments, the first threshold may also be a threshold corresponding to the length and width of the detection box, for example, the first condition is that the length of the detection box is greater than or equal to 5 cm, and the width of the detection box is greater than or equal to 3 cm. This condition can exclude detection boxes that are small or unreasonable, so as to exclude content that is misdetected or difficult to recognize by the string recognition model.

[0150] In some embodiments, the second threshold can be determined based on the string recognition model used. If the string recognition model used is more powerful, the second threshold is also set larger. The confidence level can be used to filter out recognition results with lower credibility and retain recognition results with higher credibility.

[0151] In some embodiments, the known features of the string to be recognized refer to the basic features that the string to be recognized needs to meet. For example, in the license plate recognition scenario, the string to be recognized needs to meet the first character is a Chinese character. Using known features to filter the recognition results can eliminate obviously wrong recognition results.

[0152] It should be noted that the above is only an example of the first condition, and the first condition may also be other conditions, and this application does not limit this.

[0153] According to the above method, we can select the most reliable string recognition result by sorting, but sometimes the actual scene requires us to do more things, such as our camera can capture many parking space numbers (surround view camera), and we also hope to find an alternative, or there are multiple items of labels captured on the assembly line, and we hope to find all of them. The technical solution provided in the embodiment of this application can also be implemented. Next, the technical solution of the embodiment of this application will be exemplified by taking the case where there are multiple strings to be recognized as an example.

[0154] In some embodiments, after the above step 1020, at least one of the following steps a to e is also included.

[0155] Step a, performing deduplication processing on N recognition results to obtain K non-repetitive recognition results, where K is a positive integer less than or equal to N.

[0156] In some embodiments, deduplication processing refers to screening repeated recognition results from N recognition results and retaining only one recognition result. For example, if the N recognition results are cat, cat, dat, dog, and c, the K non-duplicate recognition results obtained after deduplication processing are cat, dat, dog, and c.

[0157] Step b: determining at least one seed recognition result group from the K non-repetitive recognition results, each seed recognition result group including M seed recognition results, where M is a positive integer less than or equal to K.

[0158] In some embodiments, among K non-repetitive recognition results, M recognition results are randomly determined as a seed recognition result group.

[0159] In some embodiments, since the M seed recognition results are randomly selected, the calculation results may be biased, so it is necessary to select multiple seed recognition result groups for calculation to reduce the error caused by random selection.

[0160] In some embodiments, the value of M may be determined by any of the following methods:

[0161] M is the number of detection frames included in an image to be identified; or,

[0162] M is the number of character strings whose length is greater than the third threshold; or,

[0163] M is determined based on the elbow rule; or,

[0164] M is determined based on the silhouette coefficient.

[0165] In some embodiments, the number of detection frames included in the images to be identified is determined, and the mode of the number of detection frames included in the images to be identified is determined as the value of M. Exemplarily, the number of detection frames corresponding to the images to be identified is 1, 3, 3, 3, 5, and 4, and 3 is determined as the value of M.

[0166] In some embodiments, the third threshold may be determined based on the length of the character string to be recognized. For example, if the length of the character string to be recognized is 5, the third threshold may be determined to be 3.

[0167] In some embodiments, M may also be the number of character strings whose character string lengths are less than the third threshold. Specifically, greater than, less than or equal to needs to be determined in combination with actual conditions.

[0168] Elbow rule: We know that k-means takes minimizing the square error between samples and particles as the objective function. The sum of the square distance errors between the particles of each cluster and the sample points within the cluster is called the degree of distortion. Then, for a cluster, the lower the degree of distortion, the closer the members within the cluster are, and the higher the degree of distortion, the looser the structure within the cluster. The degree of distortion will decrease with the increase of categories, but for data with a certain degree of differentiation, the degree of distortion will be greatly improved when it reaches a certain critical point, and then slowly decrease. This critical point can be considered as a point with better clustering performance. The elbow rule is used to determine M, that is, the above critical point is determined as M.

[0169] Silhouette coefficient: For a clustering task, we hope that the category clusters we obtain are as close as possible within the clusters and as far apart as possible. The silhouette coefficient is an evaluation indicator of the density and dispersion of the clusters.

[0170] In some embodiments, the inverse of the similarity between the seed recognition result and the K non-repeating recognition results may be used as the distance between the seed recognition result and the K non-repeating recognition results to calculate the elbow rule or silhouette coefficient.

[0171] Step c: for any seed recognition result group in at least one seed recognition result group, based on the M seed recognition results in the seed recognition result group, clustering K non-repetitive recognition results to obtain M recognition result sets.

[0172] In some embodiments, a K-means clustering method may be used to cluster K non-repetitive recognition results. Of course, other clustering methods may also be used, and this application does not limit this.

[0173] In some embodiments, the K non-repeating recognition results are clustered based on the similarity between KM non-repeating recognition results other than the M seed recognition results and the M seed recognition results.

[0174] In some embodiments, for any one of the remaining KM recognition results except the M seed recognition results, the recognition result is divided into a recognition result set where a seed recognition result having the highest similarity to the recognition result is located.

[0175] In some embodiments, for the p-th recognition result among the remaining KM recognition results except the M seed recognition results, the similarity between the p-th recognition result and the M seed recognition results is calculated respectively, where p is a positive integer less than or equal to KM; the p-th recognition result is divided into the a-th recognition result set, where the a-th recognition result set includes the a-th seed recognition result, and the similarity between the p-th recognition result and the a-th seed recognition result is greater than the similarity between the p-th recognition result and the other M-1 seed recognition results, where a is a positive integer less than or equal to M.

[0176] Step d: based on the M recognition result sets, determine the character string recognition result corresponding to the seed recognition result group.

[0177] In some embodiments, for any one of the M recognition result sets, the recognition result in the recognition result set with the largest similarity label with other recognition results in the recognition result set is determined as the updated seed recognition result corresponding to the recognition result set.

[0178] In some embodiments, the above formula (1) and formula (4) are used to calculate the similarity labels corresponding to each recognition result in the recognition result set.

[0179] In some embodiments, for the ath recognition result set, the seed recognition result corresponding to the ath recognition result set is updated to obtain the ath updated seed recognition result; when the second condition is met, the M updated seed recognition results are determined as the string recognition results corresponding to the seed recognition result group; wherein the second condition is that among the M updated seed recognition results, the number of seed recognition results that are the same as the M seed recognition results is greater than a second threshold; when the second condition is not met, the M updated seed recognition results are used as the M seed recognition results, and the execution is started again from the step of calculating the similarity between the pth recognition result and the M seed recognition results for the pth recognition result among the remaining KM recognition results except the M seed recognition results.

[0180] In other words, the second condition means that in the step of updating the seed recognition result corresponding to the a-th recognition result set to obtain the a-th updated seed recognition result for the a-th recognition result set, the number of seed recognition results to be updated is less than the fifth threshold. The sum of the fifth threshold and the second threshold is M.

[0181] In some embodiments, among the M updated seed identification results, if the number of seed identification results that are the same as the M seed identification results is equal to the second threshold, it can be considered that the second condition is met, or it can be considered that the second condition is not met, and the specific situation should be based on actual application.

[0182] In some embodiments, the second threshold may be a numerical value, or a ratio or a percentage, which is not limited in the present application.

[0183] Step e: determining a character string recognition result based on the character string recognition results respectively corresponding to at least one seed recognition result group.

[0184] In some embodiments, for any one of the at least one seed recognition result groups, a similarity parameter of the string recognition result corresponding to the seed recognition result group is calculated, and the similarity parameter is used for the similarity between each recognition result included in the recognition result set corresponding to the string recognition result and the string recognition result; based on the similarity parameters of the string recognition results corresponding to the at least one seed recognition result group, the string recognition result is determined.

[0185] In some embodiments, for a string recognition result in the string recognition results corresponding to any seed recognition result group, the sum of similarities between the string recognition result and each recognition result in the recognition result set is calculated to determine the similarity parameter corresponding to the string recognition result.

[0186] In some embodiments, M string recognition results with the largest similarity parameters corresponding to the string recognition results included in each seed recognition result group are determined as the string recognition result.

[0187] In some embodiments, the M character string recognition results are M non-repetitive character string recognition results.

[0188] Through the above method, multiple character strings to be identified can be identified, and the use of multiple clustering methods can avoid the deviation caused by randomly selecting seed recognition results.

[0189] This application also provides an exemplary embodiment for how to collect multiple images to be recognized.

[0190] In some embodiments, the method may include at least one of the following steps a to c.

[0191] Step a: When the start condition is met, the character string to be recognized is photographed.

[0192] Step b, in response to the position offset corresponding to the target object exceeding the fourth threshold, photographing the character string to be recognized again, the position offset corresponding to the target object refers to the position offset of the target object, or refers to the position offset of the device photographing the target object.

[0193] Step c: when the end condition is met, stop shooting the image to be identified.

[0194] In some embodiments, the multiple images to be identified include: between the moment when the start condition is met and the moment when the end condition is met, each time the position offset corresponding to the target object reaches or exceeds the fourth threshold, an image to be identified is taken, and finally multiple images to be identified are obtained.

[0195] In some embodiments, the camera used to capture multiple images to be identified may be a camera of a computer device that executes the method of an embodiment of the present application, or an external camera device that has a communication connection with the computer device that executes the method of an embodiment of the present application. The present application does not limit this.

[0196] In some embodiments, the start condition includes at least one of the following: the target object enters the image acquisition range, and the target application enters the application process; the end condition includes at least one of the following: the target object moves outside the image acquisition range, and the target application ends the application process; the image acquisition range refers to the range within which the character string to be recognized can be photographed, and the application process needs to obtain the character string recognition result.

[0197] Taking the in-vehicle vehicle search system as an example, the start condition should be that the target application enters the application process, such as the target application detects that the vehicle is parked. The end condition should be that the target application ends the application process, such as the target application detects that the vehicle is turned off.

[0198] Taking the pipeline string recognition scenario as an example, the start condition should be that the target object enters the image acquisition range. For example, the product enters the camera's shooting range. The end condition should be that the target object moves out of the image acquisition range, for example, the product moves out of the camera's shooting range along the pipeline.

[0199] In some embodiments, in order to ensure that the contents of the multiple images to be identified are different, so as to avoid taking multiple images to be identified for the character string to be identified under occlusion, resulting in inaccurate final character string recognition results, the position offset of the target object can be used as the timing standard for collecting the image to be identified.

[0200] The position offset of the target object refers to the relative position offset between the target object and the camera.

[0201] Taking the vehicle-mounted car search system as an example, the target object refers to the parking space, and the camera is located on the vehicle. Although the parking space does not move, the position offset of the target object will change with the movement of the vehicle.

[0202] In some embodiments, in a vehicle-mounted vehicle search system, the position offset of the target object may be determined based on data from the vehicle odometer.

[0203] In some embodiments, during license plate inspection (such as license plate inspection when entering a parking lot), the moving distance of the target object can be determined through the target detection network. For example, the moving distance of the vehicle can be determined based on the size of the target detection box in which the vehicle is detected.

[0204] In some embodiments, the moving speed of the pipeline is uniform, and the position offset of the target object can be determined based on time.

[0205] Through the above method, it can be ensured that the contents included in the multiple images to be recognized are different, and it can be avoided that the target objects in the same scene are photographed, thereby affecting the accuracy of the final string recognition result.

[0206] In some embodiments, Fig.11 As shown, the technical solution provided by the embodiment of the present application may include the following steps: when the start condition is met, start shooting the image to be identified (collecting data); when the stop condition is not reached, shoot the target object based on the position offset of the target object; after the stop condition is reached, perform OCR recognition on the image to be identified, obtain multiple recognition results, filter the multiple recognition results (filter recognition results), obtain N recognition results, calculate similar labels corresponding to the N recognition results (calculate weighted cosine similarity), and determine the string recognition result based on the similar labels corresponding to the N recognition results.

[0207] The technical solution of the embodiment of the present application can be applied to various string recognition scenarios. Taking the vehicle search system as an example, Fig.12 As shown, the target application can identify the parking space number 1220 based on the collected parking space image 1210, record and inform the user of the parking space number 1220, which can greatly reduce the time and energy cost in the actual car search process. The user can also bind the in-vehicle system with other applications, so that the parking space number can be viewed on other terminal devices. Fig.13 As shown, the user can bind the vehicle-mounted system with the mobile phone and view the parking space number 1320 through the user interface 1310 of the mobile client of the target application.

[0208] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0209] Please refer to Fig.14 , which shows a block diagram of a string recognition device provided by an embodiment of the present application. The device has the function of implementing the above-mentioned string recognition method, and the function can be implemented by hardware, or by hardware executing corresponding software. The device can be the computer device introduced above, or it can be set in a computer device. Fig.14 As shown, the device 1400 may include: an acquisition module 1410 , an identification module 1420 , a calculation module 1430 and a obtaining module 1440 .

[0210] The acquisition module 1410 is used to acquire a plurality of images to be recognized, each of which includes at least one character string to be recognized;

[0211] The recognition module 1420 is used to perform character string recognition on the multiple images to be recognized to obtain N recognition results, where N is an integer greater than 1.

[0212] The calculation module 1430 is used to calculate the similarity between the N recognition results and obtain similarity labels corresponding to the N recognition results respectively; wherein the similarity label corresponding to the i-th recognition result among the N recognition results is used to characterize the similarity between the i-th recognition result and the other N-1 recognition results among the N recognition results except the i-th recognition result, where i is a positive integer less than or equal to N.

[0213] The obtaining module 1440 is used to obtain a character string recognition result based on similar tags corresponding to the N recognition results.

[0214] In some embodiments, the calculation module 1430 is used to calculate the similarity between the i-th recognition result and the other N-1 recognition results among the N recognition results except the i-th recognition result, to obtain N-1 similarities; and sum the N-1 similarities to obtain a similarity label corresponding to the i-th recognition result.

[0215] In some embodiments, the calculation module 1430 is used to calculate a first similarity parameter between the i-th recognition result and the j-th recognition result among the N recognition results, wherein the first similarity parameter is used to characterize the similarity between the i-th recognition result and the j-th recognition result in string content, j is a positive integer less than or equal to N, and i is not equal to j; based on the first similarity parameter, the similarity between the i-th recognition result and the j-th recognition result is determined.

[0216] In some embodiments, the calculation module 1430 is used to determine, for each character in the i-th recognition result, a first word frequency and a second word frequency of the character, wherein the first word frequency refers to the frequency of the character appearing in the i-th recognition result, and the second word frequency refers to the frequency of the character appearing in the j-th recognition result; the first similarity parameter is calculated based on the first word frequency and the second word frequency of each different character in the i-th recognition result.

[0217] In some embodiments, the calculation module 1430 is used to calculate a second similarity parameter between the i-th recognition result and the j-th recognition result, and the second similarity parameter is used to characterize the similarity between the i-th recognition result and the j-th recognition result in string features; determining the similarity between the j-th recognition result and the j-th recognition result based on the first similarity parameter includes: determining the similarity between the i-th recognition result and the j-th recognition result based on the first similarity parameter and the second similarity parameter.

[0218] In some embodiments, the calculation module 1430 is used to calculate a first feature parameter based on the length of the string corresponding to the i-th recognition result and the length of the string corresponding to the j-th recognition result, the first feature parameter being used to distinguish the reliability of the i-th recognition result and the j-th recognition result based on the string length; calculate a second feature parameter based on the position of the detection frame corresponding to the i-th recognition result and the position of the detection frame corresponding to the j-th recognition result, the second feature parameter being used to distinguish the reliability of the i-th recognition result and the j-th recognition result based on the detection frame position; calculate a second feature parameter based on the i-th recognition result corresponding to the j-th recognition result corresponding to the detection frame position; calculate a second feature parameter based on the i-th recognition result corresponding to the j-th recognition result corresponding to the detection frame position. The third feature parameter is calculated based on the detection frame size corresponding to the i-th recognition result and the detection frame size corresponding to the j-th recognition result, and the third feature parameter is used to distinguish the reliability of the i-th recognition result and the j-th recognition result based on the detection frame size; the fourth feature parameter is calculated based on the confidence level corresponding to the i-th recognition result and the confidence level corresponding to the j-th recognition result, and the fourth feature parameter is used to distinguish the reliability of the i-th recognition result and the j-th recognition result based on the confidence level; the second similarity parameter is calculated based on the first feature parameter, the second feature parameter, the third feature parameter and the fourth feature parameter.

[0219] In some embodiments, the calculation module 1430 is used to determine the similarity between the i-th recognition result and the j-th recognition result based on the product of the first similarity parameter and the second similarity parameter.

[0220] In some embodiments, the recognition module 1420 is used to perform character string recognition on the multiple images to be recognized to obtain recognition results corresponding to the multiple images to be recognized respectively;

[0221] The recognition results corresponding to the plurality of images to be recognized are screened to obtain the N recognition results.

[0222] In some embodiments, the recognition module 1420 is used to select a recognition result that satisfies a first condition from the recognition results corresponding to the plurality of images to be recognized, to obtain the N recognition results;

[0223] Among them, the first condition includes at least one of the following: the detection box size corresponding to the recognition result is greater than or equal to a first threshold, the confidence level corresponding to the recognition result is greater than or equal to a second threshold, and the recognition result satisfies the known features of the character string to be recognized.

[0224] In some embodiments, Fig.15 As shown, the device 1400 also includes a clustering module 1450 .

[0225] The clustering module 1450 is used to deduplicate the N recognition results to obtain K non-repeating recognition results, where K is a positive integer less than or equal to N; among the K non-repeating recognition results, at least one seed recognition result group is determined, each seed recognition result group includes M seed recognition results, and M is a positive integer less than or equal to K; for any seed recognition result group in the at least one seed recognition result group, based on the M seed recognition results in the seed recognition result group, the K non-repeating recognition results are clustered to obtain M recognition result sets; based on the M recognition result sets, the string recognition result corresponding to the seed recognition result group is determined; based on the string recognition results respectively corresponding to the at least one seed recognition result group, the string recognition result is determined.

[0226] In some embodiments, the clustering module 1450 is used to calculate, for any one of the at least one seed recognition result groups, a similarity parameter of a string recognition result corresponding to the seed recognition result group, wherein the similarity parameter is used to characterize the similarity between each recognition result included in a recognition result set corresponding to the string recognition result and the string recognition result; and determine the string recognition result based on the similarity parameters of the string recognition results corresponding to each of the at least one seed recognition result groups.

[0227] In some embodiments, the clustering module 1450 is used to calculate the similarity between the p-th recognition result and the M seed recognition results for the remaining KM recognition results except the M seed recognition results, where p is a positive integer less than or equal to KM; divide the p-th recognition result into the a-th recognition result set, the a-th recognition result set includes the a-th seed recognition result, and the similarity between the p-th recognition result and the a-th seed recognition result is greater than the similarity between the p-th recognition result and other M-1 seed recognition results, where a is a positive integer less than or equal to M.

[0228] In some embodiments, the clustering module 1450 is used to update the seed recognition result corresponding to the a-th recognition result set for the a-th recognition result set to obtain the a-th updated seed recognition result; when the second condition is met, the M updated seed recognition results are determined as the string recognition results corresponding to the seed recognition result group; wherein the second condition is that among the M updated seed recognition results, the number of seed recognition results that are the same as the M seed recognition results is greater than a second threshold; when the second condition is not met, the M updated seed recognition results are used as M seed recognition results, and the execution is started again from the step of respectively calculating the similarity between the p-th recognition result and the M seed recognition results for the p-th recognition result among the remaining KM recognition results except the M seed recognition results.

[0229] In some embodiments, M is the number of detection boxes included in an image to be identified; or, M is the number of character strings whose length is greater than a third threshold; or, M is determined based on the elbow rule; or, M is determined based on a silhouette coefficient.

[0230] In some embodiments, the acquisition module 1410 is used to photograph the character string to be identified when a start condition is met; in response to a position offset corresponding to the target object exceeding a fourth threshold, photograph the character string to be identified again, where the position offset corresponding to the target object refers to the position offset of the target object, or refers to the position offset of a device that photographs the target object; and stop photographing the image to be identified when an end condition is met.

[0231] The technical solution provided in the embodiment of the present application obtains N recognition results by performing string recognition on the image to be recognized including the string to be recognized, determines similarity labels corresponding to the N recognition results based on the similarity between the N recognition results, and determines the string recognition result based on the similarity labels corresponding to the N recognition results. Since the recognition result corresponding to the image to be recognized that fails to obtain complete string information has a shorter string length, but the similarity between it and the real result corresponding to the string to be recognized is still high, the recognition result with a shorter string length can also be used in the embodiment of the present application, which can increase the similarity labels corresponding to the real result, so as to improve the accuracy of the string recognition result finally obtained.

[0232] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the content structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0233] Please refer to Fig.16 , which shows a block diagram of a computer device 1600 provided in one embodiment of the present application. The computer device 1600 may be any electronic device with data calculation, processing and storage functions. The computer device 1600 may be used to implement the string recognition method provided in the above embodiment.

[0234] Typically, the computer device 1600 includes a processor 1601 and a memory 1602 .

[0235] The processor 1601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1601 may be integrated with a GPU (Graphic Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1601 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0236] The memory 1602 may include one or more computer-readable storage media, which may be non-transitory. The memory 1602 may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1602 is used to store a computer program, which is configured to be executed by one or more processors to implement the above-mentioned string recognition method.

[0237] Those skilled in the art will understand that Fig.16 The structure shown in the figure does not constitute a limitation on the computer device 1600, and the computer device 1600 may include more or less components than shown in the figure, or combine some components, or adopt a different arrangement of components.

[0238] In an illustrative embodiment, a computer-readable storage medium is also provided, wherein a computer program is stored in the storage medium, and the computer program implements the above-mentioned string recognition method when executed by a processor of a computer device. Optionally, the above-mentioned computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0239] In an exemplary embodiment, a computer program product is also provided, the computer program product includes a computer program, the computer program is stored in a computer-readable storage medium, a processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the above-mentioned string recognition method.

[0240] It should be noted that this application can display a prompt interface, pop-up window or output voice prompt information before collecting relevant data of users and during the process of collecting relevant data of users. The prompt interface, pop-up window or voice prompt information is used to prompt the user that relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining relevant data of users after obtaining the confirmation operation issued by the user to the prompt interface or pop-up window, otherwise (that is, when the confirmation operation issued by the user to the prompt interface or pop-up window is not obtained), the relevant steps of obtaining relevant data of users are terminated, that is, the relevant data of users are not obtained. In other words, all user data collected by this application (including license plate information of vehicles and string information of assembly line products) are processed strictly in accordance with the requirements of relevant national laws and regulations, and the informed consent or separate consent of the subject of personal information is obtained with the consent and authorization of the user. The subsequent data use and processing behavior is carried out within the scope of authorization of laws and regulations and the subject of personal information, and the collection, use and processing of relevant user data shall comply with the relevant laws, regulations and standards of relevant countries and regions.

[0241] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application are not limited to this.

[0242] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for character string recognition, characterized in that: The method comprises: Acquire multiple images to be identified, each of which includes at least one character string to be identified; Performing character string recognition on the plurality of images to be recognized to obtain N recognition results, where N is an integer greater than 1; Calculate the similarity between the N recognition results to obtain similarity labels corresponding to the N recognition results respectively; wherein the similarity label corresponding to the i-th recognition result among the N recognition results is used to characterize the similarity between the i-th recognition result and the other N-1 recognition results among the N recognition results except the i-th recognition result, where i is a positive integer less than or equal to N; Based on the similar labels respectively corresponding to the N recognition results, a character string recognition result is obtained.

2. The method according to claim 1, characterized in that The calculating the similarities between the N recognition results to obtain similar labels corresponding to the N recognition results respectively includes: For the i-th recognition result, respectively calculate the similarities between the i-th recognition result and other N-1 recognition results among the N recognition results except the i-th recognition result, to obtain N-1 similarities; The N-1 similarities are summed to obtain a similar label corresponding to the i-th recognition result.

3. The method according to claim 2, characterized in that The respectively calculating the similarities between the i-th recognition result and other N-1 recognition results among the N recognition results except the i-th recognition result to obtain N-1 similarities includes: Calculate a first similarity parameter between the i-th recognition result and the j-th recognition result among the N recognition results, wherein the first similarity parameter is used to characterize the similarity between the i-th recognition result and the j-th recognition result in terms of string content, where j is a positive integer less than or equal to N, and i is not equal to j; Based on the first similarity parameter, a similarity between the i-th recognition result and the j-th recognition result is determined.

4. The method according to claim 3, characterized in that The calculating a first similarity parameter between the i-th recognition result and the j-th recognition result among the N recognition results includes: For each character in the i-th recognition result, determine a first word frequency and a second word frequency of the character, wherein the first word frequency refers to the frequency of occurrence of the character in the i-th recognition result, and the second word frequency refers to the frequency of occurrence of the character in the j-th recognition result; The first similarity parameter is calculated based on the first word frequency and the second word frequency of each different character in the i-th recognition result.

5. The method according to claim 3, characterized in that: The method further comprises: Calculating a second similarity parameter between the i-th recognition result and the j-th recognition result, wherein the second similarity parameter is used to characterize the similarity between the i-th recognition result and the j-th recognition result in terms of character string features; The determining, based on the first similarity parameter, the similarity between the j-th recognition result and the j-th recognition result comprises: Based on the first similarity parameter and the second similarity parameter, a similarity between the i-th recognition result and the j-th recognition result is determined.

6. The method according to claim 5, characterized in that The calculating a second similarity parameter between the i-th recognition result and the j-th recognition result includes: Calculating a first characteristic parameter based on the length of the character string corresponding to the i-th recognition result and the length of the character string corresponding to the j-th recognition result, wherein the first characteristic parameter is used to distinguish the reliability of the i-th recognition result and the j-th recognition result based on the character string length; Calculating a second feature parameter based on a detection frame position corresponding to the i-th recognition result and a detection frame position corresponding to the j-th recognition result, wherein the second feature parameter is used to distinguish the reliability of the i-th recognition result and the j-th recognition result based on the detection frame position; Calculating a third feature parameter based on a detection frame size corresponding to the i-th recognition result and a detection frame size corresponding to the j-th recognition result, wherein the third feature parameter is used to distinguish the reliability of the i-th recognition result and the j-th recognition result based on the detection frame size; Based on the confidence level corresponding to the i-th recognition result and the confidence level corresponding to the j-th recognition result, calculating a fourth feature parameter, wherein the fourth feature parameter is used to distinguish the reliability of the i-th recognition result and the j-th recognition result based on the confidence level; The second similarity parameter is calculated based on the first characteristic parameter, the second characteristic parameter, the third characteristic parameter and the fourth characteristic parameter.

7. The method according to claim 5, characterized in that The determining, based on the first similarity parameter and the second similarity parameter, the similarity between the i-th recognition result and the j-th recognition result comprises: Based on the product of the first similarity parameter and the second similarity parameter, a similarity between the i-th recognition result and the j-th recognition result is determined.

8. The method according to claim 1, characterized in that The performing character string recognition on the plurality of images to be recognized to obtain N recognition results includes: Performing character string recognition on the multiple images to be recognized to obtain recognition results corresponding to the multiple images to be recognized; The recognition results corresponding to the plurality of images to be recognized are screened to obtain the N recognition results.

9. The method according to claim 8, characterized in that The step of screening the recognition results corresponding to the plurality of images to be recognized to obtain the N recognition results includes: Selecting recognition results that meet the first condition from the recognition results corresponding to the plurality of images to be recognized, to obtain the N recognition results; Among them, the first condition includes at least one of the following: the detection box size corresponding to the recognition result is greater than or equal to a first threshold, the confidence level corresponding to the recognition result is greater than or equal to a second threshold, and the recognition result satisfies the known features of the character string to be recognized.

10. The method according to claim 1, characterized in that After character string recognition is performed on the plurality of images to be recognized and N recognition results are obtained, the method further includes: De-duplication processing is performed on the N recognition results to obtain K non-duplicate recognition results, where K is a positive integer less than or equal to N; Determine at least one seed recognition result group from the K non-repetitive recognition results, each seed recognition result group includes M seed recognition results, where M is a positive integer less than or equal to K; For any one of the at least one seed recognition result group, clustering the K non-repetitive recognition results based on the M seed recognition results in the seed recognition result group to obtain M recognition result sets; Based on the M recognition result sets, determining the character string recognition result corresponding to the seed recognition result group; The character string recognition result is determined based on the character string recognition results respectively corresponding to the at least one seed recognition result group.

11. The method according to claim 10, characterized in that The determining the string recognition result based on the string recognition results respectively corresponding to the at least one seed recognition result group includes: For any one of the at least one seed recognition result group, calculating a similarity parameter of a string recognition result corresponding to the seed recognition result group, wherein the similarity parameter is used to characterize a similarity between each recognition result included in a recognition result set corresponding to the string recognition result and the string recognition result; The character string recognition result is determined based on similar parameters of the character string recognition results respectively corresponding to the at least one seed recognition result group.

12. The method according to claim 10, characterized in that The step of clustering the K non-repetitive recognition results based on the M seed recognition results in the seed recognition result group to obtain M recognition result sets includes: For the p-th recognition result among the remaining KM recognition results except the M seed recognition results, respectively calculate the similarity between the p-th recognition result and the M seed recognition results, where p is a positive integer less than or equal to KM; The p-th recognition result is divided into the a-th recognition result set, the a-th recognition result set includes the a-th seed recognition result, the similarity between the p-th recognition result and the a-th seed recognition result is greater than the similarity between the p-th recognition result and other M-1 seed recognition results, and a is a positive integer less than or equal to M.

13. The method according to claim 12, characterized in that The determining, based on the M recognition result sets, the character string recognition result corresponding to the seed recognition result group includes: For the ath recognition result set, updating the seed recognition result corresponding to the ath recognition result set to obtain the ath updated seed recognition result; When the second condition is met, the M updated seed recognition results are determined as the character string recognition results corresponding to the seed recognition result group; wherein the second condition is that among the M updated seed recognition results, the number of seed recognition results that are the same as the M seed recognition results is greater than a second threshold; When the second condition is not met, the M updated seed recognition results are used as M seed recognition results, and the process starts again from the step of calculating the similarity between the p-th recognition result and the M seed recognition results for the p-th recognition result among the remaining KM recognition results except the M seed recognition results.

14. The method according to claim 10, characterized in that The M is the number of detection frames included in an image to be identified; or, M is the number of character strings whose length is greater than a third threshold; or, The M is determined based on the elbow rule; or, The M is determined based on the silhouette coefficient.

15. The method according to claim 1, characterized in that The step of obtaining a plurality of images to be identified comprises: When the start condition is met, photographing the character string to be recognized on the target object to obtain an image to be recognized; In response to the position offset corresponding to the target object being greater than or equal to a fourth threshold, photographing the character string to be recognized on the target object again to obtain an image to be recognized; wherein the position offset corresponding to the target object refers to the position offset of the target object, or refers to the position offset of a device that photographs the target object; When the end condition is met, stop shooting the image to be recognized.

16. A character string recognition device, characterized in that: The device comprises: An acquisition module is used to acquire multiple images to be recognized, each of which includes at least one character string to be recognized; A recognition module, used for performing character string recognition on the plurality of images to be recognized to obtain N recognition results, where N is an integer greater than 1; A calculation module, used to calculate the similarity between the N recognition results, and obtain similarity labels corresponding to the N recognition results respectively; wherein the similarity label corresponding to the i-th recognition result among the N recognition results is used to characterize the similarity between the i-th recognition result and the other N-1 recognition results among the N recognition results except the i-th recognition result, i being a positive integer less than or equal to N; The obtaining module is used to obtain a character string recognition result based on similar tags corresponding to the N recognition results.

17. A computer device, characterized in that: The computer device comprises a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to execute the computer program to implement the method according to any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the method according to any one of claims 1 to 15.

19. A computer program product, characterized in that The computer program product comprises a computer program, which is loaded and executed by a processor to implement the method according to any one of claims 1 to 15.

Images