Tile identification method and device, electronic equipment and readable storage medium

Abstract

The application relates to a card identification method and device, electronic equipment and a readable storage medium. The method comprises the following steps: acquiring an original picture sequence; grouping the original picture sequence to generate at least one picture subsequence; wherein the pictures of each picture subsequence contain a same suspected card; inputting the at least one picture subsequence into a preset data model for classification prediction; calculating a confidence degree according to the classification prediction result; and retaining the picture subsequence with the confidence degree greater than a preset retention threshold as a target card. The scheme provided by the application can group the pictures in the original picture sequence to generate at least one picture subsequence, use the preset data model to perform classification prediction on the at least one picture subsequence, calculate the confidence degree according to the classification prediction result, retain the corresponding picture subsequence according to the confidence degree greater than the preset retention threshold, accurately screen the suspected card, and improve the identification accuracy of the card.

Description

Technical Field

[0001] This application relates to the field of high-precision map technology, and in particular to sign recognition methods, devices, electronic devices, and readable storage media. Background Technology

[0002] With the rapid development of autonomous driving technology, the importance of high-definition maps is becoming increasingly prominent, and they have become an indispensable part of autonomous driving and intelligent transportation. The creation of high-definition maps requires the collection of relevant raw data. When collecting raw data, high-definition maps often utilize image recognition technology to identify road signs.

[0003] In related technologies, misidentification can occur during the recognition stage of high-precision map production. These misidentified elements are regular in shape and their overall shape is similar to that of a regular square road sign, which can easily cause filtering methods to fail, resulting in low recognition accuracy of the road sign. Summary of the Invention

[0004] To address or partially address the problems existing in related technologies, this application provides a method, apparatus, electronic device, and readable storage medium for identifying square tiles, which can accurately screen suspected square tiles and improve the accuracy of square tile recognition.

[0005] The first aspect of this application provides a method for recognizing square tiles, including:

[0006] Obtain the original image sequence;

[0007] The original image sequence is grouped to generate at least one image subsequence; wherein the images in each of the image subsequences contain the same suspected square sign.

[0008] At least one of the image subsequences is input into a preset data model for classification and prediction;

[0009] Calculate the confidence level based on the classification prediction results;

[0010] The image subsequences with a confidence level greater than a preset retention threshold are retained as target signs.

[0011] In some implementations, grouping the original image sequence to generate at least one image subsequence includes:

[0012] Corner point recognition processing is performed on all frames in the original image sequence to obtain the pixel coordinate data of the suspected square in all images of the original image sequence;

[0013] Based on the pixel coordinate data of the suspected signboard and grouping them according to the Hungarian matching algorithm, at least one image subsequence is generated.

[0014] In some implementations, the step of grouping based on the Hungarian matching algorithm to generate at least one image subsequence includes:

[0015] Obtain the pixel coordinate data of the suspected signboard in the first and second images of two adjacent frames;

[0016] Calculate the center pixel coordinates of all suspected squares in the first image and the second image respectively, and obtain the first center pixel coordinate data and the second center pixel coordinate data;

[0017] Based on the pixel coordinate data of the first center point and the pixel coordinate data of the second center point, calculate the distance between the center points of all suspected squares in the first image and the second image, and obtain the center point distance matrix.

[0018] Based on the center point distance matrix and using the Hungarian matching algorithm, the two suspected signs with the smallest center point distance in the first and second images are matched into the same group;

[0019] Two suspected signs that match the same group are included in the same image subsequence, generating at least one image subsequence.

[0020] In some embodiments, after matching the two suspected signs with the smallest center point distance in the first and second images into the same group, the method further includes:

[0021] If the distance between the center points of two suspected signs in the same group is less than a preset threshold, the group is retained.

[0022] In some implementations, including two suspected signs that match the same group in the same image subsequence includes:

[0023] Two suspected plaques that match the same group are included in the same image subsequence;

[0024] Suspected signs that did not match a group were added to a newly created image subsequence.

[0025] In some implementations, inputting at least one of the image subsequences into a preset data model for classification prediction includes:

[0026] All suspected square signs in at least one of the image subsequences are cropped into a preset shape and then input into a preset data model for classification and prediction.

[0027] In some implementations, the preset data model is obtained in the following ways:

[0028] The labeled first-type and second-type datasets are input into a preset binary classification neural network for training to obtain a preset data model.

[0029] A second aspect of this application provides a sign recognition device, comprising:

[0030] The input module is used to obtain the original image sequence;

[0031] A grouping module is used to group the original image sequence obtained by the input module to generate at least one image subsequence; wherein the images in each image subsequence contain the same suspected square sign.

[0032] The classification prediction module is used to input at least one image subsequence generated by the grouping module into a preset data model for classification prediction.

[0033] The confidence calculation module is used to calculate the confidence level based on the classification prediction result of the classification prediction module.

[0034] The output module is used to retain the image subsequences whose confidence scores calculated by the calculation module are greater than a preset retention threshold as target signs.

[0035] A third aspect of this application provides an electronic device, comprising:

[0036] Processor; and

[0037] A memory that stores executable code, which, when executed by the processor, causes the processor to perform the method described above.

[0038] A fourth aspect of this application provides a computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method described above.

[0039] The technical solution provided in this application may include the following beneficial effects:

[0040] The method for recognizing square signs in this application involves grouping the images in the original image sequence to generate at least one image subsequence, using a preset data model to classify and predict the at least one image subsequence, calculating the confidence level based on the classification and prediction results, and then retaining the corresponding image subsequence based on the confidence level being greater than a preset retention threshold. This achieves accurate screening of suspected square signs and improves the recognition accuracy of square signs.

[0041] Furthermore, by grouping the original image sequence using a Hungarian matching algorithm, high-precision matching can be achieved for images containing the same suspected sign, improving the accuracy of grouping. By using a preset data model trained by a binary classification neural network to predict images in the image subsequence, non-sign image data can be effectively excluded, further improving the recognition accuracy of road signs.

[0042] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0043] The above and other objects, features and advantages of this application will become more apparent from the following description of exemplary embodiments of this application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components.

[0044] Figure 1 This is a schematic flowchart illustrating the square sign recognition method according to an embodiment of this application;

[0045] Figure 2 This is another schematic flowchart illustrating the square sign recognition method shown in the embodiments of this application;

[0046] Figure 3 This is an application block diagram illustrating the square sign recognition method shown in the embodiments of this application;

[0047] Figure 4 This is a schematic diagram illustrating the process of grouping images from the original image sequence and generating image sub-sequences in the square sign recognition method shown in the embodiments of this application;

[0048] Figure 5 This is a schematic diagram illustrating the tracking and matching process of the Hungarian matching algorithm as shown in the embodiments of this application;

[0049] Figure 6 This is a schematic diagram of the structure of the sign recognition device shown in the embodiments of this application;

[0050] Figure 7 This is another structural schematic diagram of the sign recognition device shown in the embodiments of this application;

[0051] Figure 8 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application. Detailed Implementation

[0052] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0053] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0054] It should be understood that although the terms "first," "second," "third," etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0055] In related technologies, misidentification can occur during the recognition stage of high-precision map production. These misidentified elements are regular in shape and their overall shape is similar to that of a regular square road sign, which can easily cause filtering methods to fail, resulting in low recognition accuracy of the road sign.

[0056] To address the aforementioned issues, this application provides a sign recognition method that can accurately screen suspected signs and improve the recognition accuracy of road signs.

[0057] The technical solutions of the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0058] Figure 1 This is a flowchart illustrating the square sign recognition method shown in the embodiments of this application.

[0059] See Figure 1 The method for recognizing a square sign in this application includes:

[0060] S110, Obtain the original image sequence.

[0061] In this step, the original image sequence can be a collection of images generated from image data collected by a camera device. It is understood that the original image sequence includes multiple images. The camera device can be, but is not limited to, a dashcam, camera, or other device with video recording capabilities installed on a vehicle. The camera device can be positioned at the front or top of the vehicle, with its camera angle facing the direction of travel to capture video of the vehicle's path. The image data collected by the camera device can be used to generate multiple images by frame extraction. For example, if the frame rate of the image data is 10 frames per second, 2, 5, or 10 frames can be extracted per second to obtain multiple captured images. Of course, to further improve accuracy, each frame of the image data from the camera device can be used as part of the original image sequence in the sign recognition process.

[0062] It should be noted that in this application, as long as the original image sequence is input or obtained, the card can be identified.

[0063] S120, group the original image sequence to generate at least one image subsequence.

[0064] In this step, images within the original image sequence are grouped and matched, assigning images containing the same suspected sign to the same image subsequence. It should be understood that any image within the original image sequence can contain one or more suspected signs, meaning multiple suspected signs can be identified within the same image.

[0065] In some implementations, a traversal approach is used to group and match images within the original image sequence. It can be understood that this application can group and judge images according to the order in which they are generated. For example, given the first, second, and third images generated sequentially, this application first groups and judges the suspected signs identified in the first and second images. If the same suspected sign is identified in each of the first and second images, it is classified into the same image subsequence. Then, the suspected signs identified in the second and third images are grouped and judged again. If the same suspected sign is identified in each of the second and third images, it is classified into the same image subsequence. Therefore, it can be seen that consecutively generated images can simultaneously contain the same suspected sign and be classified into the same image subsequence.

[0066] S130, input at least one image subsequence into a preset data model for classification and prediction.

[0067] In this step, any image subsequence may include at least two images from which the same suspected square tile is identified. At least one image subsequence is input into a preset data model to predict whether the suspected square tile within the image subsequence is a square tile or not, thus classifying the suspected square tiles contained within the image subsequence. The preset data model can use a labeled classification method to predict suspected square tiles; for example, a predicted square tile is labeled "1," and a predicted non-square tile is labeled "0." The preset data model can be a binary classification neural network model.

[0068] S140, Calculate the confidence level based on the classification prediction results.

[0069] In this step, based on the classification prediction results of the image subsequence using the preset data model in step S130, the confidence level of the suspected signboard in the corresponding image subsequence is calculated.

[0070] S150, retain the image subsequences with a confidence level greater than the preset retention threshold as the target signboard.

[0071] This application sets a preset retention threshold. If the confidence level of an image subsequence exceeds the preset retention threshold, and it is determined that the suspected signboard within that subsequence is the target signboard to be retained, then the corresponding image subsequence is retained as the target signboard. Image subsequences with a confidence level below the preset retention threshold can be deleted to reduce redundant data.

[0072] In this embodiment, the sign recognition method provided by this application generates at least one image subsequence by grouping the images in the original image sequence, performs one-to-one classification prediction on the at least one image subsequence using a preset data model, calculates the confidence level based on the classification prediction result, and then retains the corresponding image subsequence based on the confidence level being greater than a preset retention threshold, thereby achieving accurate screening of suspected signs and improving the recognition accuracy of signs.

[0073] Figure 2 This is another schematic flowchart illustrating the square sign recognition method shown in the embodiments of this application; Figure 3 This is an application block diagram of the square sign recognition method shown in the embodiments of this application.

[0074] See Figure 2 and Figure 3 The method for recognizing a square sign in this application includes:

[0075] S210, Obtain the original image sequence.

[0076] The original image sequence consists of multiple images. These images are typically consecutive frames spaced at equal intervals.

[0077] S220: Perform corner detection processing on all images in the original image sequence to obtain the pixel coordinate data of the suspected square signs in all images of the original image sequence.

[0078] In this step, corner detection refers to a method of obtaining image features by performing corner detection on the image. Corner detection obtains the pixel coordinates of suspected signs in all images, especially the pixel coordinates of the four corners of the suspected signs. Since the suspected signs are square or nearly square, the center point coordinates of the suspected sign can be determined by the pixel coordinates of the four corners.

[0079] S230, based on the pixel coordinate data of the suspected signboard and grouped according to the Hungarian matching algorithm, generate at least one image subsequence.

[0080] In this step, based on the pixel coordinate data of the suspected sign, the position and center distance of the suspected sign between two adjacent images can be obtained. The suspected signs between two adjacent images are tracked and matched using the Hungarian matching algorithm, and grouped. The same suspected sign between two adjacent images is classified into the same image subsequence, generating at least one image subsequence for subsequent classification prediction.

[0081] Figure 4 This is a schematic diagram illustrating the process of grouping images from the original image sequence and generating image sub-sequences in the sign recognition method shown in the embodiments of this application.

[0082] See Figure 4 This application uses a traversal approach to group and match all images within the original image sequence. Grouping the images in the original image sequence and generating image subsequences may include the following steps:

[0083] S231, Obtain the pixel coordinate data of the suspected signboard in the first and second images of two adjacent frames.

[0084] Two adjacent frames refer to two consecutive images, the first image and the second image. In this step, the pixel coordinate data of the suspected signboard in the first image and the pixel coordinate data of the suspected signboard in the second image are obtained respectively.

[0085] S232, calculate the center pixel coordinates of all suspected squares in the first image and the second image respectively, and obtain the first center pixel coordinate data and the second center pixel coordinate data.

[0086] In this step, the center pixel coordinates of all suspected signs in the first image and the center pixel coordinates of all suspected signs in the second image are calculated respectively. The above calculations can be performed separately or simultaneously. The first center pixel coordinate data corresponding to all suspected signs in the first image and the second center pixel coordinate data corresponding to all suspected signs in the second image are obtained.

[0087] S233, Based on the pixel coordinate data of the first center point and the pixel coordinate data of the second center point, calculate the distance between the center points of all suspected squares in the first image and the second image, and obtain the center point distance matrix.

[0088] In this step, based on the pixel coordinate data of the first center point and the pixel coordinate data of the second center point obtained in step S231, the distance between the center points of the suspected squares between the first image and the second image is calculated.

[0089] For example, the first image contains two suspected plaques numbered 1 and 2, and the second image contains two suspected plaques numbered 3 and 4.

[0090] The calculation process can be as follows: 1. Calculate the center distance between the suspected square card number 1 and the two suspected square cards numbered 3 and 4; 2. Calculate the center distance between the suspected square card number 2 and the two suspected square cards numbered 3 and 4.

[0091] The formula used in the calculation process can be:

[0092]

[0093] Where (x1, y1) and (x2, y2) represent the pixel coordinates of the center point of the suspected signboard in the previous frame and the next frame (i.e., the first image and the second image), respectively, and d is the distance between the center points of the two frames.

[0094] Through the above calculation process, a distance matrix consisting of the center point distances between all suspected squares in the first and second images can be obtained.

[0095] S234. Based on the center point distance matrix and using the Hungarian matching algorithm, the two suspected signs with the smallest center point distance in the first and second images are matched into the same group.

[0096] Figure 5 This is a schematic diagram illustrating the tracking and matching process of the Hungarian matching algorithm as shown in an embodiment of this application.

[0097] See Figure 5As shown, the Hungarian matching algorithm (Edmonds method) is a combinatorial optimization algorithm that solves the task assignment problem in polynomial time. It is used to perform a one-to-one matching of all suspected squares between the previous frame and the next frame.

[0098] The Hungarian matching algorithm can be used to match the two suspected cards with the smallest center point distance together, and then the two successfully matched suspected cards are grouped together.

[0099] For example, the suspected square sign number 1 in the first image is the most similar to the suspected square sign number 2 in the second image because the center point distance between them is the smallest. Similarly, the suspected square sign number 2 in the first image is the most similar to the suspected square sign number 1 in the second image because the center point distance between them is the smallest. After matching using the Hungarian matching algorithm, the final groupings (1, 2) and (2, 1) are obtained.

[0100] S235, add two suspected signs that match the same group to the same image subsequence, and generate at least one image subsequence.

[0101] In this step, based on the grouping results obtained in step S234, two suspected signs that match the same group are included in the same image subsequence, generating at least one image subsequence.

[0102] A subsequence of images may include at least three suspected plaques. Because a traversal approach is used to group and match all images within the original image sequence, during the grouping and matching of at least three consecutive images, at least three images may each contain the same suspected plaque.

[0103] In some embodiments, this application may also set a preset threshold. Before including two suspected signs that match the same group in the same image subsequence, it may also determine whether the distance between the center points of the two suspected signs in the same group is less than the preset threshold.

[0104] For example, the preset threshold can be 150. The distance between the center points of two suspected signs belonging to the same group is recalculated. If the distance is greater than 150, the two suspected signs are not the same; if the distance is less than 150, they are the same. If the distance between the center points of two suspected signs in the same group is less than the preset threshold, the group is retained. In this way, the correctness of the grouping is verified through the preset threshold, achieving a secondary detection of suspected signs matching the same group, further improving the accuracy of group matching. It should be understood that the above application process of the preset threshold can also be applied to groups with three or more suspected signs.

[0105] It should be noted that during the grouping and matching process of all images within the original image sequence, if a subsequent image successfully matches a suspected sign from a previous image, it can be grouped into the same group and added to the already generated image subsequence. If a subsequent image does not have a suspected sign that matches a sign from a previous image, it can be considered a newly added suspected sign, and the corresponding newly added suspected sign will be added to a new image subsequence. In other words, an image subsequence can contain only one suspected sign.

[0106] S240, after cropping all suspected squares in at least one image subsequence into a preset shape, input them into a preset data model for classification and prediction.

[0107] The preset shape can be the shape of the road sign targeted by the recognition method of this application. For example, if the recognition method of this application is used to detect square signs, then the preset shape is square.

[0108] In this step, all suspected square signs in at least one of the image subsequences obtained in step S230 are cropped to a preset shape, such as a square. The cropped suspected square signs are then input into a preset data model for classification prediction. The classification prediction result for each suspected square sign is whether it is classified as a square sign or not.

[0109] In this application, the pre-defined data model can be trained through the following steps:

[0110] 1) Collect and label datasets

[0111] Collect multiple sets of first-type datasets and second-type datasets, where the first-type datasets can correspond to the square tile datasets and the second-type datasets can correspond to the non-square tile datasets; and label the first-type datasets and second-type datasets, for example, labeling square tiles as "1" and non-square tiles as "2".

[0112] 2) Preset binary classification neural network

[0113] Based on the prediction requirements, design the corresponding binary classification neural network.

[0114] 3) Train the pre-defined binary classification neural network

[0115] The labeled first-type and second-type datasets are input into a preset binary classification neural network for training. After training, a preset data model is obtained.

[0116] The preset data model is trained through the above steps. When an input image that is suspected to be a square sign is given, the prediction result is "1" and the prediction result is "2" if the image is not a square sign.

[0117] S250: Calculate the confidence level based on the classification prediction results.

[0118] In this step, based on the classification prediction results of the image subsequence using the preset data model in step S240, the confidence level of the suspected signboard in the corresponding image subsequence is calculated.

[0119] As an example, the confidence level can be set as: Confidence Level = Number of images predicted as square tiles / Total number of images in the image subsequence. It can be understood that with the above confidence level setting, if all images in an image subsequence that are suspected to be square tiles are predicted as square tiles, the confidence level is 100; if all images in an image subsequence that are suspected to be square tiles are predicted as non-square tiles, the confidence level is 0.

[0120] S260, retain the image subsequences with a confidence level greater than the preset retention threshold as the target signboard.

[0121] In this step, based on the confidence level of an image subsequence being greater than a preset retention threshold, the suspected signboard in that subsequence is determined to be the target signboard to be retained, and the corresponding image subsequence is then retained as the target signboard. For example, if the preset retention threshold is set to 0.5, a confidence level greater than 0.5 indicates a high confidence level for the image subsequence, and the image in that subsequence is considered a signboard; conversely, a confidence level less than 0.5 indicates a low confidence level for the image subsequence, and the image in that subsequence is considered a non-signboard.

[0122] In this embodiment, the sign recognition method provided by this application uses a Hungarian matching algorithm to group the original image sequence, which can achieve high-precision matching of images containing the same suspected sign, thereby improving the accuracy of grouping; and uses a preset data model trained by a binary classification neural network to predict the images in the image subsequence, effectively excluding non-sign image data, and further improving the recognition accuracy of road signs.

[0123] Corresponding to the aforementioned application function implementation method embodiments, this application also provides a sign recognition device, an electronic device, and corresponding embodiments.

[0124] Figure 6 This is a schematic diagram of the structure of the sign recognition device shown in the embodiments of this application.

[0125] See Figure 6 The sign recognition device of this application includes: an input module 310, a grouping module 320, a classification prediction module 330, a confidence calculation module 340, and an output module 350.

[0126] Input module 310 is used to acquire the original image sequence. The original image sequence is a collection of images that can be generated from image data acquired by a camera device. It can be understood that the original image sequence includes multiple images.

[0127] The grouping module 320 is used to group the original image sequence obtained by the input module 310 to generate at least one image subsequence. Each image subsequence contains images of the same suspected plaque.

[0128] The classification prediction module 330 is used to input at least one image subsequence generated by the grouping module 320 into a preset data model for classification prediction. The preset data model can use a labeled classification method to predict suspected square signs; for example, a predicted square sign is labeled "1", and a predicted non-square sign is labeled "0". The preset data model can be a binary classification neural network model.

[0129] The confidence calculation module 340 is used to calculate the confidence level based on the classification prediction result of the classification prediction module 330. The confidence level can be set as: Confidence Level = Number of images predicted as square tiles / Total number of images in the image subsequence. It can be understood that, using the above confidence level setting, when all images in an image subsequence that are suspected to be square tiles are predicted as square tiles, the confidence level is 100; when all images in an image subsequence that are suspected to be square tiles are predicted as non-square tiles, the confidence level is 0.

[0130] Output module 350 is used to retain image subsequences whose confidence scores calculated by confidence calculation module 340 are greater than a preset retention threshold as target signs.

[0131] In this embodiment, the sign recognition device provided in this application generates at least one image subsequence by grouping the images in the original image sequence, performs one-to-one classification prediction on the at least one image subsequence using a preset data model, calculates the confidence level based on the classification prediction result, and then retains the corresponding image subsequence based on the confidence level being greater than a preset retention threshold, thereby achieving accurate screening of suspected signs and improving the recognition accuracy of signs.

[0132] Figure 7 This is another structural schematic diagram of the sign recognition device shown in the embodiments of this application.

[0133] See Figure 7 The sign recognition device of this application includes: an input module 310, a grouping module 320, a classification prediction module 330, a confidence calculation module 340, and an output module 350.

[0134] The input module 310, grouping module 320, classification prediction module 330, confidence calculation module 340, and output module 350 can be found in [reference needed]. Figure 4The description in the text.

[0135] The grouping module 320 may include: a data acquisition submodule 321, a center point coordinate calculation submodule 322, a center point distance matrix submodule 323, a grouping submodule 324, and an image sequence generation submodule 325.

[0136] The data acquisition submodule 321 is used to acquire the pixel coordinate data of the suspected signboard in the first and second images of two adjacent frames.

[0137] The center point coordinate calculation submodule 322 is used to calculate the center point pixel coordinates of all suspected square signs in the first and second images obtained by the data acquisition submodule 321, and to obtain the first center point pixel coordinate data and the second center point pixel coordinate data.

[0138] The center point distance matrix submodule 323 and the center point coordinate calculation submodule 322 calculate the distance between the center points of all suspected squares in the first image and the second image based on the first center point pixel coordinate data and the second center point pixel coordinate data, and obtain the center point distance matrix.

[0139] Grouping submodule 324 uses the center point distance matrix calculated by center point distance matrix submodule 323 and the Hungarian matching algorithm to match the two suspected signs with the smallest center point distance in the first and second images into the same group.

[0140] Image sequence generation submodule 325 includes two suspected signs that are matched into the same group by grouping submodule 324 into the same image subsequence, generating at least one image subsequence.

[0141] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated further here.

[0142] Figure 8 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application.

[0143] See Figure 8 The electronic device 1000 includes a memory 1010 and a processor 1020.

[0144] The processor 1020 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0145] Memory 1010 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by processor 1020 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1010 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital multifunction optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, a high-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0146] The memory 1010 stores executable code, which, when processed by the processor 1020, can cause the processor 1020 to execute part or all of the methods described above.

[0147] Furthermore, the method according to this application can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the method described above.

[0148] Alternatively, this application may be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium) storing executable code (or computer program or computer instruction code) thereon, which, when executed by a processor of an electronic device (or server, etc.), causes the processor to perform part or all of the steps of the methods described above according to this application.

[0149] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for recognizing square signs, characterized in that: Obtain the original image sequence; The original image sequence is grouped to generate at least one image subsequence; wherein the images in each of the image subsequences contain the same suspected square sign. The process includes: performing corner detection processing on all frames in the original image sequence to obtain pixel coordinate data of suspected signs in all images of the original image sequence; obtaining pixel coordinate data of the suspected signs in the first and second images of two adjacent frames; calculating the center point pixel coordinates of all suspected signs in the first and second images respectively to obtain first center point pixel coordinate data and second center point pixel coordinate data; calculating the distance between the center points of all suspected signs in the first and second images based on the first and second center point pixel coordinate data to obtain a center point distance matrix; matching the two suspected signs with the smallest center point distance in the first and second images into the same group based on the center point distance matrix and using the Hungarian matching algorithm; and including the two suspected signs that are matched into the same group in the same image subsequence to generate at least one image subsequence. At least one of the image subsequences is input into a preset data model for classification and prediction; Calculate the confidence level based on the classification prediction results; The image subsequences with a confidence level greater than a preset retention threshold are retained as target signs.

2. The square sign recognition method according to claim 1, characterized in that, After matching the two suspected signs with the smallest center point distance in the first and second images into the same group, the method further includes: If the distance between the center points of two suspected signs in the same group is less than a preset threshold, the group is retained.

3. The square sign recognition method according to claim 2, characterized in that, The step of including two suspected plaques that match the same group in the same image subsequence includes: Two suspected plaques that match the same group are included in the same image subsequence; Suspected signs that did not match a group were added to a newly created image subsequence.

4. The square sign recognition method according to claim 1, characterized in that, The step of inputting at least one of the image sub-sequences into a preset data model for classification and prediction includes: All suspected square signs in at least one of the image subsequences are cropped into a preset shape and then input into a preset data model for classification and prediction.

5. The square sign recognition method according to claim 1, characterized in that, The preset data model is obtained through the following methods: The labeled first-type and second-type datasets are input into a preset binary classification neural network for training to obtain a preset data model.

6. A sign recognition device, characterized in that, include: The input module is used to obtain the original image sequence; A grouping module is used to group the original image sequence obtained by the input module to generate at least one image subsequence; wherein the images in each image subsequence contain the same suspected square sign. The process includes: performing corner detection processing on all frames in the original image sequence to obtain pixel coordinate data of suspected signs in all images of the original image sequence; obtaining pixel coordinate data of the suspected signs in the first and second images of two adjacent frames; calculating the center point pixel coordinates of all suspected signs in the first and second images respectively to obtain first center point pixel coordinate data and second center point pixel coordinate data; calculating the distance between the center points of all suspected signs in the first and second images based on the first and second center point pixel coordinate data to obtain a center point distance matrix; matching the two suspected signs with the smallest center point distance in the first and second images into the same group based on the center point distance matrix and using the Hungarian matching algorithm; and including the two suspected signs that are matched into the same group in the same image subsequence to generate at least one image subsequence. The classification prediction module is used to input at least one image subsequence generated by the grouping module into a preset data model for classification prediction. The confidence calculation module is used to calculate the confidence level based on the classification prediction result of the classification prediction module. The output module is used to retain the image subsequences whose confidence scores calculated by the calculation module are greater than a preset retention threshold as target signs.

7. An electronic device, characterized in that, include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 1-5.

8. A computer-readable storage medium having executable code stored thereon, characterized in that: When the executable code is executed by the processor of the electronic device, the processor performs the method as described in any one of claims 1-5.

Images