Image recognition method, training method, device, system and storage medium

Abstract

The disclosure provides an image recognition method and device, which are suitable for the fields of artificial intelligence technology and Internet of Things technology. The image recognition method comprises the following steps: recording a vehicle identifier of a vehicle entering a work site from an entrance of the work site, and obtaining a vehicle identifier set comprising at least one vehicle identifier; collecting image information to be processed of a vehicle working around a work platform, wherein the work platform is located in the work site; and identifying a target vehicle identifier of a target vehicle in the image information to be processed according to a predicted matching result of the image information to be processed and the vehicle identifiers in the vehicle identifier set. The disclosure further provides a training method and device of an image recognition model, a computer system, a computer readable storage medium and a computer program product.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence technology and the field of Internet of Things, and more particularly, to an image recognition method, a training method, an apparatus, a computer system, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] Image information such as videos and pictures is acquired by a camera device, and image recognition technology is used to process the image information to quickly recognize text information in the image information, which is widely used in many application scenarios such as transportation and warehousing logistics. For example, by capturing image information of vehicles driving on a highway, a vehicle that violates traffic rules can be determined by recognizing the license plate of the vehicle in the image information.

[0003] In the process of implementing the present disclosure, the inventors have found that at least the following technical problems exist in the related art: Due to defects such as image blur or image loss in image information, errors may occur in the text information recognized from the image information. SUMMARY

[0004] Therefore, the present disclosure provides an image recognition method, a training method, an apparatus, a computer system, a computer readable storage medium, and a computer program product.

[0005] One aspect of the present disclosure provides an image recognition method, comprising:

[0006] Recording vehicle identification of a vehicle entering a work site from an entrance of the work site to obtain a vehicle identification set comprising at least one vehicle identification;

[0007] Collecting image information to be processed of a vehicle working around a work platform, wherein the work platform is located in the work site;

[0008] According to a predicted matching result of the image information to be processed and vehicle identification in the vehicle identification set, identifying target vehicle identification of a target vehicle in the image information to be processed.

[0009] According to an embodiment of the present disclosure, according to a predicted matching result of the image information to be processed and vehicle identification in the vehicle identification set, identifying target vehicle identification of a target vehicle in the image information to be processed comprises:

[0010] Performing image feature extraction on the image information to be processed to obtain an image feature vector, wherein the image information to be processed records target information;

[0011] Performing text feature extraction on vehicle identification in the vehicle identification set to obtain a text feature vector;

[0012] fusing the image feature vector and the text feature vector based on the attention mechanism to obtain an image weighted feature vector for the image information to be processed;

[0013] determining a predicted matching result of the image information to be processed based on the image weighted feature vector and the text feature vector; and

[0014] in a case where the predicted matching result represents that the image information to be processed matches the vehicle identifier, identifying target information in the image information to be processed as the target vehicle identifier.

[0015] According to an embodiment of the present disclosure, the image feature extraction on the image information to be processed to obtain an image feature vector includes:

[0016] inputting the image information to be processed into a first image feature extraction layer to extract image features of the image information to be processed in a first direction and output a first image feature vector; and

[0017] inputting the image information to be processed into a second image feature extraction layer to extract image features of the image information to be processed in a second direction and output a second image feature vector;

[0018] wherein the image feature vector includes the first image feature vector and the second image feature vector, and the first direction is perpendicular to the second direction.

[0019] According to an embodiment of the present disclosure, the fusing the image feature vector and the text feature vector based on the attention mechanism to obtain an image weighted feature vector for the image information to be processed includes:

[0020] inputting the first image feature vector and the text feature vector into a first attention layer to output a first image weighted feature vector; and

[0021] inputting the second image feature vector and the text feature vector into a second attention layer to output a second image weighted feature vector;

[0022] wherein the image weighted feature vector includes the first image weighted feature vector and the second image weighted feature vector.

[0023] According to an embodiment of the present disclosure, the determining a predicted matching result of the image information to be processed based on the image weighted feature vector and the text feature vector includes:

[0024] fusing and encoding the image weighted feature vector and the text feature vector to generate a predicted input vector; and

[0025] The prediction input vector is input into a prediction classification layer, and a prediction matching result of the image information to be processed is output.

[0026] According to an embodiment of the present disclosure, the text feature extraction on the vehicle identifiers in the vehicle identifier set comprises:

[0027] The vehicle identifiers in the vehicle identifier set are input into a text feature extraction layer, and the text feature vector is output.

[0028] According to an embodiment of the present disclosure, the text feature extraction on the vehicle identifiers in the vehicle identifier set further comprises:

[0029] Before the vehicle identifiers are input into the text feature extraction layer, the vehicle identifiers are completed so that the number of characters in the vehicle identifiers meets a preset rule.

[0030] According to an embodiment of the present disclosure, the recording of the vehicle identifiers of the vehicles entering the work site from the work site entrance to obtain the vehicle identifier set comprising at least one vehicle identifier comprises:

[0031] Vehicle image information of the vehicles entering the work site from the work site entrance is collected;

[0032] According to the vehicle image information, the vehicle identifiers of the vehicles entering the work site from the work site entrance are determined;

[0033] The vehicle identifier set is determined according to the vehicle identifiers of the vehicles entering the work site from the work site entrance.

[0034] Another aspect of the present disclosure provides a training method of an image recognition model, comprising:

[0035] Obtaining a training sample, wherein the training sample comprises sample image information to be processed, a sample vehicle identifier, and a label corresponding to the sample image information to be processed; and

[0036] Training an initial image recognition model using the training sample to obtain a trained image recognition model;

[0037] The image recognition model is used to implement the image recognition method.

[0038] Another aspect of the present disclosure provides an image recognition device, comprising:

[0039] A recording module is configured to record vehicle identifiers of vehicles entering a work site from a work site entrance to obtain a vehicle identifier set comprising at least one vehicle identifier;

[0040] The collection module is configured to collect image information of a vehicle to be processed working around a work platform, wherein the work platform is located in the work site.

[0041] The identification module is configured to identify a target vehicle identifier of a target vehicle in the image information to be processed according to a predicted matching result of the image information to be processed and the vehicle identifiers in the vehicle identifier set.

[0042] Another aspect of the present disclosure provides a training device of an image recognition model, comprising:

[0043] The sample acquisition module is configured to acquire a training sample, wherein the training sample comprises sample image information to be processed, a sample vehicle identifier, and a label corresponding to the sample image information to be processed.

[0044] The training module is configured to train an initial image recognition model by using the training sample to obtain a trained image recognition model.

[0045] The image recognition model is configured to implement the image recognition method.

[0046] Another aspect of the present disclosure provides a computer system, comprising: one or more processors; a memory configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the identification method of the embodiments of the present disclosure, or implement the training method of the embodiments of the present disclosure.

[0047] Another aspect of the present disclosure provides a computer-readable storage medium having stored executable instructions, which, when executed by a processor, cause the processor to implement the identification method of the embodiments of the present disclosure, or implement the training method of the embodiments of the present disclosure.

[0048] Another aspect of the present disclosure provides a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the identification method of the embodiments of the present disclosure, or implements the training method of the embodiments of the present disclosure.

[0049] According to the embodiment of the present disclosure, because the vehicle identifier of the vehicle entering the work site from the work site entrance is obtained as a vehicle identifier set of the vehicle entering the work site, the target vehicle identifier of the target vehicle in the to-be-processed image information is identified by collecting the to-be-processed image information of the vehicle working around the work platform and according to the predicted matching result of the to-be-processed image information and the vehicle identifier in the vehicle identifier set of the vehicle entering the work site, which can at least partially avoid the technical problems in the related art that the identification accuracy of the target vehicle identifier is low or even the vehicle identifier cannot be identified due to unclear to-be-processed image information or the vehicle identifier in the to-be-processed image information being contaminated, avoid the identification error of the vehicle identifier, improve the identification accuracy of the vehicle identifier, and improve the intelligent management level of the vehicle working in the work site. BRIEF DESCRIPTION OF DRAWINGS

[0050] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure, taken in conjunction with the accompanying drawings, in which:

[0051] Figure 1 An exemplary system architecture to which the image recognition method and the image recognition device of the present disclosure can be applied is schematically shown;

[0052] Figure 2 A flowchart of the image recognition method according to the embodiment of the present disclosure is schematically shown;

[0053] Figure 3A A flowchart of identifying the target vehicle identifier of the target vehicle in the to-be-processed image information according to the predicted matching result of the to-be-processed image information and the vehicle identifier in the vehicle identifier set according to the embodiment of the present disclosure is schematically shown;

[0054] Figure 3B An application scenario diagram of image feature extraction on the to-be-processed image information according to the embodiment of the present disclosure is schematically shown;

[0055] Figure 3C Another application scenario diagram of image feature extraction on the to-be-processed image information according to the embodiment of the present disclosure is schematically shown;

[0056] Figure 4 An application scenario diagram of the image recognition method according to the embodiment of the present disclosure is schematically shown;

[0057] Figure 5 A flowchart of the training method of the image recognition model according to the embodiment of the present disclosure is schematically shown;

[0058] Figure 6 A block diagram of the image recognition device according to the embodiment of the present disclosure is schematically shown;

[0059] Figure 7A block diagram of a training apparatus of an image recognition model is shown schematically.

[0060] Figure 8 A block diagram of an electronic device suitable for implementing the method provided by the embodiments of the present disclosure is shown schematically. DETAILED DESCRIPTION

[0061] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood, however, that the description which follows is merely exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description of the embodiments of the present disclosure, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to one skilled in the art that the embodiments of the present disclosure can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the present disclosure.

[0062] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the present disclosure. As used herein, the term "including" and "comprising" and the like are meant to be inclusive in nature, that is, specifying features, steps, operations, and / or components but do not preclude the addition of one or more other features, steps, operations, and / or components.

[0063] All terms used herein, including technical and scientific terms, have the meanings commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings that are consistent with the context of the specification, and should not be interpreted in an idealized or overly formal manner.

[0064] In the case of using expressions similar to "at least one of A, B, and C, etc.", it should generally be interpreted to include at least one of A, B, or C, in other words, to include systems that have at least one of A, B, or C, as well as systems that have two or more of A, B, and C.

[0065] Embodiments of the present disclosure provide an image recognition method, a training method, an apparatus, a computer system, a computer-readable storage medium, and a computer program product. The image recognition method includes recording vehicle identification of a vehicle entering a work site from a work site entrance to obtain a vehicle identification set including at least one vehicle identification; collecting image information of a vehicle working around a work platform, wherein the work platform is located in the work site; and identifying target vehicle identification of a target vehicle in the image information to be processed according to a predicted matching result of the image information to be processed and the vehicle identification in the vehicle identification set.

[0066] According to the embodiment of the present disclosure, because the vehicle identifier of the vehicle entering the work site from the work site entrance is obtained to obtain a vehicle identifier set of the vehicle entering the work site, the target vehicle identifier of the target vehicle in the to-be-processed image information is identified by collecting the to-be-processed image information of the vehicle working around the work platform and according to the prediction matching result of the vehicle identifier in the vehicle identifier set of the vehicle entering the work site, the technical problem that the target vehicle identifier recognition accuracy is low or even the vehicle identifier cannot be recognized due to unclear to-be-processed image information or the vehicle identifier in the to-be-processed image information being polluted in the related art can be at least partially avoided, the vehicle identifier recognition error can be avoided, the target vehicle identifier recognition accuracy can be improved, and the intelligent management level of the vehicle entering the work site to work can be improved.

[0067] In the technical solutions of the present disclosure, the acquisition, storage and application of user personal information comply with relevant laws and regulations, necessary security measures are taken, and the public order and good customs are not violated.

[0068] In the technical solutions of the present disclosure, the authorization or consent of the user is obtained before the user personal information is acquired or collected.

[0069] Figure 1 An exemplary system architecture to which the image recognition method and the image recognition device of the present disclosure can be applied is schematically shown. It should be noted that, Figure 1 The shown is only an example of the system architecture to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0070] As Figure 1 shown, the system architecture 100 according to the embodiment can include terminal devices 101, 102, 103, a network 104, a server 105, an image collection device 106 and a vehicle 107. The network 104 is a medium for providing a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired and / or wireless communication links, etc.

[0071] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software, etc. (only as examples).

[0072] The terminal devices 101, 102, and 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0073] The server 105 can be a server providing various services, such as a background management server supporting websites browsed by users using the terminal devices 101, 102, and 103 (only as an example). The background management server can perform analysis and the like on received user requests and the like, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests, and the like) to the terminal devices.

[0074] The image acquisition device 106 can be a monitoring camera or the like, and the vehicle 107 can be a vehicle working around a work platform. The image acquisition device 106 can perform information transmission with the server 105 through the network 104, and correspondingly, the image acquisition device 106 can also perform information transmission with the terminal devices 101, 102, and 103 through the network 104.

[0075] It should be noted that the image recognition method provided by the embodiments of the present disclosure can generally be executed by the server 105. For example, the image acquisition device 106 (such as a monitoring camera) arranged in a work site (such as a logistics industrial park) can acquire image information of the vehicle 107 working around a work platform, and the image information containing the vehicle 106 can be regarded as to-be-processed image information. The to-be-processed image information records the license plate number (i.e., vehicle identification) of the vehicle 107. When the vehicle enters the work site, the vehicle identification of the vehicle entering the work site from the entrance of the work site can be recorded by relevant personnel or equipment, and the vehicle identification set can include the license plate number information of all vehicles entering the work area. The to-be-processed image information and the license plate number information of the vehicle recorded by the relevant personnel or equipment can be sent to the server 105, and the server 105 can execute the image recognition method provided by the embodiments of the present disclosure.

[0076] Correspondingly, the image recognition device provided by the embodiments of the present disclosure can generally be arranged in the server 105. The image recognition method provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, and 103 and / or the server 105.

[0077] Accordingly, the image recognition apparatus provided by the embodiments of the present disclosure can also be arranged in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and / or the server 105. Alternatively, the image recognition method provided by the embodiments of the present disclosure can also be executed by the terminal devices 101, 102, or 103, or by other terminal devices different from the terminal devices 101, 102, or 103. Accordingly, the image recognition apparatus provided by the embodiments of the present disclosure can also be arranged in the terminal devices 101, 102, or 103, or in other terminal devices different from the terminal devices 101, 102, or 103.

[0078] It should be understood that Figure 1 The number of terminal devices, networks and servers in the above description is only illustrative. Any number of terminal devices, networks and servers can be provided according to the implementation needs.

[0079] Figure 2 A flowchart of the image recognition method according to the embodiments of the present disclosure is schematically shown.

[0080] As Figure 2 shown, the image recognition method includes operations S210-S230.

[0081] In operation S210, a vehicle identification of a vehicle entering a work site from a work site entrance is recorded, obtaining a vehicle identification set including at least one vehicle identification.

[0082] According to the embodiments of the present disclosure, the work site may, for example, include a logistics work site, a building construction work site, an industrial park, etc., and the vehicle identification may, for example, include a license plate number of the vehicle, a work identification number of the vehicle, etc. A picture of the vehicle entering the work site from the work site entrance can be captured by a camera arranged at the work site entrance, and the picture can be identified by using a related technology to obtain the recorded vehicle identification, or the vehicle identification of the vehicle can also be recorded manually, and the embodiments of the present disclosure do not limit the specific technical means for recording the vehicle identification.

[0083] In operation S220, image information to be processed of a vehicle working around a work platform is captured, wherein the work platform is located in the work site.

[0084] In operation S230, a target vehicle identification of a target vehicle in the image information to be processed is recognized according to a predicted matching result of the image information to be processed and the vehicle identification in the vehicle identification set.

[0085] According to an embodiment of the present disclosure, the work platform can include an area in the work site, and vehicles can perform loading and unloading, road flattening, etc. around the work platform after entering the work site. The image information of the vehicles working around the work platform can be collected by the monitoring camera, and the image information can include one or more vehicles entering the work site and working around the work platform. The vehicles in the image information can be target vehicles, and the vehicle identifiers of the target vehicles can be target vehicle identifiers.

[0086] According to an embodiment of the present disclosure, because the vehicle identifiers of the vehicles entering the work site from the work site entrance are obtained to obtain a vehicle identifier set of the vehicles entering the work site, the target vehicle identifiers of the target vehicles in the image information are identified according to the image information of the vehicles working around the work platform and the predicted matching result of the vehicle identifiers in the vehicle identifier set, which can at least partially avoid the technical problems in the related art that the identification accuracy of the target vehicle identifiers is low or the vehicle identifiers cannot be identified due to unclear image information or the vehicle identifiers being polluted in the image information, avoid identification errors of the vehicle identifiers, improve the identification accuracy of the vehicle identifiers, and improve the intelligent management level of the vehicles working in the work site.

[0087] According to an embodiment of the present disclosure, operation S210, recording the vehicle identifiers of the vehicles entering the work site from the work site entrance to obtain a vehicle identifier set including at least one vehicle identifier can include the following operations.

[0088] Collecting vehicle image information of the vehicles entering the work site from the work site entrance; determining the vehicle identifiers of the vehicles entering the work site from the work site entrance according to the vehicle image information; and determining the vehicle identifier set according to the vehicle identifiers of the vehicles entering the work site from the work site entrance.

[0089] According to an embodiment of the present disclosure, the vehicle image information of the vehicles entering the work site from the work site entrance can be collected by an image collection device such as a video monitoring device, and the vehicle identifiers of the vehicles entering the work site can be identified by a related image recognition device, for example, an image recognition device based on a convolutional neural network. The vehicle identifiers of the vehicles entering the work site can be summarized by determining the vehicle identifier set according to the identified vehicle identifiers, thereby providing a necessary basis for subsequently obtaining a predicted matching result for the image information according to the image information and the vehicle identifiers.

[0090] It should be noted that the vehicle image information can also be identified by manual identification, and the specific technical means for determining the vehicle identifiers are not limited in the embodiments of the present disclosure.

[0091] In this embodiment, the vehicle identification can also be generated by recording the license plate number of the vehicle entering the work site, and then generating a vehicle identification set containing the license plate numbers of different vehicles. Using the image recognition method provided by the embodiment of the present disclosure, the license plate number of the vehicle being worked on in the to-be-processed image information can be matched with the license plate number of the vehicle entering the work site, and the license plate number of the vehicle recorded in the to-be-processed image information is recognized as the license plate number contained in the vehicle identification set, so that the recognition result of the to-be-processed image information can be avoided to be inconsistent with the vehicle identification, resulting in image recognition error, and thus the image recognition accuracy is improved.

[0092] It should be noted that after the vehicle leaves the work site, the vehicle identification corresponding to the vehicle leaving the work site in the vehicle identification set can also be deleted to reduce the number of vehicle identifications in the vehicle identification set, thereby reducing the computational load of text feature extraction of the vehicle identification in the vehicle identification set, improving the efficiency of identification for the to-be-processed image information, and avoiding the to-be-processed image information being identified as the vehicle identification of the vehicle that has left the work site, thereby improving the accuracy of image recognition.

[0093] According to the embodiment of the present disclosure, after the vehicle arrives at the work platform, the to-be-processed image information of the vehicle around the work platform can be collected by the monitoring camera. Since the license plate identification area of the vehicle in the to-be-processed image information is small, the target image area corresponding to the license plate can be automatically detected by the WPOD algorithm, and the target image area is intercepted to generate target to-be-processed image information. Then, resize operation can be performed on the target to-be-processed image information to unify the position, size, angle and format of the license plate in the target to-be-processed image information, so as to facilitate subsequent processing of the target to-be-processed image information.

[0094] Figure 3A A flowchart of identifying a target vehicle identification of a target vehicle in to-be-processed image information according to a predicted matching result of the to-be-processed image information and a vehicle identification in a vehicle identification set according to an embodiment of the present disclosure is schematically shown.

[0095] As shown in Figure 3A Operation S230 can include operations S310-S350.

[0096] In operation S310, image feature extraction is performed on the to-be-processed image information to obtain an image feature vector, wherein the to-be-processed image information records target information.

[0097] According to an embodiment of the present disclosure, the target information recorded in the to-be-processed image information can include text information, for example, in the case that the to-be-processed image information is a monitoring photo in an industrial park, the target information can include text information of vehicle license plates in the industrial park, text information in a sign in the industrial park, and the like.

[0098] According to an embodiment of the present disclosure, the image feature extraction on the to-be-processed image information can be performed based on a network model constructed by a neural network, for example, a convolutional neural network model, a long short-term memory neural network model, and the like.

[0099] It should be noted that, before the feature extraction on the to-be-processed image information, the to-be-processed image information can be pre-processed, for example, can include a pre-processing operation of cutting, stretching, and the like on the to-be-processed image information.

[0100] In operation S320, text feature extraction is performed on the vehicle identifiers in the vehicle identifier set to obtain a text feature vector.

[0101] According to an embodiment of the present disclosure, the text feature extraction on the vehicle identifiers can include extracting character-level features and word-level features in the vehicle identifiers, or can further include extracting context features in the vehicle identifiers, so that the obtained text feature vector contains the context features in the vehicle identifiers, thereby avoiding the missing of the extracted text features.

[0102] In operation S330, the image feature vector and the text feature vector are fused based on an attention mechanism to obtain an image weighted feature vector for the to-be-processed image information.

[0103] According to an embodiment of the present disclosure, the fusion of the text features in the text feature vector and the image features in the image feature vector based on the attention mechanism enables the obtained image weighted feature vector to fuse the correlation between the image features and the text features, thereby saving the calculation time and improving the calculation efficiency for subsequent determination of the prediction result of the to-be-processed image information.

[0104] In operation S340, a prediction matching result of the to-be-processed image information is determined based on the image weighted feature vector and the text feature vector.

[0105] In operation S350, in the case that the prediction matching result represents that the to-be-processed image information matches the vehicle identifier, the target information in the to-be-processed image information is recognized as the target vehicle identifier.

[0106] According to an embodiment of the present disclosure, the image weighted feature vector and the text feature vector can be processed through a network model constructed based on a neural network, and when the prediction result represents that the to-be-processed image information matches the vehicle identifier, it can be determined that the target information in the to-be-processed image information is the same as the vehicle identifier, so that the target information in the to-be-processed image information can be recognized.

[0107] According to an embodiment of the present disclosure, because the target information is recorded in the to-be-processed image information, the image feature vector obtained by performing image feature extraction on the to-be-processed image information can contain the target information; the text feature vector is obtained by performing text feature extraction on the vehicle identifiers in the vehicle identifier set; and the image weighted feature vector for the to-be-processed image information obtained by fusing the image feature vector and the text feature vector based on the attention mechanism can fuse the relevance of the image feature vector and the text feature vector, so that the prediction result of the to-be-processed image information determined based on the image weighted feature vector and the text feature vector can represent that the to-be-processed image information matches the vehicle identifier, and the target information in the to-be-processed image information is identified as the vehicle identifier, avoiding the target information recognition error, and achieving the technical effect of improving the accuracy of identifying the target information recorded in the to-be-processed image.

[0108] According to an embodiment of the present disclosure, the image feature extraction on the to-be-processed image information to obtain the image feature vector can include the following operations.

[0109] The to-be-processed image information is input into the first image feature extraction layer to extract the image features of the to-be-processed image information in the first direction and output the first image feature vector; and the to-be-processed image information is input into the second image feature extraction layer to extract the image features of the to-be-processed image information in the second direction and output the second image feature vector; wherein the image feature vector includes the first image feature vector and the second image feature vector, and the first direction is perpendicular to the second direction.

[0110] According to an embodiment of the present disclosure, the first image feature extraction layer and the second image feature extraction layer can be neural network layers constructed based on a convolutional neural network. When the to-be-processed image information is input into the first image feature extraction layer, the first convolution kernel can be moved on the to-be-processed image information along the first direction to extract the image features of the to-be-processed image information in the first direction. When the to-be-processed image information is input into the first image feature extraction layer, the first convolution kernel can be moved on the to-be-processed image information along the first direction to extract the image features of the to-be-processed image information in the first direction. When the to-be-processed image information is input into the second image feature extraction layer, the second convolution kernel can be moved on the to-be-processed image information along the second direction to extract the image features of the to-be-processed image information in the second direction. Therefore, the output image feature vector contains the first image feature vector and the second image feature vector, which can make the image feature vector retain the relevance of the to-be-processed image information in the first direction and the second direction at the same time, so as to realize the extraction of the image features of different text information formats recorded in the to-be-processed image information.

[0111] It should be noted that the first convolution kernel of the first image feature extraction layer and the second convolution kernel of the second image feature extraction layer can be the same or different, and those skilled in the art can design according to actual needs.

[0112] Figure 3B An application scenario diagram of image feature extraction on to-be-processed image information according to an embodiment of the present disclosure is schematically shown.

[0113] Figure 3C Another application scenario diagram of image feature extraction on to-be-processed image information according to an embodiment of the present disclosure is schematically shown.

[0114] In combination Figure 3B With Figure 3C As shown in the figure, the to-be-processed image information 310 can be picture information of a license plate, and the to-be-processed image information 310 can record target information "Guang A 201 XYZ". The text information format of the target information can include that the text information "Guang A" is recorded in the first row and the text information "201 XYZ" is recorded in the second row.

[0115] The to-be-processed image information 310 is input into the first image feature extraction layer 321 to extract the image features of the to-be-processed image information 310 in the first direction, and the first image feature vector 331 is output. For example, the first convolution kernel 341 can be used to extract the image features of the to-be-processed image information 310 in the horizontal direction, the first convolution kernel 341 can be a 1x4-dimensional matrix vector, and the first direction can be the horizontal direction of the to-be-processed image 310.

[0116] The to-be-processed image information 310 is input to the second image feature extraction layer 322 to extract the image features of the to-be-processed image information 310 in the second direction, and a second image feature vector 332 is output. For example, the image features of the to-be-processed image information 310 in the vertical direction can be extracted by using a second convolution kernel 342, which can be a 4*1 matrix vector, and the first direction can be the vertical direction of the to-be-processed image 310.

[0117] By extracting the image features of the to-be-processed image information 310 in the horizontal direction and the image features of the to-be-processed image information 310 in the vertical direction respectively, the first image feature vector 331 and the second image feature vector 332 contained in the image feature vector can simultaneously retain the correlation of the to-be-processed image information in the horizontal direction and the vertical direction, so that the image features of different text information formats recorded in the to-be-processed image information 310 can be extracted.

[0118] According to an embodiment of the present disclosure, the operation S320 of performing text feature extraction on the vehicle identifiers in the vehicle identifier set to obtain a text feature vector can include the following operations.

[0119] The vehicle identifiers in the vehicle identifier set are input to a text feature extraction layer, and a text feature vector is output.

[0120] According to an embodiment of the present disclosure, the text feature extraction layer can be a network layer constructed based on a neural network, for example, a network layer constructed based on a recurrent neural network, a long short-term memory network, or a bidirectional long short-term memory network.

[0121] In this embodiment, the text feature extraction layer can adopt a network layer constructed based on a bidirectional long short-term memory network (BiLSTM) to extract the context relationship features of the text in the vehicle identifier, so as to avoid the loss of feature information in the output text feature vector.

[0122] According to an embodiment of the present disclosure, the vehicle identifiers can be preprocessed before being input to the text feature extraction layer. For example, onehot hot encoding operation can be performed on each character in the vehicle identifier to obtain a target text encoding matrix. The target text encoding matrix is processed by dimension reduction to reduce the complexity of the target text encoding matrix, and a dimension-reduced target text encoding matrix is obtained. The dimension-reduced target text encoding matrix is input to the text feature extraction layer, and a text feature vector is output. In this way, the calculation speed of the text feature extraction layer can be improved, and the calculation efficiency can be improved.

[0123] It should be noted that the specific method for preprocessing the vehicle identifiers in the present embodiment is not limited, and those skilled in the art can design and select the preprocessing method according to actual needs.

[0124] According to an embodiment of the present disclosure, the text feature extraction on the vehicle identifiers in the vehicle identifier set to obtain the text feature vectors can further include the following operations.

[0125] Before the vehicle identifiers are input into the text feature extraction layer, the vehicle identifiers are completed so that the number of characters in the vehicle identifiers meets the preset rule.

[0126] According to an embodiment of the present disclosure, since the vehicle identifiers can include license plate numbers, the number of characters of the license plate numbers can be different, for example, the license plate number of a new energy related vehicle can contain 8 characters, and the license plate number of part of the vehicles can contain 7 characters. Therefore, the vehicle identifiers are completed, so that the number of characters in the vehicle identifiers meets the unified preset rule, so as to facilitate the subsequent feature extraction of the vehicle identifiers, avoid the need to design different text feature extraction methods for vehicle identifiers with different number of characters due to the different number of characters of the vehicle identifiers, thereby reducing the operation steps of extracting the text features subsequently, and improving the computing efficiency.

[0127] According to an embodiment of the present disclosure, the operation S330, the image weighted feature vector for the image information to be processed can be obtained by fusing the image feature vector and the text feature vector based on the attention mechanism, can include the following operations.

[0128] The first image feature vector and the text feature vector are input into the first attention layer to output the first image weighted feature vector, and the second image feature vector and the text feature vector are input into the second attention layer to output the second image weighted feature vector; wherein the image weighted feature vector includes the first image weighted feature vector and the second image weighted feature vector.

[0129] According to an embodiment of the present disclosure, the first attention layer or the second attention layer can be a neural network layer constructed based on the attention mechanism, for example, the first attention layer or the second attention layer can be constructed based on SoftAttention, but not limited to this, the first attention layer or the second attention layer can also be constructed based on Hard Attention, the network structure of the first attention layer and the second attention layer can be the same or different, and those skilled in the art can design the network structure of the first attention layer and the second attention layer according to actual needs.

[0130] According to an embodiment of the present disclosure, inputting the first image feature vector and the text feature vector into the first attention layer can make the first image feature vector pay attention to the feature information associated with the text feature in the image information to be processed, so that the output first image weighted feature vector contains the association between the first image feature vector and the text feature vector. Inputting the second image feature vector and the text feature vector into the second attention layer can make the second image feature vector pay attention to the feature information associated with the text feature in the image information to be processed, so that the output second image weighted feature vector contains the association between the second image feature vector and the text feature vector.

[0131] According to an embodiment of the present disclosure, operation S340, determining the predicted matching result of the image information to be processed based on the image weighted feature vector and the text feature vector can include the following operations.

[0132] Fusing and encoding the image weighted feature vector and the text feature vector to generate a prediction input vector; and inputting the prediction input vector into a prediction classification layer to output the predicted matching result of the image information to be processed.

[0133] According to an embodiment of the present disclosure, the prediction classification layer can include a network layer constructed based on a neural network, for example, a prediction classification layer constructed based on a multilayer perceptron (MLP). Fusing and encoding the image weighted feature vector and the text feature vector can make the image weighted feature and the text feature fully fused to improve the accuracy of the prediction result output by the prediction classification layer.

[0134] According to an embodiment of the present disclosure, in the case where the image weighted feature vector includes the first image feature vector and the second image feature vector, the first image feature vector, the second image feature vector and the text feature vector can be fused and encoded to fully fuse the image feature of the image information to be processed in the first direction, the image feature of the image information to be processed in the second direction and the text feature of the vehicle identification, thereby avoiding the loss of features of the image to be processed and improving the accuracy of the prediction result output by the prediction classification layer.

[0135] Figure 4 An application scenario diagram of the image recognition method according to an embodiment of the present disclosure is schematically shown.

[0136] As shown in Figure 4 The first image feature vector 411 can be used to represent the image feature of the image information to be processed in the first direction, and the second image feature vector 412 can be used to represent the image feature of the image information to be processed in the second direction. The first image feature vector 411 can be represented as a matrix vector u w，i , and the second image feature vector 412 can be represented as a matrix vector u s，iThe text feature vector 420 can represent text features of the vehicle identifications in the vehicle identification set. The text feature vector 420 can be represented as a matrix vector h i .

[0137] It is to be noted that the matrix vector u w，i , the matrix vector u s，i and the matrix vector h i may have the same dimension.

[0138] The first image feature vector 411 and the text feature vector 420 can be input into a first attention layer 431, and a first image weighted feature vector 441 can be output. The first attention layer 431 can be a neural network layer constructed based on Soft Attention. The output first image weighted feature vector 441 can be obtained by, for example, formula (1) and formula (2).

[0139]

[0140] θ w，i =∑ j α w，ij u w，ij (2)

[0141] In formula (1) and formula (2), the matrix vector u w，i represents the first image feature vector 411, and θ w，i represents the first image weighted feature vector 441.

[0142] Based on the same or similar method, the second image feature vector 412 and the text feature vector 420 can be input into a second attention layer 432, and a second image weighted feature vector 442 can be output. The second attention layer 432 can be a neural network layer constructed based on Soft Attention. The output second image weighted feature vector 442 can be obtained by, for example, formula (3) and formula (4).

[0143]

[0144] θ s，i =∑ j α s，ij u s，ij (4)

[0145] In formula (3) and formula (4), the matrix vector u s，i represents the second image feature vector 412, and θ s，i represents the second image weighted feature vector 442.

[0146] The first image weighted feature vector 441, the second image weighted feature vector 442 and the text feature vector 420 are input into the fusion encoding layer 450, and a prediction input vector 460 can be output. In this embodiment, the first image weighted feature vector 441, the second image weighted feature vector 442 and the text feature vector 420 can be fused based on a concat operation to generate the prediction input vector 460.

[0147] The prediction input vector 460 is input into the prediction classification layer 470, and a prediction result 480 can be output. In this embodiment, the prediction classification layer 470 can be constructed based on a multilayer perceptron (MLP), and the prediction result 480 can be output based on a Sigmoid function in the prediction classification layer 470. The prediction result 480 can be a probability value representing a match between the to-be-processed image and the vehicle identifier. A match probability threshold can be set, and the prediction result 480 can be determined to represent a match between the to-be-processed image and the vehicle identifier when the prediction result 480 is greater than or equal to the match probability threshold. Alternatively, a vehicle identifier corresponding to the highest probability value can be selected from a plurality of prediction results as the vehicle identifier matching the to-be-processed image information.

[0148] By determining the vehicle identifier matching the to-be-processed image information, the target information in the to-be-processed image information can be recognized as the vehicle identifier, thereby achieving the recognition of the target information in the to-be-processed image information.

[0149] According to the embodiments of the present disclosure, because the target information is recorded in the to-be-processed image information, the image feature vector obtained by performing image feature extraction on the to-be-processed image information can contain the target information; the text feature vector is obtained by performing text feature extraction on the vehicle identifiers in the vehicle identifier set; and the image weighted feature vector for the to-be-processed image information can fuse the relevance of the image feature vector and the text feature vector based on the attention mechanism, so that the prediction result of the to-be-processed image information determined based on the image weighted feature vector and the text feature vector can represent a match between the to-be-processed image information and the vehicle identifier, thereby recognizing the target information in the to-be-processed image information as the vehicle identifier, avoiding target information recognition errors, and achieving the technical effect of improving the accuracy of recognizing the target information recorded in the to-be-processed image.

[0150] The present disclosure also provides a training method of an image recognition model, which will be described below in combination with Figure 5 The method will be described in detail.

[0151] Figure 5 A flowchart of the training method of the image recognition model according to the embodiments of the present disclosure is schematically shown.

[0152] As Figure 5 shown, the training method of the image recognition model includes operations S510-S520.

[0153] At operation S510, a training sample is obtained, where the training sample includes sample image information to be processed, a sample vehicle identifier, and a label corresponding to the sample image information to be processed.

[0154] At operation S520, the initial image recognition model is trained using the training sample to obtain a trained image recognition model; where the image recognition model is used to implement the image recognition method provided by the embodiments of the present disclosure.

[0155] According to embodiments of the present disclosure, the training sample can include positive samples and negative samples. For example, the sample vehicle identifier and the image information to be processed that records the same sample vehicle identifier can be matched to form a positive sample, and the label corresponding to the positive sample can represent that the sample image information to be processed matches the sample vehicle identifier. The sample vehicle identifier and the image information to be processed that records a different sample vehicle identifier can be matched to form a negative sample, and the label corresponding to the negative sample can represent that the sample image information to be processed does not match the sample vehicle identifier. The positive sample and the negative sample are input to the initial image recognition model, and the initial image recognition model is trained using the label corresponding to the positive sample and the label corresponding to the negative sample, respectively, to obtain the trained image recognition model.

[0156] Based on the above image recognition method, the present disclosure further provides an image recognition device. The device will be described in detail below. Figure 6

[0157] Figure 6 A block diagram of an image recognition device according to an embodiment of the present disclosure is schematically shown.

[0158] As Figure 6 shown, the image recognition device 600 can include a recording module 610, a collection module 620, and an identification module 630.

[0159] The recording module 610 is configured to record vehicle identifiers of vehicles entering a work site from an entrance of the work site to obtain a vehicle identifier set including at least one vehicle identifier;

[0160] The collection module 620 is configured to collect image information to be processed of vehicles working around a work platform, where the work platform is located in the work site; and

[0161] The identification module 630 is configured to identify a target vehicle identifier of a target vehicle in the image information to be processed according to a predicted matching result of the image information to be processed and the vehicle identifiers in the vehicle identifier set.​

[0162] According to an embodiment of the present disclosure, the identification module can comprise: an image feature extraction sub-module, a text feature extraction sub-module, a feature fusion sub-module, a prediction sub-module, and an identification sub-module.

[0163] The image feature extraction sub-module is configured to perform image feature extraction on the to-be-processed image information to obtain an image feature vector, wherein the to-be-processed image information records target information.

[0164] The text feature extraction sub-module is configured to perform text feature extraction on the vehicle identifiers in the vehicle identifier set to obtain a text feature vector.

[0165] The feature fusion sub-module is configured to fuse the image feature vector and the text feature vector based on an attention mechanism to obtain an image weighted feature vector for the to-be-processed image information.

[0166] The prediction sub-module is configured to determine a prediction matching result of the to-be-processed image information based on the image weighted feature vector and the text feature vector.

[0167] The identification sub-module is configured to identify the target information in the to-be-processed image information as a target vehicle identifier in a case where the prediction matching result represents that the to-be-processed image information matches the vehicle identifier.

[0168] According to an embodiment of the present disclosure, the image feature extraction sub-module can comprise: a first image feature extraction unit and a second image feature extraction unit.

[0169] The first image feature extraction unit is configured to input the to-be-processed image information into a first image feature extraction layer to extract image features of the to-be-processed image information in a first direction and output a first image feature vector.

[0170] The second image feature extraction unit is configured to input the to-be-processed image information into a second image feature extraction layer to extract image features of the to-be-processed image information in a second direction and output a second image feature vector; wherein the image feature vector comprises the first image feature vector and the second image feature vector, and the first direction is perpendicular to the second direction.

[0171] According to an embodiment of the present disclosure, the feature fusion sub-module comprises: a first attention unit and a second attention unit.

[0172] The first attention unit is configured to input the first image feature vector and the text feature vector into a first attention layer to output a first image weighted feature vector.

[0173] The second attention unit is configured to input the second image feature vector and the text feature vector into a second attention layer, and output a second image weighted feature vector; wherein the image weighted feature vector comprises the first image weighted feature vector and the second image weighted feature vector.

[0174] According to an embodiment of the present disclosure, the prediction sub-module can comprise a fusion unit and a prediction unit.

[0175] The fusion unit is configured to fuse and encode the image weighted feature vector and the text feature vector, and generate a prediction input vector.

[0176] The prediction unit is configured to input the prediction input vector into a prediction classification layer, and output a prediction matching result of the to-be-processed image information.

[0177] According to an embodiment of the present disclosure, the text feature extraction sub-module can comprise a text feature extraction unit.

[0178] The text feature extraction unit is configured to input a vehicle identification in the vehicle identification set into a text feature extraction layer, and output a text feature vector.

[0179] According to an embodiment of the present disclosure, the text feature extraction sub-module can further comprise a preprocessing unit.

[0180] The preprocessing unit is configured to complete the vehicle identification before inputting the vehicle identification into the text feature extraction layer, so as to make the number of characters in the vehicle identification meet a preset rule.

[0181] According to an embodiment of the present disclosure, the recording module can comprise a collection sub-module, a determination sub-module and a recording sub-module.

[0182] The collection sub-module is configured to collect vehicle image information of a vehicle entering a work site from a work site entrance.

[0183] The determination sub-module is configured to determine a vehicle identification of the vehicle entering the work site from the work site entrance according to the vehicle image information.

[0184] The recording sub-module is configured to determine a vehicle identification set according to the vehicle identification of the vehicle entering the work site from the work site entrance.

[0185] The present disclosure further provides a training device of an image recognition model. The following will be described in detail in combination with Figure 7 The training device will be described in detail.

[0186] Figure 7 A block diagram of the training device of the image recognition model according to an embodiment of the present disclosure is schematically shown.

[0187] As Figure 7As shown, the training apparatus 700 of the image recognition model can include a sample acquisition module 710 and a training module 720.

[0188] The sample acquisition module 710 is configured to acquire a training sample, where the training sample includes sample image information to be processed, a sample vehicle identifier, and a label corresponding to the sample image information to be processed.

[0189] The training module 720 is configured to train an initial image recognition model by using the training sample to obtain a trained image recognition model.

[0190] The image recognition model is configured to implement the image recognition method provided in the present disclosure.

[0191] It should be noted that the embodiments of the device part of the present disclosure correspond to the same or similar embodiments of the method part of the present disclosure, and the present disclosure will not be repeated here.

[0192] Any one or more of the modules, sub-modules, units and sub-units according to the embodiments of the present disclosure, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules and units according to the embodiments of the present disclosure can be split into multiple modules for implementation. Any one or more of the modules, sub-modules, units and sub-units according to the embodiments of the present disclosure can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of hardware or firmware through integration or packaging of circuits, or in any one of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, one or more of the modules, sub-modules, units and sub-units according to the embodiments of the present disclosure can be at least partially implemented as computer program modules that can perform corresponding functions when executed.

[0193] For example, any of the recording module 610, the collecting module 620, the identifying module 630, the sample obtaining module 710 and the training module 720 can be combined in one module / unit / sub-unit, or any of the modules / sub-modules / units / sub-units can be split into multiple modules / sub-modules / units / sub-units. Alternatively, at least part of the functions of one or more of the modules / sub-modules / units / sub-units can be combined with at least part of the functions of other modules / sub-modules / units / sub-units, and implemented in one module / sub-module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the recording module 610, the collecting module 620, the identifying module 630, the sample obtaining module 710 and the training module 720 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable manner of hardware or firmware that can be integrated or packaged with a circuit, or implemented in any one of software, hardware and firmware or in a proper combination of any of them. Alternatively, at least one of the recording module 610, the collecting module 620, the identifying module 630, the sample obtaining module 710 and the training module 720 can be at least partially implemented as a computer program module that can perform corresponding functions when the computer program module is run.

[0194] Figure 8 A block diagram of an electronic device suitable for implementing the method provided by the embodiments of the present disclosure is schematically shown. Figure 8 The electronic device shown is merely an example and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0195] As Figure 8 shown, the electronic device 800 according to the embodiments of the present disclosure includes a processor 801 that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 802 or loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 can include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset, and / or a special-purpose microprocessor (such as an application specific integrated circuit (ASIC)), etc. The processor 801 can also include an on-board memory for cache use. The processor 801 can include a single processing unit or a plurality of processing units for performing different actions of the method processes according to the embodiments of the present disclosure.

[0196] In the RAM 803, various programs and data required for the operation of the electronic device 800 are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other via the bus 804. The processor 801 performs various operations of the method flow according to the embodiments of the present disclosure by executing the programs in the ROM 802 and / or the RAM 803. It is noted that the programs can also be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 can also perform various operations of the method flow according to the embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0197] According to an embodiment of the present disclosure, the electronic device 800 can further include an input / output (I / O) interface 805, which is also connected to the bus 804. The system 800 can further include one or more of the following components connected to the I / O interface 805: an input part 806 including a keyboard, a mouse, etc.; an output part 807 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 808 including a hard disk, etc.; and a communication part 809 including a network interface card such as a LAN card, a modem, etc. The communication part 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 810 as necessary, so that a computer program read out therefrom is installed in the storage part 808 as necessary.

[0198] According to an embodiment of the present disclosure, the method flow according to the embodiments of the present disclosure can be implemented as a computer software program. For example, the embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable storage medium, the computer program containing program codes for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above-described functions defined in the system of the embodiments of the present disclosure are performed. According to an embodiment of the present disclosure, the system, the device, the apparatus, the module, the unit, etc. described above can be implemented by computer program modules.

[0199] The present disclosure also provides a computer-readable storage medium, which can be included in the device / apparatus / system described in the above embodiments, or can exist separately without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, which, when executed, implement the methods according to the embodiments of the present disclosure.

[0200] According to an embodiment of the present disclosure, the computer readable storage medium can be a non-transitory computer readable storage medium. For example, it can include, but is not limited to, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), portable compact disk read only memory (CD-ROM), optical memory devices, magnetic memory devices, or any suitable combination of the foregoing. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0201] For example, according to an embodiment of the present disclosure, the computer readable storage medium can include one or more memories other than the ROM 802 and / or the RAM 803 described above.

[0202] The flow diagrams and the block diagrams in the drawings are illustrations of possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0203] Those skilled in the art will understand that the features recited in the various embodiments and / or claims of the present disclosure can be combined and / or integrated in a variety of ways, even if such combinations or integrations are not expressly specified in the present disclosure. In particular, the features recited in the various embodiments and / or claims of the present disclosure can be combined and / or integrated in a variety of ways without departing from the spirit and teachings of the present disclosure. All such combinations and / or integrations are within the scope of the present disclosure.

[0204] The above describes embodiments of the present disclosure. However, these embodiments are merely for illustrative purposes, and are not intended to limit the scope of the present disclosure. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Those skilled in the art can make various substitutions and modifications without departing from the scope of the present disclosure, and these substitutions and modifications should all fall within the scope of the present disclosure.

Claims

1. An image recognition method, comprising: Record the vehicle identifiers of vehicles entering the work site from the work site entrance, and obtain a set of vehicle identifiers including at least one vehicle identifier; Collect image information to be processed from vehicles working around the work platform, wherein the work platform is located in the work site; Image feature extraction is performed on the image information to be processed to obtain an image feature vector, wherein the image information to be processed contains target information; Text feature extraction is performed on the vehicle identifiers in the vehicle identifier set to obtain text feature vectors; Based on the attention mechanism, the image feature vector and the text feature vector are fused to obtain an image weighted feature vector for the image information to be processed; Based on the image weighted feature vector and the text feature vector, the predicted matching result of the image information to be processed is determined; and If the predicted matching result indicates that the image information to be processed matches the vehicle identifier, the target information in the image information to be processed is identified as the target vehicle identifier.

2. The method according to claim 1, wherein, Image feature extraction is performed on the image information to be processed to obtain an image feature vector, including: The image information to be processed is input into the first image feature extraction layer to extract image features of the image information in the first direction and output a first image feature vector; and The image information to be processed is input into the second image feature extraction layer to extract the image features of the image information to be processed in the second direction and output the second image feature vector; The image feature vector includes the first image feature vector and the second image feature vector, wherein the first direction is perpendicular to the second direction.

3. The method according to claim 2, wherein, The process of fusing the image feature vector and the text feature vector based on the attention mechanism to obtain the image weighted feature vector for the image information to be processed includes: The first image feature vector and the text feature vector are input into the first attention layer, and the first image weighted feature vector is output; and The second image feature vector and the text feature vector are input into the second attention layer, and the second image weighted feature vector is output. The image weighted feature vector includes the first image weighted feature vector and the second image weighted feature vector.

4. The method according to claim 1, wherein, The step of determining the prediction matching result of the image information to be processed based on the image weighted feature vector and the text feature vector includes: The image weighted feature vector and the text feature vector are fused and encoded to generate a prediction input vector; and The predicted input vector is input into the prediction classification layer, and the predicted matching result of the image information to be processed is output.

5. The method according to claim 1, wherein, Text feature extraction is performed on the vehicle identifiers in the vehicle identifier set to obtain text feature vectors, including: The vehicle identifiers in the vehicle identifier set are input into the text feature extraction layer, and the text feature vector is output.

6. The method according to claim 5, wherein, The step of extracting text features from vehicle identifiers in the vehicle identifier set to obtain text feature vectors further includes: Before inputting the vehicle identifier into the text feature extraction layer, the vehicle identifier is completed so that the number of characters in the vehicle identifier meets the preset rules.

7. The method according to claim 1, wherein, Record the vehicle identifiers of vehicles entering the work site from the work site entrance, resulting in a vehicle identifier set including at least one vehicle identifier, including: Collect vehicle image information of vehicles entering the work site from the entrance of the work site; Based on the vehicle image information, determine the vehicle identification of the vehicle entering the work site from the work site entrance; The vehicle identifier set is determined based on the vehicle identifiers of vehicles entering the work site from the work site entrance.

8. A method for training an image recognition model, comprising: Acquire training samples, wherein the training samples include sample image information to be processed, sample vehicle identifiers, and labels corresponding to the sample image information to be processed; as well as The initial image recognition model is trained using the training samples to obtain the trained image recognition model; The image recognition model is used to implement the image recognition method according to any one of claims 1 to 7.

9. An image recognition device, comprising: The recording module is used to record the vehicle identification of vehicles entering the work site from the work site entrance, and to obtain a set of vehicle identification including at least one vehicle identification. A data acquisition module is used to acquire image information to be processed from vehicles working around the work platform, wherein the work platform is located in the work area; and The identification module is used to identify the target vehicle identifier of the target vehicle in the image information to be processed based on the predicted matching result between the image information to be processed and the vehicle identifiers in the vehicle identifier set. The identification module includes: The image feature extraction submodule is used to extract image features from the image information to be processed to obtain an image feature vector, wherein the image information to be processed contains target information; The text feature extraction submodule is used to extract text features from the vehicle identifiers in the vehicle identifier set to obtain text feature vectors. The feature fusion submodule is used to fuse the image feature vector and the text feature vector based on an attention mechanism to obtain an image weighted feature vector for the image information to be processed. The prediction submodule is used to determine the prediction matching result of the image information to be processed based on the image weighted feature vector and the text feature vector; and The identification submodule is used to identify the target information in the image information to be processed as the target vehicle identifier when the prediction matching result indicates that the image information to be processed matches the vehicle identifier.

10. A training device for an image recognition model, comprising: The sample acquisition module is used to acquire training samples, wherein the training samples include sample image information to be processed, sample vehicle identifiers, and labels corresponding to the sample image information to be processed. as well as The training module is used to train an initial image recognition model using the training samples to obtain a trained image recognition model. The image recognition model is used to implement the image recognition method according to any one of claims 1 to 7.

11. A computer system, comprising: One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 7, or implement the training method of claim 8.

12. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to implement the method of any one of claims 1 to 7, or to implement the training method of claim 8.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 7, or implements the training method according to claim 8.

Images