Component recognition method and device based on image recognition model

Abstract

The application provides a component recognition method and device based on an image recognition model. The model comprises a first feature extraction layer, a second feature extraction layer, a first feature fusion layer, a second feature fusion layer, and a component recognition layer. The method comprises: performing feature extraction on a to-be-recognized image comprising a loading tool through the first feature extraction layer to obtain a global feature map and a probability feature map; performing feature fusion on the global feature map and the probability feature map through the first feature fusion layer to obtain an intermediate feature map; performing feature extraction on the intermediate feature map through the second feature extraction layer to obtain an edge feature map of each component, a main body feature map, and a background feature map of the loading tool; performing feature fusion on the edge feature map, the main body feature map, and the background feature map through the second feature fusion layer to obtain a target feature map; and performing component recognition through the component recognition layer based on the target feature map to obtain a component recognition result. In this way, the recognition efficiency and accuracy of the components on the loading tool can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a test question classification model training method and device. BACKGROUND

[0002] With the progress of science and technology and the increase of national import and export trade, the logistics transportation industry is in an environment of rapid development and increasing demand year by year, and the demand for loading tools is also increasing year by year. When some parts of the loading tool are lost or damaged, it will affect the continued use of the loading tool and the successful transportation of the goods to some extent. Therefore, in order to ensure the safety of goods during transportation, quality detection needs to be performed on the parts on the loading tool to ensure that the corresponding loading tool is not damaged. However, the types of parts on the loading tool are various, which reduces the efficiency and accuracy of identifying the corresponding parts during quality detection of the parts on the loading tool. SUMMARY

[0003] The embodiment of the present application provides a component identification method and device based on an image recognition model, electronic equipment, computer readable storage medium and computer program product, which can improve the identification efficiency and accuracy of the components on the loading tool.

[0004] The technical scheme of the embodiment of the present application is as follows:

[0005] The embodiment of the present application provides a component identification method based on an image recognition model, wherein the image recognition model comprises a first feature extraction layer, a second feature extraction layer, a first feature fusion layer, a second feature fusion layer and a component identification layer, and comprises the following steps:

[0006] The first feature extraction layer is used for performing image feature extraction on a to-be-identified image comprising a loading tool to obtain a global feature map corresponding to the loading tool, and performing probability feature extraction on the to-be-identified image to obtain a probability feature map used for indicating the connection probability between at least two components;

[0007] The loading tool comprises at least two components, and the at least two components have a fixed component connection relationship.

[0008] The first feature fusion layer is used for performing feature fusion on the global feature map and the probability feature map to obtain an intermediate feature map;

[0009] The second feature extraction layer is used for performing feature extraction on the intermediate feature map to obtain an edge feature map of each component, a main body feature map of each component and a background feature map of the loading tool;

[0010] fusing the edge feature map, the main body feature map and the background feature map through the second feature fusion layer to obtain a target feature map;

[0011] Based on the target feature map, identifying each component included in the loading tool in the image to be recognized through the component recognition layer to obtain a corresponding component recognition result.

[0012] Embodiments of the present application provide a training method based on an image recognition model, the image recognition model comprising a first feature extraction layer, a second feature extraction layer, a first feature fusion layer, a second feature fusion layer and a component recognition layer, comprising:

[0013] extracting image features of an image training sample carrying a label and comprising a loading tool through the first feature extraction layer to obtain a global feature map corresponding to the loading tool, and extracting probability features of the image training sample to obtain a probability feature map indicating a connection probability between at least two components;

[0014] The loading tool comprises at least two components, and the at least two components have a fixed component connection relationship.

[0015] fusing the global feature map and the probability feature map through the first feature fusion layer to obtain an intermediate feature map;

[0016] extracting features of the intermediate feature map through the second feature extraction layer to obtain an edge feature map of each component, a main body feature map of each component and a background feature map of the loading tool;

[0017] fusing the edge feature map, the main body feature map and the background feature map through the second feature fusion layer to obtain a target feature map;

[0018] Based on the target feature map, identifying each component included in the loading tool in the image to be recognized through the component recognition layer to obtain a corresponding component recognition result;

[0019] obtaining a difference between the component recognition result and the label, and training the image recognition model based on the difference;

[0020] The image recognition model obtained by training is used to identify each component of a target loading tool in an image to be recognized comprising the target loading tool, and the target loading tool and the loading tool have the same component structure.

[0021] The embodiment of the application provides a component recognition device based on an image recognition model, the image recognition model comprises a first feature extraction layer, a second feature extraction layer, a first feature fusion layer, a second feature fusion layer and a component recognition layer, and comprises:

[0022] The first feature extraction module is configured to perform image feature extraction on the to-be-recognized image comprising the loading tool through the first feature extraction layer, obtain a global feature map corresponding to the loading tool, and perform probability feature extraction on the to-be-recognized image, to obtain a probability feature map used to indicate a connection probability between at least two components.

[0023] The first feature fusion module is configured to perform feature fusion on the global feature map and the probability feature map through the first feature fusion layer, to obtain an intermediate feature map.

[0024] The second feature extraction module is configured to perform feature extraction on the intermediate feature map through the second feature extraction layer, to obtain an edge feature map of each component, a main body feature map of each component and a background feature map of the loading tool.

[0025] The second feature fusion module is configured to perform feature fusion on the edge feature map, the main body feature map and the background feature map through the second feature fusion layer, to obtain a target feature map.

[0026] The recognition module is configured to perform recognition on each component included in the loading tool in the to-be-recognized image through the component recognition layer based on the target feature map, to obtain a corresponding component recognition result.

[0027] In the above scheme, the first feature extraction layer comprises a global feature extraction layer and a probability feature extraction layer; the first feature extraction module is further configured to perform image feature extraction on the to-be-recognized image comprising the loading tool through the global feature extraction layer, to obtain a global feature map corresponding to the loading tool; and perform probability feature extraction on the to-be-recognized image through the probability feature extraction layer, to obtain a probability feature map used to indicate a connection probability between at least two components.

[0028] In the above scheme, the first feature extraction module is further configured to perform image feature extraction on the to-be-recognized image comprising the loading tool through the global feature extraction layer, to obtain at least two global sub-feature maps; wherein the sizes of the global sub-feature maps are different; and perform feature splicing on the global sub-feature maps, to obtain a global feature map corresponding to the loading tool.

[0029] In the foregoing scheme, the first feature extraction module is further configured to perform feature extraction on the to-be-identified image through the probability feature extraction layer to obtain a part feature map of each part and a connectivity feature map used for indicating a connection relationship between the at least two parts; and perform feature fusion on the part feature map of each part and the connectivity feature map to obtain a probability feature map used for indicating a connection probability between the at least two parts.

[0030] In the foregoing scheme, the first feature extraction module is further configured to, for each part, perform feature fusion on the part feature map of the part and the connectivity feature map of the part to obtain an intermediate probability feature map of the part, where the intermediate probability feature map is used for indicating a connection probability between the part and any one of the other parts except the part; and perform feature splicing on the intermediate probability feature map of each part to obtain the probability feature map used for indicating the connection probability between the at least two parts.

[0031] In the foregoing scheme, the first feature fusion module is further configured to perform product processing on the global feature map and the probability feature map through the first feature fusion layer to obtain a product feature map; and use the product feature map as the intermediate feature map.

[0032] In the foregoing scheme, the second feature extraction layer includes a loading tool feature extraction layer, a part main body feature extraction layer, and a part edge feature extraction layer; and the second feature extraction module is further configured to perform feature extraction on the intermediate feature map through the part main body feature extraction layer to obtain a main body feature map of each part; perform feature extraction on the intermediate feature map through the part edge feature extraction layer to obtain an edge feature map of each part; and perform feature extraction on the intermediate feature map through the loading tool feature extraction layer to obtain a background feature map of the loading tool.

[0033] In the foregoing scheme, the apparatus further includes an image acquisition module, which is configured to acquire an image obtained by image acquisition on the loading tool by an image acquisition device as the to-be-identified image.

[0034] In the foregoing scheme, the apparatus further includes a detection module, which is configured to perform defect detection on each part identified based on the part identification result to obtain a detection result; and when the detection result indicates that a target part in the at least two parts has a defect, output defect prompt information corresponding to the target part.

[0035] Embodiments of the present application provide a training apparatus of an image recognition model, the image recognition model including a first feature extraction layer, a second feature extraction layer, a first feature fusion layer, a second feature fusion layer, and a part identification layer, and the training apparatus includes:

[0036] a first feature extraction module configured to perform image feature extraction on image training samples carrying labels and including loading tools by the first feature extraction layer to obtain a global feature map corresponding to the loading tools, and perform probability feature extraction on the image training samples to obtain a probability feature map indicating a connection probability between at least two components; the loading tools include the at least two components, and the at least two components have a fixed component connection relationship;

[0037] a first feature fusion module configured to perform feature fusion on the global feature map and the probability feature map by the first feature fusion layer to obtain an intermediate feature map;

[0038] a second feature extraction module configured to perform feature extraction on the intermediate feature map by the second feature extraction layer to obtain an edge feature map of each of the components, a main body feature map of each of the components, and a background feature map of the loading tools;

[0039] a second feature fusion module configured to perform feature fusion on the edge feature map, the main body feature map, and the background feature map by the second feature fusion layer to obtain a target feature map;

[0040] a recognition module configured to identify each component included in the loading tools in the image training samples by the component recognition layer based on the target feature map to obtain a corresponding component recognition result;

[0041] a training module configured to obtain a difference between the component recognition result and the label, and train the image recognition model based on the difference; the trained image recognition model is used to identify each component of a target loading tool in a to-be-identified image including the target loading tool, and the target loading tool has the same component structure as the loading tools.

[0042] An electronic device is provided in an embodiment of the present application, and the electronic device includes:

[0043] a memory configured to store executable instructions;

[0044] a processor configured to execute the executable instructions stored in the memory to implement a component recognition method based on an image recognition model provided in an embodiment of the present application.

[0045] An electronic device is provided in an embodiment of the present application, and the electronic device includes:

[0046] a memory configured to store executable instructions;

[0047] a processor configured to execute the executable instructions stored in the memory to implement a training method of an image recognition model provided in an embodiment of the present application.

[0048] The embodiment of the present application provides a computer readable storage medium, which stores executable instructions, and is used for causing a processor to execute the method for identifying components based on an image recognition model.

[0049] The embodiment of the present application provides a computer readable storage medium, which stores executable instructions, and is used for causing a processor to execute the method for training an image recognition model.

[0050] The embodiment of the present application provides a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method for identifying components based on an image recognition model.

[0051] The embodiment of the present application provides a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method for training an image recognition model.

[0052] The embodiment of the present application has the following beneficial effects:

[0053] First, feature extraction is performed on the to-be-identified image to obtain a global feature map of the loading tool and a probability feature map of a component on the loading tool, and then feature extraction is performed on a feature map obtained by fusing the global feature map and the probability feature map to obtain an edge feature map of each component, a main body feature map and a background feature map of the loading tool, so that component identification is performed based on a feature map obtained by fusing the edge feature map of each component, the main body feature map and the background feature map of the loading tool, and a component identification result is obtained. In this way, through two feature extraction processes, multi-dimensional features corresponding to the loading tool are obtained, so that the components on the loading tool are identified according to the multi-dimensional features, the comprehensiveness of the obtained features is improved, and the identification efficiency and the identification accuracy of the components on the loading tool are improved. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 FIG. 1 is a schematic diagram of a plurality of to-be-identified images collected by the embodiment of the present application;

[0055] Figure 2 FIG. 2 is a schematic diagram of a plate surface structure of a container according to the embodiment of the present application;

[0056] Figure 3 is a schematic diagram of a corner post structure and a beam structure of a container provided by an embodiment of the present application;

[0057] Figure 4 is a schematic diagram of a door lock rod structure and a corner piece structure of a container provided by an embodiment of the present application;

[0058] Figure 5 is a schematic diagram of an architecture of a component recognition system 100 based on an image recognition model provided by an embodiment of the present application;

[0059] Figure 6 is a schematic diagram of a structure of an electronic device provided by an embodiment of the present application;

[0060] Figure 7 is a schematic diagram of a flow of a component recognition method based on an image recognition model provided by an embodiment of the present application;

[0061] Figure 8 is a schematic diagram of a structure of an image recognition model provided by an embodiment of the present application;

[0062] Figure 9 is a schematic diagram of an image to be recognized including a loading tool captured by a camera provided by an embodiment of the present application;

[0063] Figure 10 is a schematic diagram of a flow of determining a global feature map and a probability feature map corresponding to a loading tool based on a first feature extraction layer provided by an embodiment of the present application;

[0064] Figure 11 is a schematic diagram of a structure of a first feature extraction layer provided by an embodiment of the present application;

[0065] Figure 12 is a schematic diagram of a structure of a global feature extraction layer provided by an embodiment of the present application;

[0066] Figure 13 is a schematic diagram of a structure of a probability feature extraction layer provided by an embodiment of the present application;

[0067] Figure 14 is a schematic diagram of a structure of a first feature fusion layer provided by an embodiment of the present application;

[0068] Figure 15 is a schematic diagram of a structure of a second feature extraction layer provided by an embodiment of the present application;

[0069] Figure 16 is a comparison schematic diagram of a main feature map of each component and an image to be recognized provided by an embodiment of the present application;

[0070] Figure 17 is a comparison schematic diagram of an edge feature map of each component and an image to be recognized provided by an embodiment of the present application;

[0071] Figure 18 is a comparison diagram of the background feature map of the loading tool provided by the embodiment of the present application and the image to be recognized;

[0072] Figure 19 is a structural diagram of the second feature fusion layer provided by the embodiment of the present application;

[0073] Figure 20 is a flow diagram of the training method of the image recognition model provided by the embodiment of the present application;

[0074] Figure 21 is a flow diagram of the component recognition method based on the image recognition model provided by the embodiment of the present application;

[0075] Figure 22 is a flow diagram of the component recognition method based on the image recognition model provided by the embodiment of the present application;

[0076] Figure 23 is a processing diagram of the attention mechanism module provided by the embodiment of the present application;

[0077] Figure 24 is a processing diagram of the global feature extraction module provided by the embodiment of the present application;

[0078] Figure 25 is a processing diagram of the three-branch network output structure provided by the embodiment of the present application;

[0079] Figure 26 is a structural diagram of the training device 2600 of the image training model provided by the embodiment of the present application. DETAILED DESCRIPTION

[0080] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be described in further detail below with reference to the drawings, and the described embodiments should not be regarded as limiting the present application. All other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0081] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0082] In the following description, the terms "first", "second", "third" are merely used to distinguish similar objects, and do not represent a specific order of the objects. Understandably, "first", "second", "third" can be interchanged in a specific order or sequence as allowed, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In the following description, the term "a plurality of" refers to at least two.

[0083] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terms used herein are merely for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0084] The relevant data collection process in the embodiments of the present application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0085] Before further detailing the embodiments of the present application, the terms and phrases involved in the embodiments of the present application are explained, and the terms and phrases involved in the embodiments of the present application are applicable to the following explanations.

[0086] 1) Fixed connection structure, a fixed structure for connecting parts on a loading tool, such as the connection structure between the door handle and the door lock rod on a container, or the connection structure between the corner piece and the corner column on a container, etc.

[0087] 2) Container parts, parts for assembling containers, such as corrugated plate surfaces, door lock rods, door handles, corner columns, corner pieces, bottom crossbeams, and floors, etc., different parts have different sizes and structures.

[0088] 3) Client, an application program running in a terminal for providing various services, such as a video client, an instant messaging client, a browser client, an education client, a part identification client, etc.

[0089] 4) Attention mechanism, an optimization structure commonly used in natural language processing, which has also shown good optimization effect in the visual field recently. By directly learning the direct action weight information between pixels, the relationship between the blocks of the image can be more accurately displayed, which is more conducive to the segmentation of pixels.

[0090] 5) Global pool, a common pooling operation in convolutional neural networks, which is used to help obtain the feature map with the largest receptive field, so as to facilitate the subsequent feature map to have better global features.

[0091] 6) Receptive Field: The size of the region on the input image that each pixel in the feature map output by each layer of the convolutional neural network maps to. In other words, each point in the feature map corresponds to a region in the input image.

[0092] 7) Convolutional Neural Network (CNN): A type of feedforward neural network, generally composed of one or more convolutional layers (network layers that use convolutional mathematical operations) and fully connected layers at the end. The neurons inside the network can respond to a part of the input image, and it generally has excellent performance in the field of visual image processing.

[0093] The rise of globalization has deepened the flow of goods between countries, and containers, as the main means of loading goods during maritime transport, are widely used in various transportation scenarios such as ports and railways. A container is a standardized steel box with a uniform structure that facilitates mechanized loading and unloading. Container quality inspection is a crucial link in ensuring the continued long-term use of containers. When some components of a container are lost or damaged, it will affect the continued use of the container and the successful transportation of goods to a certain extent. However, different components of a container have different quality control standards, which makes the overall quality inspection of containers, including the defect detection of individual components, diverse and complex, requiring a large investment of human resources and low efficiency. Specifically, there are three main difficulties in the quality inspection of containers: First, when identifying components of a container by capturing images, the mobility, unfixed viewing angle, and unfixed position of the image acquisition equipment result in a variety of container shapes in the acquired images, and the size and angle of the same component also vary. Figure 1 As shown, Figure 1 This is a schematic diagram of multiple images to be identified collected according to the embodiments of this application; and, in addition to the container area to be inspected in the image, there are also various background areas, such as the ground, the sky, and even other stacked containers, etc. Therefore, the diversity of the containers themselves and the background increases the difficulty of identifying container components in the images collected by the handheld device; secondly, a container is a standardized and sophisticated loading tool with numerous parts, reaching hundreds of types, and varying in size, shape, and color. For example, parts such as air vents and door cam heads have complex and delicate structures; while the corrugated panels, floors, and door panels of the container have large areas and simple panel structures, such as Figure 2 As shown, Figure 2 This is a schematic diagram of the container panel structure provided in the embodiments of this application, where the corner posts and crossbeams have elongated edges, such as... Figure 3 As shown, Figure 3is a schematic diagram of the corner post structure and the beam structure of the container provided by the embodiments of the present application, and the door lock rod and the corner piece are structurally fine, the division of the edges is difficult to achieve accurate boundaries, and other types of blocks such as panel blocks are also easily mixed in, as shown in Figure 4 , Figure 4 is a schematic diagram of the door lock rod structure and the corner piece structure of the container provided by the embodiments of the present application; third, the components of the container have great differences in tolerance standards for defects due to differences in structure and function, so the same defect such as deformation requires different requirements on different components, and therefore the accuracy of component identification is a prerequisite for this problem, that is, higher accuracy requirements are put forward for component identification.

[0094] Based on this, the embodiments of the present application provide a component identification method and device based on an image recognition model, an electronic device, a computer readable storage medium and a computer program product, a mobile handheld camera device is used to obtain a photo of a container, and then a deep learning image segmentation algorithm is used to perform pixel-level identification on the component regions of the container main body in the collected photo, thereby providing specific component positions and other information for subsequent defect detection and other related tasks on the components, so that manpower, material resources can be saved, and the quality detection efficiency and accuracy can be further improved.

[0095] Referring to Figure 5 , Figure 5 is a schematic diagram of the architecture of the component identification system 100 based on the image recognition model provided by the embodiments of the present application, in order to realize the application scenario of component identification based on the image recognition model (for example, the application scenario of component identification based on the image recognition model can be the application scenario of the component identification APP in the identification of the components of the container, for example, the user uses a handheld camera device to take a photo of the container, and then the component identification APP identifies the components of the container based on the photo to determine the components on the container), the terminal (exemplarily shown as the terminal 400) is connected to the server 200 through the network 300, and the network 300 can be a wide area network or a local area network, or a combination of the two.

[0096] The terminal 400 is used for a user to use the client 401 to display on the display interface 401-1. The terminal 400 and the server 200 are connected to each other through a wired or wireless network.

[0097] The terminal 400 is configured to obtain a to-be-identified image including a loading tool, and send the to-be-identified image to the server 200.

[0098] The server 200 is configured to receive the to-be-identified image including the loading tool sent by the terminal 400; perform image feature extraction on the to-be-identified image including the loading tool through a first feature extraction layer to obtain a global feature map corresponding to the loading tool, and perform probability feature extraction on the to-be-identified image to obtain a probability feature map used to indicate a connection probability between at least two components; wherein the loading tool includes the at least two components, and the at least two components have a fixed component connection relationship; perform feature fusion on the global feature map and the probability feature map through a first feature fusion layer to obtain an intermediate feature map; perform feature extraction on the intermediate feature map through a second feature extraction layer to obtain an edge feature map of each component, a main body feature map of each component, and a background feature map of the loading tool; perform feature fusion on the edge feature map, the main body feature map, and the background feature map through a second feature fusion layer to obtain a target feature map; identify each component included in the loading tool in the to-be-identified image based on the target feature map through a component identification layer to obtain a corresponding component identification result; and send the component identification result corresponding to the to-be-identified image to the terminal 400.

[0099] The terminal 400 is configured to receive the component identification result corresponding to the to-be-identified image sent by the server 200, so as to present the corresponding component identification result in the display interface 401-1.

[0100] In some embodiments, the server 200 can be a stand-alone physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal 400 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a set-top box, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, and a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated message device, a portable game device, a smart speaker, and a smart watch), but is not limited thereto. The terminal device and the server can be connected directly or indirectly through wired or wireless communication, and the present application is not limited in this embodiment.

[0101] Referring to Figure 6 , Figure 6 FIG. 1 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. In actual application, the electronic device can be Figure 5 the server 200 or the terminal 400 shown in the figure, referring to Figure 6 , Figure 6The illustrated electronic device includes at least one processor 410, memory 450, at least one network interface 420, and a user interface 430. The various components of terminal 400 are coupled together by a bus system 440, which is configured to permit communication between the components. The bus system 440 includes a power bus, a control bus, and a status bus, among others. For purposes of illustration, the bus system 440 is shown in the illustrated embodiment as a single bus system; however, it should be understood that the bus system 440 can include a combination of multiple buses that function together as a bus system. Figure 6 Various buses are shown as the bus system 440 in

[0102] Processor 410 can be an integrated circuit chip having signal processing capabilities, such as a general purpose processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like. The general purpose processor can be a microprocessor, or any conventional processor, or the like.

[0103] User interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. User interface 430 also includes one or more input devices 432 that facilitate user input, such as user interface components that facilitate input of a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls, and the like.

[0104] Memory 450 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, and the like. Memory 450 optionally includes one or more storage devices remotely located from the processor 410.

[0105] Memory 450 includes volatile memory or nonvolatile memory, or both. Nonvolatile memory can be read only memory (ROM), programmable ROM (PROM), erasable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, or the like. Volatile memory can include random access memory (RAM), which acts as external cache memory. By way of illustration, and

[0106] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or superset thereof, which are described in the example below.

[0107] Operating system 451 includes systems programming that provides services for other programs, such as a framework layer, a core library layer, a driver layer, and the like, to implement various basic system services and perform hardware-dependent tasks, such as power management, memory management, storage management, and the like.

[0108] a network communication module 452 for communicating to other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, Wireless Fidelity (WiFi), and Universal Serial Bus (USB), etc.;

[0109] a presentation module 453 for enabling presentation of information via one or more output devices 431 associated with the user interface 430 (e.g., a display screen, a speaker, etc.) (e.g., a user interface for operating the peripheral device and displaying content and information);

[0110] an input processing module 454 for detecting and interpreting one or more user inputs or interactions from one or more input devices 432.

[0111] In some embodiments, the component recognition apparatus based on image recognition model provided by the embodiments of the present application can be implemented in a software manner, Figure 6 A component recognition apparatus based on image recognition model 455 stored in the memory 450 is shown, which can be software in the form of programs and plug-ins, etc., including the following software modules: a first feature extraction module 4551, a first feature fusion module 4552, a second feature extraction module 4553, a second feature fusion module 4554, and a recognition module 4555. These modules are logical, and thus can be combined or further split according to the implemented functions.

[0112] In some other embodiments, the component recognition apparatus based on image recognition model provided by the embodiments of the present application can be implemented in a hardware manner. As an example, the component recognition apparatus based on image recognition model provided by the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the component recognition method based on image recognition model provided by the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can use one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic elements.

[0113] In some embodiments, the terminal or the server can implement the component recognition method based on the image recognition model provided in the embodiments of the present application by running a computer program. For example, the computer program can be a native program or a software module in an operating system; can be a native application (APP), i.e., a program that needs to be installed in an operating system to run, such as an instant messaging APP, a web browser APP; can also be a mini-program, i.e., a program that only needs to be downloaded into a browser environment to run; and can also be a mini-program that can be embedded into any APP. In summary, the above computer program can be any form of application program, module or plug-in.

[0114] Based on the above description of the component recognition system and the electronic device based on the image recognition model provided in the embodiments of the present application, the component recognition method based on the image recognition model provided in the embodiments of the present application is described below. In actual implementation, the component recognition method based on the image recognition model provided in the embodiments of the present application can be implemented by a terminal or a server alone, or by a terminal and a server cooperatively, to execute the component recognition method based on the image recognition model provided in the embodiments of the present application by the terminal alone or by the server alone. Figure 5 The component recognition method based on the image recognition model provided in the embodiments of the present application is described below by taking the case where the server 200 in the terminal 100 and the server 200 in the server 200 alone executes the component recognition method based on the image recognition model provided in the embodiments of the present application. Referring to FIG. 1, Figure 7 Figure 7 is a flowchart of the component recognition method based on the image recognition model provided in the embodiments of the present application. It should be noted that the image recognition model includes a first feature extraction layer, a second feature extraction layer, a first feature fusion layer, a second feature fusion layer and a component recognition layer, referring to Figure 8 Figure 8 is a structural diagram of the image recognition model provided in the embodiments of the present application, which is described in combination with Figure 7 and Figure 8 .

[0115] In step 101, the server performs image feature extraction on the to-be-recognized image including the loading tool by using the first feature extraction layer, to obtain a global feature map corresponding to the loading tool, and performs probability feature extraction on the to-be-recognized image, to obtain a probability feature map used to indicate a connection probability between at least two components.

[0116] The loading tool includes at least two components, and the at least two components have a fixed component connection relationship.

[0117] In actual implementation, the to-be-recognized image including the loading tool can be pre-stored locally, can be obtained from the outside world (such as the Internet), or can be an image obtained by image acquisition on the loading tool by using an image acquisition device.

[0118] ​​As an example, the loading tool can be a box structure such as a container and the like, and the image acquisition device can be a camera. The camera can be a monocular camera, a binocular camera, a depth camera, a three-dimensional (3D) camera, and the like. In actual implementation, the camera is called to start a scanning mode, to scan the loading tool in the field of view of the camera in real time, and to generate images at a specified frame rate, as shown in Figure 9 Figure 9 is a schematic diagram of an image to be recognized including a loading tool acquired by the camera according to an embodiment of the present application.

[0119] As another example, the image acquisition device can also be a radar device such as a laser radar, a millimeter wave radar, and the like. The laser radar is a radar device that detects position, speed, attitude, shape, and the like of the loading tool by emitting a laser beam. The millimeter wave radar is a radar device that detects in a millimeter wave band. The radar device emits a detection signal to the loading tool in real time, receives a return signal reflected by the loading tool, and determines the characteristic data of the target object based on the difference between the detection signal and the return signal. The radar device uses multiple transmitters and receivers, and the image obtained thereby is a three-dimensional point cloud image.

[0120] It should be noted that the image acquisition device can be located at the mobile terminal, so that image acquisition and corresponding component recognition are performed based on the mobile terminal device. In this way, the flexibility and low cost of the mobile terminal device can greatly improve the usability and convenience of recognizing components of the loading tool.

[0121] In actual implementation, after obtaining the image to be recognized including the loading tool, referring to Figure 10 Figure 10 is a flowchart of determining a global feature map and a probability feature map corresponding to the loading tool based on a first feature extraction layer according to an embodiment of the present application. Here, the first feature extraction layer includes a global feature extraction layer and a probability feature extraction layer, referring to Figure 11 Figure 11 is a structural diagram of the first feature extraction layer according to an embodiment of the present application, based on Figure 10 and Figure 11 Step 101 can also be implemented in the following manner:

[0122] In step 1011, the global feature extraction layer is used to perform image feature extraction on the image to be recognized including the loading tool, to obtain a global feature map corresponding to the loading tool.

[0123] ​​​In actual implementation, since the areas of the left and right plates, the floor and other components of the loading tool are large, the areas account for a large proportion in the to-be-recognized image, and in order to improve the recognition accuracy of such components and reduce the block recognition error, the image feature extraction is performed on the to-be-recognized image including the loading tool through the global feature extraction layer to obtain the global feature map corresponding to the loading tool, refer to Figure 12 , Figure 12 is a structural schematic diagram of the global feature extraction layer provided in the embodiments of the present application, based on Figure 12 , the process of performing the image feature extraction on the to-be-recognized image including the loading tool through the global feature extraction layer to obtain the global feature map corresponding to the loading tool, specifically includes: performing the image feature extraction on the to-be-recognized image including the loading tool through the global feature extraction layer to obtain at least two global sub-feature maps; wherein the sizes of the global sub-feature maps are different; and then performing feature splicing on the global sub-feature maps to obtain the global feature map corresponding to the loading tool.

[0124] As an example, the size of the to-be-recognized image is H*W*C, wherein H, W and C are respectively the values of length, width and channel, and after the image feature extraction of the to-be-recognized image through the global feature extraction layer, a global sub-feature map with a size of 1*1*C and a global sub-feature map with a size of 2*2*C are generated, and then the two generated global sub-feature maps are spliced through to obtain a global feature map with a size of H*W*2C.

[0125] In this way, for the components with large areas, in order to reduce the misjudgment of other blocks in the component recognition area, the global feature map with multiple scales is obtained, so as to improve the receptive field of such components in the process of recognizing the corresponding components and reduce the interference of related noise.

[0126] In step 1012, the probability feature extraction is performed on the to-be-recognized image through the probability feature extraction layer to obtain the probability feature map used for indicating the connection probability between the at least two components.

[0127] In actual implementation, since the loading tool is a fixedly connected structural tool, the components have fixed connection results, such as that the door handle is only connected to the door lock rod and the adhesive tape is only on the door, and the like. Therefore, the connection relationship between different two components is different, so that the correlation between any two components can be determined to recognize the components, specifically refer to Figure 13 , Figure 13 is a structural schematic diagram of the probability feature extraction layer provided in the embodiments of the present application, based on Figure 13The probabilistic feature extraction layer extracts features from the image to be identified, obtaining component feature maps of each component and connectivity feature maps indicating the connection relationship between at least two components. Feature fusion is performed on the component feature maps and connectivity feature maps of each component to obtain a probabilistic feature map indicating the connection probability between at least two components.

[0128] It should be noted that after determining the component feature map of each component and the connectivity feature map used to indicate the connection relationship between at least two components, it is necessary to determine the connection relationship between each individual component and other components, that is, the probability of connection between the component and other components. Specifically, for each component, the component feature map and the connectivity feature map of the component are fused to obtain the intermediate probability feature map of the component; wherein, the intermediate probability feature map is used to indicate the connection probability between the component and any other component other than the component itself; the intermediate probability feature maps of each component are spliced ​​to obtain the probability feature map used to indicate the connection probability between at least two components.

[0129] In practical implementation, the process of determining the intermediate probability feature map of each component specifically includes: first, determining at least one inner product of the corresponding component based on the component feature map and the component connectivity feature map; determining the correlation between the component and other components based on the at least one inner product; summing the correlations to obtain the sum of correlations corresponding to the component; determining the ratio of each correlation to the sum of correlations; and determining the intermediate probability feature map of the component based on at least one ratio. For example, assuming the size of the image to be recognized is H*W*C, for the same channel, the set of components in the image to be recognized is X = {x1, x2, ..., x...} M}, where M = H * W, and the component is X. t The component feature diagram is Vx t The connectivity characteristic of the component is Ux c Then the intermediate probability feature map of this component is:

[0130]

[0131] Where S is at least one inner product corresponding to the component, i.e. and This refers to the degree of correlation between this component and other components.

[0132] Since a container is a relatively regular hexahedron with specific components on each face and relatively fixed positional information between them, based on this fixed connection structure, by calculating the probability of possible connections between adjacent components, the correlation between components can be better learned, thereby reducing false detections of non-connected components and obtaining more stable component identification results.

[0133] In step 102, the global feature map and the probability feature map are fused by the first feature fusion layer to obtain an intermediate feature map.

[0134] Referring to Figure 14 , Figure 14 is a structural schematic diagram of the first feature fusion layer provided by the embodiment of the present application, based on Figure 14 After obtaining the global feature map and the probability feature map, the global feature map and the probability feature map are multiplied by the first feature fusion layer to obtain a product feature map; and the product feature map is taken as the intermediate feature map.

[0135] In this way, after obtaining the global feature map and the probability feature map, the global feature map and the probability feature map are fused to identify the corresponding components, so that the fixed connection structure information between the components is better utilized, the misidentification of the non-associated blocks is reduced, and finally the accuracy of component identification is improved.

[0136] It should be noted that when the loading tool only includes one component, since there is no other component, the connection probability between the component and other components is 0, that is, the intermediate probability feature map for indicating the connection probability between the component and any one of the other components is 0, that is, the probability feature map of the probability feature map for indicating the connection probability between at least two components is 0, so that the global feature map of the component is obtained by fusing the global feature map and the probability feature map by the first feature fusion layer.

[0137] In step 103, the intermediate feature map is extracted by the second feature extraction layer to obtain the edge feature map of each component, the main body feature map of each component, and the background feature map of the loading tool.

[0138] In actual implementation, the second feature extraction layer includes a loading tool feature extraction layer, a component main body feature extraction layer, and a component edge feature extraction layer, referring to Figure 15 , Figure 15 is a structural schematic diagram of the second feature extraction layer provided by the embodiment of the present application, based on Figure 15 The process of extracting the intermediate feature map by the second feature extraction layer to obtain the edge feature map of each component, the main body feature map of each component, and the background feature map of the loading tool specifically includes: extracting the intermediate feature map by the component main body feature extraction layer to obtain the main body feature map of each component; extracting the intermediate feature map by the component edge feature extraction layer to obtain the edge feature map of each component; and extracting the intermediate feature map by the loading tool feature extraction layer to obtain the background feature map of the loading tool.

[0139] It should be noted that, due to the large variety of components in loading tools, reaching hundreds, and considering factors such as the probability of component damage and the similarity of visual features, a component main feature extraction layer is used to perform pixel-level fine-grained identification of components on the loading tool, such as the top plate, door panel, left and right panels, front panel, door lock rod, main bracket, cam head, hinge plate, door rubber strip, front bottom beam, side bottom beam, bottom beam, door crossbeam, and corner pieces. For example, see [link to relevant documentation]. Figure 16 , Figure 16 This is a comparative schematic diagram of the main feature maps of each component provided in the embodiments of this application and the image to be identified, based on... Figure 16 , Figure 16 The image on the left is the acquired image to be identified, and the image on the right is the main feature map of the component in the corresponding image to be identified.

[0140] It should be noted that, for loading tools, in addition to large components such as corrugated panels and flooring, there are also delicate and small components such as door handles and hinge plates. These components often occupy a small proportion in the image and their boundary resolution is not clear enough. Therefore, an edge feature extraction layer is used to extract the edge features of delicate components on the loading tool, such as door handles, large brackets, small brackets, air vents, and rubber strips, to obtain the corresponding edge feature maps, thereby improving the accuracy of component boundary recognition. For example, see [link to relevant documentation]. Figure 17 , Figure 17 This is a schematic diagram comparing the edge feature maps of each component provided in the embodiments of this application with the image to be identified, based on... Figure 17 , Figure 17 The top center shows the acquired image to be identified, and the bottom center shows the edge feature map of the corresponding component in the image to be identified.

[0141] It should be noted that when acquiring images of the loading vehicle to be identified using acquisition devices, many complex backgrounds are included, such as the sky, ground, and other loading vehicles. Therefore, in order to identify only the components on the loading vehicle to be examined in the image and eliminate interference from other loading vehicles in the background, a binary classification of the loading vehicle and the background is performed. That is, through a loading vehicle feature extraction layer, features are extracted from the intermediate feature map to obtain the background feature map of the loading vehicle. For example, see [link to relevant documentation]. Figure 18 , Figure 18 This is a schematic diagram comparing the background feature map of the loading tool provided in this application embodiment with the image to be identified, based on... Figure 18 , Figure 18 The image on the left is the acquired image to be identified, and the image on the right is the background feature map of the loading tool in the corresponding image to be identified. Here, the black area is the background area relative to the loading tool to be identified, and the gray area is the main body area of ​​the loading tool to be identified.

[0142] Therefore, the three-branch feature extraction structure is designed in combination with the complex background and the fine components, further promotes the identification effect of the main body region and the edge region of the components, reduces the interference of the background factors and other loading tools, and improves the accuracy of component identification of the corresponding loading tool.

[0143] In step 104, the edge feature map, the main body feature map and the background feature map are fused by the second feature fusion layer to obtain a target feature map.

[0144] Referring to Figure 19 , Figure 19 is a structural schematic diagram of the second feature fusion layer provided by the embodiment of the present application, based on Figure 19 After determining the edge feature map of each component, the main body feature map of each component and the background feature map of the loading tool, the edge feature map, the main body feature map and the background feature map are fused by the second feature fusion layer to obtain a target feature map.

[0145] In step 105, based on the target feature map, each component included in the loading tool in the to-be-identified image is identified by a component identification layer to obtain a corresponding component identification result.

[0146] In actual implementation, after obtaining the target feature map, the target feature map can be processed by the component identification layer to obtain a corresponding dimension-reduced target feature map, and each component included in the loading tool in the to-be-identified image is identified based on the dimension-reduced target feature map to obtain a corresponding component identification result. Here, the presentation form of the component identification result can be a picture, a text or the like, and the embodiment of the present application does not limit this. It should be noted that the dimension reduction processing here can be processing by global average pooling of the target feature map, so as to convert the multi-dimensional target feature map into a one-dimensional target feature map.

[0147] In some embodiments, after determining the corresponding component identification, the identified components can be subjected to defect detection based on the component identification result to obtain a detection result; when the detection result indicates that a target component in at least two components has a defect, a defect prompt information corresponding to the target component is output.

[0148] It should be noted that the defect prompt information here can be used to remind the user to repair the corresponding component, or can be used to remind the user to replace the corresponding component. Specifically, based on the detection result, the defect existing in the target component of the at least two components is scored to obtain a scoring result; the scoring result is compared with a pre-set score threshold to obtain a comparison result; when the comparison result represents that the scoring result is greater than or equal to the score threshold, the prompt information for reminding the user to repair the corresponding component is output; when the comparison result represents that the scoring result is less than the score threshold, the prompt information for reminding the user to replace the corresponding component is output. It should be noted that the scoring result can also be used as the defect prompt information, so that the scoring result is directly output to enable the user to determine whether to repair or replace the corresponding component based on the output scoring result.

[0149] In some embodiments, before component identification based on the image recognition model, the image recognition model is first trained, see Figure 20 , Figure 20 is a flowchart of a training method of an image recognition model provided by an embodiment of the present application. Next, the steps shown in Figure 20 will be described.

[0150] Step 201, the server extracts image features of the image training sample carrying a label and including a loading tool through a first feature extraction layer to obtain a global feature map corresponding to the loading tool, and extracts probability features of the image training sample to obtain a probability feature map indicating a connection probability between the at least two components.

[0151] The loading tool includes at least two components, and the at least two components have a fixed component connection relationship, and the label is used to indicate the components actually existing on the loading tool included in the image training sample.

[0152] Step 202, the global feature map and the probability feature map are fused through a first feature fusion layer to obtain an intermediate feature map.

[0153] Step 203, the intermediate feature map is feature-extracted through a second feature extraction layer to obtain an edge feature map of each component, a main body feature map of each component, and a background feature map of the loading tool.

[0154] Step 204, the edge feature map, the main body feature map, and the background feature map are fused through a second feature fusion layer to obtain a target feature map.

[0155] Step 205, based on the target feature map, the components included in the loading tool in the image training sample are identified through a component identification layer to obtain a corresponding component identification result.

[0156] In step 206, the difference between the component recognition result and the label is obtained, and the image recognition model is trained based on the difference.

[0157] The image recognition model obtained by training is used to recognize each component of the target loading tool in a to-be-recognized image including the target loading tool, and the target loading tool has the same component structure as the loading tool.

[0158] In actual implementation, first, a loss function corresponding to the classification model, such as a cross-entropy function, is obtained, then the value of the loss function is determined based on the difference between the obtained recognition result and the label, and the model parameters of the classification model are updated based on the value of the loss function.

[0159] For example, the cross-entropy function here can be

[0160]

[0161] Wherein, Loss represents the cross-entropy loss function, R represents the total number of samples in the image training sample, r represents the serial number of the sample in the image training sample, z (r) represents the label corresponding to the rth sample in the image training sample, s (r) represents the probability of recognizing the component of the rth sample in the image training sample input into the model.

[0162] By applying the above embodiments of the present application, first, feature extraction is performed on the to-be-recognized image to obtain a global feature map of the loading tool and a probability feature map of the components on the loading tool, then feature extraction is performed on the feature map obtained by fusing the global feature map and the probability feature map to obtain an edge feature map of each component, a main body feature map, and a background feature map of the loading tool, and component recognition is performed based on the feature map obtained by fusing the edge feature map of each component, the main body feature map, and the background feature map of the loading tool to obtain a component recognition result. In this way, through two feature extraction processes, multi-dimensional features corresponding to the loading tool are obtained, so as to identify the components on the loading tool according to the multi-dimensional features, which can improve the comprehensiveness of the obtained features, thereby improving the identification efficiency and accuracy of the components on the loading tool.

[0163] Next, the component recognition method based on the image recognition model provided by the embodiments of the present application will be introduced, Figure 21 is a flowchart of the component recognition method based on the image recognition model provided by the embodiments of the present application, referring to Figure 21 The component recognition method based on the image recognition model provided by the embodiments of the present application is implemented by the client and the server in cooperation.

[0164] In step 301, the client obtains a plurality of image training samples including loading tools and corresponding labels in response to an upload operation for the image training samples.

[0165] Here, the client can be an identification client arranged on the terminal to identify the components of the loading tool, and the image training sample can be uploaded by the user based on a human-computer interaction interface of the client, triggering an upload function item in the human-computer interaction interface to cause the client to present an image selection interface in the human-computer interaction interface, and the user uploads the image training sample from the terminal based on the image selection interface, so that the client obtains the uploaded image training sample.

[0166] In some embodiments, the image training sample can also be obtained by a camera in communication connection with the terminal, and after the camera obtains the image training sample, the camera transmits the image training sample to the terminal and the terminal automatically uploads the image training sample to the client.

[0167] Step 302, the client sends the image training sample and the corresponding label to the server.

[0168] Step 303, the server inputs the received image training sample into the image recognition model.

[0169] Step 304, outputting the component identification result for the image training sample.

[0170] Step 305, obtaining the difference between the component identification result and the label, and training the image recognition model based on the difference.

[0171] In actual implementation, the server iterates the above training process until the loss function converges, and completes the training of the image recognition model.

[0172] Step 306, the server generates a prompt message that the image recognition model training is completed.

[0173] Step 307, the server sends the prompt message to the client.

[0174] Step 308, the client obtains the to-be-identified image in response to an upload operation for the to-be-identified image including the target loading tool.

[0175] It should be noted that the to-be-identified image can also be sent to the client by other devices in communication connection with the terminal; the target loading tool and the loading tool have the same component structure.

[0176] Step 309, the client sends the to-be-identified image to the server in response to an identification instruction for the to-be-identified image.

[0177] In actual implementation, the identification instruction of the to-be-identified image can be automatically generated by the client under certain trigger conditions, for example, the client automatically generates the identification instruction for the to-be-identified image after obtaining the to-be-identified image, can be sent to the client by other devices in communication connection with the terminal, and can be generated by the user based on the man-machine interaction interface of the client after triggering the corresponding component identification function item.

[0178] In step 310, the server inputs the received to-be-identified image into the image recognition model to enable the image recognition model to perform component identification on the to-be-identified image, and obtain a target component identification result of a component on a target loading tool included in the to-be-identified image.

[0179] In step 311, the server sends the target component identification result to the client.

[0180] In step 312, the client outputs the received target component identification result.

[0181] In actual implementation, the client can present the component identification result for the to-be-identified image in the man-machine interaction interface of the client, can save the component identification result to the terminal locally, and can send the component identification result to other devices in communication connection with the terminal.

[0182] According to the above embodiments of the present application, firstly, feature extraction is performed on the to-be-identified image to obtain a global feature map of the loading tool and a probability feature map of the component on the loading tool, and then feature extraction is performed on the feature map obtained by fusing the global feature map and the probability feature map to obtain an edge feature map of each component, a main body feature map, and a background feature map of the loading tool, so as to perform component identification based on the feature map obtained by fusing the edge feature map of each component, the main body feature map, and the background feature map of the loading tool, and obtain a component identification result. In this way, through two feature extraction processes, multi-dimensional features corresponding to the loading tool are obtained, so as to identify the component on the loading tool according to the multi-dimensional features, which can improve the comprehensiveness of the obtained features, thereby improving the identification efficiency and identification accuracy of the component on the loading tool.

[0183] In the following, an exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0184] A container is a widely used standardized transport packaging or a component tool of non-packaging goods. The arrangement, statistics, quality detection and defect detection of the container are very important links. Intelligent implementation of these operations on the container can greatly improve the efficiency and reduce the labor cost. However, in the related art, in the detection of the quality of the container, only a traditional image processing method is used to simply segment the multiple containers in the collected container image into multiple single container images. Specifically, an image threshold is calculated according to the received scanning image of the container; the scanning image is binarized according to the image threshold to obtain a binarized image; in the case that the image contour of the binarized image has a segmentation point, a segmentation operation is performed on the scanning image according to the segmentation point to obtain multiple single container images. However, such recognition effect is rough, and the components of the container cannot be effectively extracted and further component information cannot be provided.

[0185] Therefore, the embodiment of the present application provides a component recognition method and device based on an image recognition model, an electronic device, a computer readable storage medium and a computer program product, so as to realize the intelligent container inspection of the container, collect the container picture to be inspected by the handheld mobile device, and combine the deep learning image segmentation technology in the computer vision field and the traditional image processing technology to finely recognize the components of the container at the pixel level, which can greatly improve the flexibility of the container inspection, reduce the imaging cost, and is easy to promote, and provides a solid foundation for the defect detection of the subsequent components. Specifically, referring to Figure 22 , Figure 22 is a flowchart of the component recognition method based on the image recognition model provided by the embodiment of the present application, based on Figure 22 , the component recognition method based on the image recognition model is realized by steps 401 to 406, specifically, first, input the input picture (to-be-recognized image) into the skeleton network to obtain the input feature, then obtain the image feature (global feature map) based on the global feature extraction module (global feature extraction layer), and obtain the target feature (component feature) and the connectivity feature (connectivity feature map) based on the attention mechanism module (probability feature extraction layer), so as to obtain the connectivity probability (probability feature map) based on the target feature and the connectivity feature, then the image feature and the connectivity probability are spliced to obtain the spliced feature (intermediate feature map), so as to obtain the feature of the corresponding component edge region (edge feature map), the feature of the corresponding component main body region (main body feature map), and the feature of the corresponding container main body and background region (background feature map) based on the three-branch network output structure (second feature extraction layer), and finally, based on the feature of the corresponding component edge region, the feature of the corresponding component main body region, and the feature of the corresponding container main body and background region, the final recognition image (component recognition result) is obtained.

[0186] For the attention mechanism module, the target feature and the connectivity feature are obtained, and the connectivity probability is obtained based on the target feature and the connectivity feature. For details, refer to Figure 23 , Figure 23 is a processing schematic diagram of the attention mechanism module provided in the embodiment of the present application, based on Figure 23 After obtaining the target feature and the connectivity feature, the target feature and the possible connectivity feature are multiplied by each other to obtain the connectivity probability between each other, that is, the correlation between the target feature and other features is calculated by the above formula (1) to measure the connection possibility between components.

[0187] For the global feature extraction module, the image feature is obtained, and the splicing feature is obtained based on the image feature and the connectivity probability. For details, refer to Figure 23 , based on Figure 23 After obtaining the image feature and the connectivity probability, the image feature and the connectivity probability are multiplied to obtain the splicing feature, so as to improve the relevance between the features with connectivity and reduce the size of the non-relevant features, and finally output the splicing feature.

[0188] It should be noted that, since the areas of the left and right plates, floor and other components of the container are relatively large, the proportion in the picture is large, in order to improve the segmentation accuracy of such components and reduce the block recognition error, therefore, a pooling module for outputting different size feature maps needs to be present in the global feature extraction module. For example, refer to Figure 24 , Figure 24 is a processing schematic diagram of the global feature extraction module provided in the embodiment of the present application, based on Figure 24 In the global feature extraction module, there are two pooling layers of different scales. Based on the input feature map with a size of H*W*C, the two pooling layers generate feature maps with sizes of 1*1*C and 2*2*C respectively, then the two feature maps are respectively convolved and up-sampled to the original feature map size H*W*C, and finally the two feature maps are spliced in the channel layer to obtain a global feature map with a size of H*W*2C, that is, the image feature. Here, H, W and C are respectively the values of length, width and channel.

[0189] Thus, since the container is a relatively regular hexahedron, each side has specific components, and the positional information between them is relatively fixed, based on the fixed connection structure, the application designs an attention mechanism module according to the positional connection relationship between the components to calculate the connection probability of the possible connection between adjacent components; and for the components with larger area, in order to reduce the false detection of other blocks in the component recognition area, the application designs a global feature extraction module to calculate the overall image features to improve the receptive field of such components and reduce the interference of related noise; at the same time, after obtaining the connection probability and the overall image features, the connection probability is weighted to the overall image features, so as to better utilize the fixed connection structure information between the components and help improve the accuracy of component segmentation.

[0190] For the three-branch network output structure, in order to better identify various components of the container to be tested, eliminate the interference of the container components in the background, and improve the identification effect of components with smaller area and more delicate structure, a three-branch network output structure is designed, as shown in Figure 25 , Figure 25 is a processing schematic diagram of the three-branch network output structure provided by the embodiment of the application, based on Figure 25 For the features corresponding to the component main area, since the types of components of the loading tool are various, reaching hundreds, and considering factors such as the probability of component damage and the similarity of visual features, pixel-level fine identification is performed on components on the loading tool such as the top plate, the door plate, the left and right plates, the front plate, the door lock rod, the large bracket, the cam head, the hinge plate, the door rubber strip, the front bottom beam, the side bottom beam, the bottom beam, the door upper beam, and the corner piece, as shown in Figure 16 , Figure 16 The left side in the figure is the input image collected, and the right side is the feature map of the corresponding component main area in the corresponding input image.

[0191] Based on Figure 25 For the features corresponding to the component edge area, for the container, in addition to large components such as corrugated plates and floors, there are also delicate and small components such as door handles and hinge plates. Such components often account for a small proportion in the image, and the boundary resolution is not clear enough. Therefore, edge segmentation is performed on delicate components on the container such as door handles, large brackets, small brackets, air windows, and rubber strips to obtain the segmentation result of the edge line of the corresponding component, so as to improve the fine degree of the boundary of the identified component, as shown in Figure 17 , Figure 17 The upper side in the figure is the input image collected, and the lower side is the feature map of the corresponding component edge area in the corresponding input image.

[0192] Based on Figure 25For the features of the corresponding container body and the background area, since when the handheld device collects the container picture, it will include many complex backgrounds such as the sky, the ground and other containers, etc., in order to only identify the parts on the container in the picture and exclude the interference of other containers in the background, the container and the background are segmented by two-classification, for example, see Figure 18 , Figure 18 The left side of the container picture collected, and the right side is the feature map of the corresponding container body and the background area in the corresponding container picture. Here, the black area is the background area of the container to be inspected, and the gray area is the main body area of the container to be inspected.

[0193] In actual implementation, after obtaining the features of the corresponding part edge area, the features of the corresponding part main body area, and the features of the corresponding container body and the background area, the features of the corresponding part edge area, the features of the corresponding part main body area, and the features of the corresponding container body and the background area are fused to obtain the final identification image.

[0194] It should be noted that since the containers in the background also contain left and right corrugated plates, beams, corner pieces, and column parts, in order to improve the labeling efficiency and reduce the recognition interference of these parts, in the training process of the corresponding model, all pixels in the background area are not involved in the loss calculation and gradient backpropagation of the branch related to part recognition. The background area in this branch is given by the branch corresponding to the container body and the background area.

[0195] Therefore, in the container picture taken by the handheld device, the average accuracy rate of the region positioning of each part of the container reaches 89%; at the same time, for the identification result, it provides a prerequisite for defect detection and reporting of different parts in subsequent tasks; at the same time, the flexibility and low cost of mobile phone shooting greatly improve the usability and convenience of the technical solution.

[0196] By applying the above embodiments of the present application, first, the features of the to-be-identified image are extracted to obtain the global feature map of the loading tool and the probability feature map of the parts on the loading tool, and then the global feature map and the probability feature map are fused to obtain the feature map for feature extraction, to obtain the edge feature map of each part, the main body feature map and the background feature map of the loading tool, so as to perform part recognition based on the feature map obtained by fusing the edge feature map of each part, the main body feature map and the background feature map of the loading tool, to obtain the part recognition result. In this way, through two feature extraction processes, the multi-dimensional features corresponding to the loading tool are obtained, so as to identify the parts on the loading tool according to the multi-dimensional features, which can improve the comprehensiveness of the obtained features, thereby improving the identification efficiency and accuracy of the parts on the loading tool.

[0197] The following continues to illustrate an exemplary structure of the embodiment of the component recognition device 455 based on the image recognition model provided by the present application, wherein the image recognition model comprises a first feature extraction layer, a second feature extraction layer, a first feature fusion layer, a second feature fusion layer, and a component recognition layer. In some embodiments, as shown in the figure, the software module stored in the component recognition device 455 based on the image recognition model in the memory 450 can include: Figure 6

[0198] The first feature extraction module 4551 is configured to perform image feature extraction on the to-be-recognized image including the loading tool through the first feature extraction layer to obtain a global feature map corresponding to the loading tool, and perform probability feature extraction on the to-be-recognized image to obtain a probability feature map indicating the connection probability between at least two components; wherein the loading tool includes at least two components, and the at least two components have a fixed component connection relationship.

[0199] The first feature fusion module 4552 is configured to perform feature fusion on the global feature map and the probability feature map through the first feature fusion layer to obtain an intermediate feature map.

[0200] The second feature extraction module 4553 is configured to perform feature extraction on the intermediate feature map through the second feature extraction layer to obtain an edge feature map of each component, a main body feature map of each component, and a background feature map of the loading tool.

[0201] The second feature fusion module 4554 is configured to perform feature fusion on the edge feature map, the main body feature map, and the background feature map through the second feature fusion layer to obtain a target feature map.

[0202] The recognition module 4555 is configured to perform recognition on each component included in the loading tool in the to-be-recognized image through the component recognition layer based on the target feature map to obtain a corresponding component recognition result.

[0203] In some embodiments, the first feature extraction layer includes a global feature extraction layer and a probability feature extraction layer; and the first feature extraction module 4551 is further configured to perform image feature extraction on the to-be-recognized image including the loading tool through the global feature extraction layer to obtain a global feature map corresponding to the loading tool; and perform probability feature extraction on the to-be-recognized image through the probability feature extraction layer to obtain a probability feature map indicating the connection probability between at least two components.

[0204] ​In some embodiments, the first feature extraction module 4551 is further configured to perform image feature extraction on the to-be-identified image including the loading tool by the global feature extraction layer to obtain at least two global sub-feature maps; wherein each of the global sub-feature maps has different sizes; and perform feature splicing on each of the global sub-feature maps to obtain a global feature map corresponding to the loading tool.

[0205] In some embodiments, the first feature extraction module 4551 is further configured to perform feature extraction on the to-be-identified image by the probability feature extraction layer to obtain a component feature map of each of the components and a connectivity feature map indicating the connection relationship between the at least two components; and perform feature fusion on the component feature map of each of the components and the connectivity feature map to obtain a probability feature map indicating the connection probability between the at least two components.

[0206] In some embodiments, the first feature extraction module 4551 is further configured to, for each of the components, perform feature fusion on the component feature map of the component and the connectivity feature map of the component to obtain an intermediate probability feature map of the component; wherein the intermediate probability feature map is used to indicate the connection probability between the component and any one of the other components except the component; and perform feature splicing on the intermediate probability feature map of each of the components to obtain the probability feature map indicating the connection probability between the at least two components.

[0207] In some embodiments, the first feature fusion module 4552 is further configured to perform product processing on the global feature map and the probability feature map by the first feature fusion layer to obtain a product feature map; and use the product feature map as the intermediate feature map.

[0208] In some embodiments, the second feature extraction layer includes a loading tool feature extraction layer, a component main body feature extraction layer, and a component edge feature extraction layer; and the second feature extraction module 4553 is further configured to perform feature extraction on the intermediate feature map by the component main body feature extraction layer to obtain a main body feature map of each of the components; perform feature extraction on the intermediate feature map by the component edge feature extraction layer to obtain an edge feature map of each of the components; and perform feature extraction on the intermediate feature map by the loading tool feature extraction layer to obtain a background feature map of the loading tool.

[0209] In some embodiments, the apparatus further includes an image acquisition module configured to acquire an image obtained by image acquisition of the loading tool by an image acquisition device as the to-be-identified image.

[0210] In some embodiments, the device further comprises a detection module configured to perform defect detection on each identified component based on the component identification result to obtain a detection result; and when the detection result indicates that a target component among the at least two components has a defect, output a defect prompt information corresponding to the target component.

[0211] In the following, an exemplary structure of the implementation of the training device 2600 of the image training model provided in the embodiments of the present application as a software module is described. In some embodiments, referring to Figure 26 , Figure 26 is a structural schematic diagram of the training device 2600 of the image training model provided in the embodiments of the present application, as Figure 26 indicated, the software module in the training device 2600 of the image training model stored in the memory 450 can include:

[0212] The first feature extraction module 2610 is configured to perform image feature extraction on the image training sample carrying a label and including a loading tool through the first feature extraction layer to obtain a global feature map corresponding to the loading tool, and perform probability feature extraction on the image training sample to obtain a probability feature map indicating a connection probability between at least two components; wherein the loading tool includes the at least two components, and the at least two components have a fixed component connection relationship.

[0213] The first feature fusion module 2620 is configured to perform feature fusion on the global feature map and the probability feature map through the first feature fusion layer to obtain an intermediate feature map.

[0214] The second feature extraction module 2630 is configured to perform feature extraction on the intermediate feature map through the second feature extraction layer to obtain an edge feature map of each component, a main body feature map of each component, and a background feature map of the loading tool.

[0215] The second feature fusion module 2640 is configured to perform feature fusion on the edge feature map, the main body feature map, and the background feature map through the second feature fusion layer to obtain a target feature map.

[0216] The identification module 2650 is configured to identify each component included in the loading tool in the image training sample based on the target feature map through the component identification layer to obtain a corresponding component identification result.

[0217] The training module 2660 obtains a difference between the component recognition result and the label, and trains the image recognition model based on the difference; the image recognition model trained by the training is used for recognizing each component of the target loading tool in a to-be-recognized image including the target loading tool, and the target loading tool has the same component structure as the loading tool.

[0218] The embodiment of the present application provides a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the component recognition method based on an image recognition model provided in the embodiment of the present application.

[0219] The embodiment of the present application provides a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the component recognition method based on an image recognition model provided in the embodiment of the present application.

[0220] The embodiment of the present application provides a computer readable storage medium storing executable instructions, wherein the computer readable storage medium stores executable instructions. When the executable instructions are executed by a processor, the processor executes the component recognition method based on an image recognition model provided in the embodiment of the present application, for example, as shown in the component recognition method based on an image recognition model. Figure 3 The embodiment of the present application provides a computer readable storage medium storing executable instructions, wherein the computer readable storage medium stores executable instructions. When the executable instructions are executed by a processor, the processor executes the component recognition method based on an image recognition model provided in the embodiment of the present application, for example, as shown in the component recognition method based on an image recognition model.

[0221] The embodiment of the present application provides a computer readable storage medium storing executable instructions, wherein the computer readable storage medium stores executable instructions. When the executable instructions are executed by a processor, the processor executes the component recognition method based on an image recognition model provided in the embodiment of the present application, for example, as shown in the component recognition method based on an image recognition model.

[0222] In some embodiments, the computer readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or various devices including one or any combination of the above memories.

[0223] In some embodiments, the executable instructions can be in the form of a program, software, software module, script or code, written in any form of programming language (including a compiled or interpreted language, or a declarative or procedural language), and can be deployed in any form, including being deployed as a standalone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0224] By way of example, executable instructions can correspond to a file in a file system, but are not necessarily limited thereto. The executable instructions can be stored in part or in whole within a file that is stored in a file system, for example, one or more scripts stored in a Hyper Text Markup Language (HTML) document stored in a file system, one or more scripts stored in a single file dedicated to the program in question, or one or more scripts stored in multiple cooperating files, e.g., files storing one or more modules, sub programs, or code portions.

[0225] By way of example, executable instructions can be deployed to execute on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed among multiple sites and interconnected through a communication network.

[0226] In summary, through the embodiments of the present application, the following technical effects can be achieved:

[0227] (1) Through the two feature extraction processes, the multi-dimensional features corresponding to the loading tool are obtained, so as to identify the components on the loading tool according to the multi-dimensional features, which can improve the comprehensiveness of the obtained features, thereby improving the identification efficiency and identification accuracy of the components on the loading tool.

[0228] (2) For the components with a larger area, in order to reduce the false detection of other blocks in the component identification area, the multi-scale global feature map is obtained, thereby improving the receptive field of the components in the identification process of the corresponding components, and reducing the interference of related noise.

[0229] (3) Since the container is a relatively regular hexahedron, each face has specific components, and the positional information between them is relatively fixed. Therefore, based on the fixed connection structure, the connectivity probability of the possible connection between adjacent components is calculated, which can better learn the relevance between the components, thereby reducing the false detection of non-connected components and obtaining more stable component identification results.

[0230] (4) After obtaining the global feature map and the probability feature map, the global feature map and the probability feature map are fused to identify the corresponding components, thereby better utilizing the fixed connection structure information between the components, reducing the misidentification of non-associated blocks, and ultimately improving the accuracy of component identification.

[0231] (5) Combined with complex background and fine components, a three-branch feature extraction structure is designed, which further promotes the identification effect of the main region and the edge region of the components, reduces the interference of background factors and other loading tools, and improves the accuracy of component identification of the corresponding loading tool.

[0232] The above merely provides an example of the present application, but is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, and improvement made within the spirit and scope of the present application shall be included in the protection scope of the present application.

Claims

1. A component recognition method based on an image recognition model, characterized by, The image recognition model comprises a first feature extraction layer, a second feature extraction layer, a first feature fusion layer, a second feature fusion layer, and a component recognition layer, and the method comprises: Through the first feature extraction layer, image feature extraction is performed on a to-be-recognized image comprising a loading tool, a global feature map corresponding to the loading tool is obtained, and feature extraction is performed on the to-be-recognized image through the first feature extraction layer, component feature maps of each component and a connectivity feature map used to indicate a connection relationship between at least two components are obtained; feature fusion is performed on the component feature maps of each component and the connectivity feature map, and a probability feature map used to indicate a connection probability between at least two components is obtained; The loading tool comprises at least two components, and the at least two components have a fixed component connection relationship. Through the first feature fusion layer, feature fusion is performed on the global feature map and the probability feature map, and an intermediate feature map is obtained. Through the second feature extraction layer, feature extraction is performed on the intermediate feature map, and an edge feature map of each component, a main body feature map of each component, and a background feature map of the loading tool are obtained. Through the second feature fusion layer, feature fusion is performed on the edge feature map, the main body feature map, and the background feature map, and a target feature map is obtained. Based on the target feature map, through the component recognition layer, each component included in the loading tool in the to-be-recognized image is recognized, and a corresponding component recognition result is obtained.

2. The method of claim 1, wherein, The first feature extraction layer comprises a global feature extraction layer and a probability feature extraction layer, and the image feature extraction performed on the to-be-recognized image comprising the loading tool through the first feature extraction layer to obtain the global feature map corresponding to the loading tool comprises: Through the global feature extraction layer, image feature extraction is performed on the to-be-recognized image comprising the loading tool, and the global feature map corresponding to the loading tool is obtained.

3. The method of claim 2, wherein, The image feature extraction performed on the to-be-recognized image comprising the loading tool through the global feature extraction layer to obtain the global feature map corresponding to the loading tool comprises: Through the global feature extraction layer, image feature extraction is performed on the to-be-recognized image comprising the loading tool, and at least two global sub-feature maps are obtained; the sizes of the global sub-feature maps are different; Feature splicing is performed on the global sub-feature maps, and the global feature map corresponding to the loading tool is obtained.

4. The method of claim 1, wherein, The feature fusion performed on the component feature maps of each component and the connectivity feature map to obtain the probability feature map used to indicate the connection probability between at least two components comprises: For each component, feature fusion is performed on the component feature map of the component and the connectivity feature map of the component, and an intermediate probability feature map of the component is obtained; The intermediate probability feature map is used to indicate the connection probability between the component and any other component except the component; Feature splicing is performed on the intermediate probability feature maps of each component, and the probability feature map used to indicate the connection probability between at least two components is obtained.

5. The method of claim 1, wherein, The first feature fusion layer is used for performing feature fusion on the global feature map and the probability feature map to obtain an intermediate feature map. The first feature fusion layer is used for performing product processing on the global feature map and the probability feature map to obtain a product feature map. The product feature map is used as the intermediate feature map.

6. The method of claim 1, wherein, The second feature extraction layer includes a loading tool feature extraction layer, a component main body feature extraction layer and a component edge feature extraction layer. The second feature extraction layer is used for performing feature extraction on the intermediate feature map to obtain an edge feature map of each component, a main body feature map of each component and a background feature map of the loading tool. The component main body feature extraction layer is used for performing feature extraction on the intermediate feature map to obtain the main body feature map of each component. The component edge feature extraction layer is used for performing feature extraction on the intermediate feature map to obtain the edge feature map of each component. The loading tool feature extraction layer is used for performing feature extraction on the intermediate feature map to obtain the background feature map of the loading tool.

7. The method of claim 1, wherein, Before the image feature extraction on the image to be recognized including the loading tool is performed by the first feature extraction layer, the method further includes: An image obtained by image acquisition on the loading tool by an image acquisition device is acquired as the image to be recognized.

8. The method of claim 1, wherein, After the target feature map is used to identify each component included in the loading tool in the image to be recognized by the component identification layer to obtain a corresponding component identification result, the method further includes: Defect detection is performed on each identified component based on the component identification result to obtain a detection result. When the detection result indicates that a target component in the at least two components has a defect, a defect prompt information corresponding to the target component is output. 9.A method for training an image recognition model, the method comprising: The image recognition model includes a first feature extraction layer, a second feature extraction layer, a first feature fusion layer, a second feature fusion layer and a component identification layer. The method includes: The first feature extraction layer is used for performing image feature extraction on an image training sample including a loading tool and carrying a label to obtain a global feature map corresponding to the loading tool. The first feature extraction layer is used for performing feature extraction on the image training sample to obtain a component feature map of each component and a connectivity feature map used for indicating a connection relationship between the at least two components. Feature fusion is performed on the component feature map of each component and the connectivity feature map to obtain a probability feature map used for indicating a connection probability between the at least two components. The loading tool includes at least two components, and the at least two components have a fixed component connection relationship. The first feature fusion layer is used for performing feature fusion on the global feature map and the probability feature map to obtain an intermediate feature map. The second feature extraction layer is used for performing feature extraction on the intermediate feature map to obtain an edge feature map of each component, a main body feature map of each component and a background feature map of the loading tool. The second feature fusion layer is used for performing feature fusion on the edge feature map, the main body feature map and the background feature map to obtain a target feature map; Based on the target feature map, the component recognition layer is used for identifying each component included in the loading tool in the image training sample to obtain a corresponding component recognition result; A difference between the component recognition result and the label is obtained, and the image recognition model is trained based on the difference; The image recognition model obtained through the training is used for identifying each component of a target loading tool in a to-be-identified image including the target loading tool, and the target loading tool has the same component structure as the loading tool.

10. A component recognition apparatus based on an image recognition model, characterized by, The image recognition model includes a first feature extraction layer, a second feature extraction layer, a first feature fusion layer, a second feature fusion layer and a component recognition layer, and the device includes: A first feature extraction module is configured to perform image feature extraction on a to-be-identified image including a loading tool through the first feature extraction layer to obtain a global feature map corresponding to the loading tool, and perform feature extraction on the to-be-identified image through the first feature extraction layer to obtain a component feature map of each component and a connectivity feature map indicating a connection relationship between at least two components; and perform feature fusion on the component feature map of each component and the connectivity feature map to obtain a probability feature map indicating a connection probability between at least two components; wherein the loading tool includes at least two components, and the at least two components have a fixed component connection relationship; A first feature fusion module is configured to perform feature fusion on the global feature map and the probability feature map through the first feature fusion layer to obtain an intermediate feature map; A second feature extraction module is configured to perform feature extraction on the intermediate feature map through the second feature extraction layer to obtain an edge feature map of each component, a main body feature map of each component and a background feature map of the loading tool; A second feature fusion module is configured to perform feature fusion on the edge feature map, the main body feature map and the background feature map through the second feature fusion layer to obtain a target feature map; An identification module is configured to identify each component included in the loading tool in the to-be-identified image based on the target feature map through the component recognition layer to obtain a corresponding component recognition result.

11. A training device for an image recognition model, characterized in that, The image recognition model includes a first feature extraction layer, a second feature extraction layer, a first feature fusion layer, a second feature fusion layer and a component recognition layer, and the device includes: The first feature extraction module is configured to perform image feature extraction on the image training sample carrying the label and including the loading tool through the first feature extraction layer to obtain a global feature map corresponding to the loading tool, and perform feature extraction on the image training sample through the first feature extraction layer to obtain a component feature map of each component and a connectivity feature map indicating a connection relationship between at least two components; and perform feature fusion on the component feature map of each component and the connectivity feature map to obtain a probability feature map indicating a connection probability between at least two components; wherein the loading tool includes at least two components, and the at least two components have a fixed component connection relationship. The first feature fusion module is configured to perform feature fusion on the global feature map and the probability feature map through the first feature fusion layer to obtain an intermediate feature map. The second feature extraction module is configured to perform feature extraction on the intermediate feature map through the second feature extraction layer to obtain an edge feature map of each component, a main body feature map of each component, and a background feature map of the loading tool. The second feature fusion module is configured to perform feature fusion on the edge feature map, the main body feature map, and the background feature map through the second feature fusion layer to obtain a target feature map. The recognition module is configured to identify each component included in the loading tool in the image training sample based on the target feature map through the component recognition layer to obtain a corresponding component recognition result. The training module obtains a difference between the component recognition result and the label, and trains the image recognition model based on the difference; the image recognition model obtained through the training is used to identify each component of a target loading tool in a to-be-identified image including the target loading tool, and the target loading tool has the same component structure as the loading tool.

12. An electronic device, comprising: comprise: a memory configured to store executable instructions; a processor configured to execute the executable instructions stored in the memory to implement the component recognition method based on the image recognition model according to any one of claims 1 to 8; or execute the executable instructions stored in the memory to implement the training method of the image recognition model according to claim 9.

13. A computer-readable storage medium, characterized in that, executable instructions stored in the memory are configured to cause the processor to execute the component recognition method based on the image recognition model according to any one of claims 1 to 8; or cause the processor to execute the training method of the image recognition model according to claim 9.

14. A computer program product, characterised in that, The computer program product comprises computer instructions, and the computer program is executed by the processor to implement the component recognition method based on the image recognition model according to any one of claims 1 to 8; or the computer program is executed by the processor to implement the training method of the image recognition model according to claim 9.

Images