Image processing method and device, computer device and storage medium

Abstract

The application provides an image processing method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining m maintenance entries and n damage pictures of a target box, wherein m and n are positive integers; obtaining damage information reflected by each maintenance entry in the m maintenance entries, and obtaining damage information reflected by each damage picture in the n damage pictures; performing damage matching on the m maintenance entries and the n damage pictures based on the damage information reflected by each maintenance entry and the damage information reflected by each damage picture, to obtain a plurality of damage maintenance relationship groups, each damage maintenance relationship group comprising one maintenance entry and at least one damage picture, and the damage information reflected by the maintenance entry in each damage maintenance relationship group and the damage information reflected by the damage picture being matched, so that the efficiency and accuracy of matching the maintenance entry and the damage picture can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an image processing method and device, computer equipment and storage medium. BACKGROUND

[0002] The target box is a widely used shipping tool in the scenes of ships, ports, routes, highways, etc., and is standardized and convenient for mechanical equipment to load and unload and transport. Since the damage of the target box will lead to the failure of loading the goods, the deformation of the target box will make it difficult to stack the target box, etc., the quality monitoring, defect detection and maintenance of the target box become an important link. Among them, after the target box is transported to the yard for inspection and the maintenance items are opened for the damage detected, there is still a link, that is, the damage matching between the damage pictures in the target box and the maintenance items is verified.

[0003] At present, the damage matching between the maintenance items and the damage pictures in the target box is usually performed by manual damage matching, which is time-consuming and laborious and has low processing efficiency. Therefore, how to improve the damage matching efficiency between the maintenance items and the damage pictures is a technical problem to be solved at present. SUMMARY

[0004] The embodiments of the present application provide an image processing method and device, computer equipment and storage medium, which can improve the efficiency and accuracy of the damage matching between the maintenance items and the damage pictures.

[0005] In one aspect, the present application provides an image processing method, which comprises:

[0006] Obtaining m maintenance items and n damage pictures of a target box, m and n are positive integers;

[0007] Obtaining damage information reflected by each maintenance item in the m maintenance items, and obtaining damage information reflected by each damage picture in the n damage pictures;

[0008] Based on the damage information reflected by each maintenance item and the damage information reflected by each damage picture, the m maintenance items and the n damage pictures are damage matched to obtain a plurality of damage maintenance relationship groups, each damage maintenance relationship group includes one maintenance item and at least one damage picture, and the damage information reflected by the maintenance item in each damage maintenance relationship group and the damage information reflected by the damage picture are matched.

[0009] In one aspect, the present application provides an image processing device, which comprises:

[0010] An acquisition unit is configured to acquire m maintenance items and n damage pictures of a target box, m and n are positive integers;

[0011] The acquisition unit is further configured to acquire damage information reflected by each of the m maintenance entries and to acquire damage information reflected by each of the n damaged pictures;

[0012] The processing unit is configured to perform damage matching on the m maintenance entries and the n damaged pictures based on the damage information reflected by each of the maintenance entries and the damage information reflected by each of the damaged pictures, to obtain a plurality of damage maintenance relationship groups, each of which includes one maintenance entry and at least one damaged picture, and the damage information reflected by the maintenance entry in each of the damage maintenance relationship groups matches the damage information reflected by the damaged picture.

[0013] In a possible implementation, the damage information includes a damage position and damage detail information of the damage on the target box body.

[0014] When the processing unit performs damage matching on the m maintenance entries and the n damaged pictures based on the damage information reflected by each of the maintenance entries and the damage information reflected by each of the damaged pictures to obtain a plurality of damage maintenance relationship groups, the processing unit is configured to perform the following operations:

[0015] The maintenance entries and the damaged pictures having similar damage positions are divided into a set to obtain W to-be-matched sets, W being a positive integer.

[0016] In one to-be-matched set, the maintenance entries and the damaged pictures having the same damage detail information are determined as one damage maintenance relationship group.

[0017] In a possible implementation, the target box body includes a left face, a right face, a front face, a top face, a door panel face, an inner floor face, and a bottom floor face, each of which corresponds to a face identifier; the left face, the right face, and the front face are divided into a first face group, the top face is divided into a second face group, the door panel face is divided into a third face group, and the inner floor face and the bottom floor face are divided into a fourth face group.

[0018] The damage position includes a face identifier, and the i th maintenance entry and the j th damaged picture have similar damage positions means that the face indicated by the face identifier in the damage position of the i th maintenance entry and the face indicated by the face identifier in the damage position of the j th damaged picture belong to the same face group, i being a positive integer and i≤m, and j being a positive integer and j≤n.

[0019] In a possible implementation, the damage position of the j th damaged picture is obtained by performing damage position recognition processing on the j th damaged picture, and the processing unit is further configured to perform the following operations:

[0020] The j th damaged picture is subjected to position code recognition, and if the position code recognition is successful, the face identifier in the recognized position code is determined as the face identifier corresponding to the j th damaged picture.

[0021] If the position code recognition fails, the jth damaged picture is subjected to component recognition to determine that the jth damaged picture is obtained by photographing a target component on the target box body;

[0022] The face identifier corresponding to the target component is determined as the face identifier corresponding to the jth damaged picture.

[0023] In a possible implementation, the processing unit subjects the jth damaged picture to component recognition to determine that the jth damaged picture is obtained by photographing a target component on the target box body, and is configured to perform the following operations:

[0024] The jth damaged picture is subjected to component recognition processing to obtain a component recognition image, and the component recognition image includes image elements corresponding to multiple components on the target box body;

[0025] The target component is determined from the multiple components according to an area occupied by each component in the component recognition image.

[0026] In a possible implementation, the components on the target box body include any one or more of the following: a side plate, a front plate, a corner column, a door lock rod, a door handle, and a bottom cross beam.

[0027] When the processing unit determines the target component from the multiple components according to an area occupied by each component in the component recognition image, the processing unit is configured to perform the following operations:

[0028] If the multiple components include a bottom cross beam, and a ratio of an area occupied by the image element corresponding to the bottom cross beam in the component recognition image to a total area of the component recognition image reaches a reference threshold, the target component is determined to be the bottom cross beam.

[0029] If the multiple components do not include a bottom cross beam, a component corresponding to an image element with the largest area in the component recognition image is determined to be the target component.

[0030] In a possible implementation, the jth damaged picture belongs to a kth to-be-matched set, k is a positive integer and k≤W; and the damaged detail information of the jth damaged picture includes a damaged area code and a damaged type.

[0031] When the processing unit obtains the damaged detail information of the jth damaged picture, the processing unit is configured to perform the following operations:

[0032] A target face group corresponding to the kth to-be-matched set is determined, the face group corresponding to the kth to-be-matched set being a face group to which a face indicated by a face identifier in a damaged position of any one damaged picture in the kth to-be-matched set belongs.

[0033] determine the damage area code corresponding to the jth damaged picture based on the face identifier corresponding to the jth damaged picture and the damage area identifier of the damage area on the jth damaged picture;

[0034] determine the damage identifier of the jth damaged picture according to the damage feature corresponding to the target face group.

[0035] In a possible implementation, the target face group includes a first face group or a second face group, and the damage feature corresponding to the first face group and the second face group includes: the damage type of any face on which damage occurs is a first type of damage, and the ratio of the damage area on any face to the total area of any face is less than a first proportion threshold;

[0036] When determining the damage identifier of the jth damaged picture according to the damage feature corresponding to the target face group, the processing unit is configured to perform the following operation:

[0037] obtain an associated picture corresponding to the jth damaged picture;

[0038] perform damage identification on the associated picture corresponding to the jth damaged picture by using a damage identification algorithm corresponding to the first type of damage to determine the damage identifier of the damage area on which damage occurs in the jth damaged picture.

[0039] In a possible implementation, the target face group includes a third face group, and the damage feature corresponding to the third face group includes: the damage type of any face on which damage occurs is a first type of damage, and the components on the door panel face include a tape and other components, and the area of the tape on the door panel face is less than a second proportion threshold;

[0040] When determining the damage identifier of the jth damaged picture according to the damage feature corresponding to the target face group, the processing unit is configured to perform the following operation:

[0041] perform tape damage identification on the jth damaged picture to obtain a first identification result, and the first identification result is used to indicate whether the tape is damaged or not damaged;

[0042] perform other component damage identification on the jth damaged picture by using a damage identification algorithm corresponding to the first type of damage to obtain a second identification result, and the second identification result is used to indicate whether other components on the jth damaged picture are damaged or not damaged;

[0043] determine the damage identifier corresponding to the jth damaged picture according to the first identification result and the second identification result.

[0044] In a possible implementation, the jth damaged picture is composed of a red channel picture, a blue channel picture, and a green channel picture; the door panel face includes a left door and a right door, and the damage area identifier corresponding to the jth damaged picture includes a left door identifier and a right door identifier;

[0045] The processing unit is configured to perform the following operation when determining the damage area identifier of the jth damaged picture on which damage occurs:

[0046] replacing the blue channel picture in the jth damaged picture with a component identification picture of the jth damaged picture;

[0047] inputting the replaced jth damaged picture into the classification network model for classification processing to determine whether the jth damaged picture is obtained by photographing the left door or the right door;

[0048] adding the left door identifier or the right door identifier to the damage area identifier corresponding to the jth damaged picture.

[0049] In a possible implementation, the components on the door panel surface include a rubber strip.

[0050] The processing unit is configured to perform the following operation when performing rubber strip damage identification on the jth damaged picture to obtain a first identification result:

[0051] obtaining a double-branch attention model, the double-branch attention model including a rubber strip edge line segmentation and identification branch and a rubber strip damage classification branch, the rubber strip edge line segmentation and identification branch including a first feature extraction module, and the rubber strip damage classification branch including a second feature extraction module and a classification processing module;

[0052] performing mask processing on the jth damaged picture to obtain a mask picture, the mask picture including only information of image elements corresponding to the rubber strip;

[0053] calling the first feature extraction module to perform feature extraction on the mask picture to obtain edge line segmentation features of the rubber strip, and calling the second feature extraction module to perform feature extraction on the mask picture to obtain damage classification features of the rubber strip;

[0054] performing splicing processing on the edge line segmentation features and the damage classification features, and calling the classification processing module to perform classification processing based on a splicing processing result to obtain the first identification result.

[0055] In a possible implementation, the target face group corresponding to the kth to-be-matched set includes a fourth face group, and the damage characteristics corresponding to the fourth face group include that the damage category occurring on any face in the fourth face group includes one or more of a first type of damage and a second type of damage.

[0056] The processing unit is configured to perform the following operation when determining the damage identifier of the jth damaged picture according to the damage characteristics corresponding to the target face group:

[0057] The jth damage picture is subjected to damage identification by using the first-type damage corresponding damage identification algorithm and the second-type damage corresponding damage identification algorithm respectively, to obtain a third identification result and a fourth identification result; the third identification result is used to indicate whether there is or is not damage belonging to the first-type damage on the jth damage picture, and the fourth identification result is used to indicate whether there is or is not damage belonging to the second-type damage on the jth damage picture.

[0058] The damage identifier corresponding to the jth damage picture is determined according to the third identification result and the fourth identification result.

[0059] In an aspect, an embodiment of the present application provides a computer device, which comprises a memory and a processor, the memory stores one or more computer programs, and the computer program is executed by the processor to make the processor execute the image processing method.

[0060] In an aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is read and executed by a processor of a computer device to make the computer device execute the image processing method.

[0061] In an aspect, an embodiment of the present application provides a computer program product or a computer program, which comprises a computer program stored in a computer readable storage medium. The computer program is read by a processor of a computer device from the computer readable storage medium, and the processor executes the computer program to make the computer device execute the image processing method.

[0062] In the embodiment of the present application, after the m maintenance entries and the n damage pictures of the target box body are obtained, m and n are positive integers; the damage information reflected by each maintenance entry in the m maintenance entries can be obtained, and the damage information reflected by each damage picture in the n damage pictures can be obtained; and based on the damage information reflected by each maintenance entry and the damage information reflected by each damage picture, the m maintenance entries and the n damage pictures are subjected to damage matching to obtain a plurality of damage maintenance relationship groups. Each damage maintenance relationship group comprises one maintenance entry and at least one damage picture, and the damage information reflected by the maintenance entry in each damage maintenance relationship group and the damage information reflected by the damage picture are matched. It can be seen that, compared with manual matching, the intelligent damage matching between the maintenance entries and the damage pictures can be realized based on the damage information reflected by the maintenance entries and the damage information reflected by the damage pictures in the present application, and therefore the efficiency and accuracy of the matching between the maintenance entries and the damage pictures can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0064] Figure 1 is a structural schematic diagram of an image processing system provided by an embodiment of the present application;

[0065] Figure 2 is a flowchart of an image processing method provided by an embodiment of the present application;

[0066] Figure 3a is a schematic diagram of a maintenance item provided by an embodiment of the present application;

[0067] Figure 3b is a schematic diagram of dividing each face of a target box provided by an embodiment of the present application;

[0068] Figure 3c is a schematic diagram of dividing each region of a face provided by an embodiment of the present application;

[0069] Figure 4 is a schematic diagram of a damage picture of different types of damage provided by an embodiment of the present application;

[0070] Figure 5a is a schematic diagram of a matching result of a damage picture and a maintenance item provided by an embodiment of the present application;

[0071] Figure 5b is a schematic diagram of a retention map provided by an embodiment of the present application;

[0072] Figure 6 is a flowchart of a damage matching method provided by an embodiment of the present application;

[0073] Figure 7a is a schematic diagram of damage picture position code recognition provided by an embodiment of the present application;

[0074] Figure 7b is a schematic diagram of another damage picture position code recognition provided by an embodiment of the present application;

[0075] Figure 7c is a schematic diagram of damage picture component recognition provided by an embodiment of the present application;

[0076] Figure 7d is a schematic diagram of identifying a bottom crossbeam provided by an embodiment of the present application;

[0077] Figure 8ais a schematic diagram of a damaged picture provided by an embodiment of the present application;

[0078] Figure 8b is a schematic diagram of various damaged pictures provided by an embodiment of the present application;

[0079] Figure 8c is a schematic diagram of a maintenance item provided by an embodiment of the present application;

[0080] Figure 8d is a schematic diagram of a maintenance item in order provided by an embodiment of the present application;

[0081] Figure 9a is a schematic diagram of a damaged picture and an associated picture provided by an embodiment of the present application;

[0082] Figure 9b is a schematic diagram of damaged identification of a damaged picture provided by an embodiment of the present application;

[0083] Figure 10a is a schematic diagram of a door plate of a target box provided by an embodiment of the present application;

[0084] Figure 10b is a structural schematic diagram of a classification network model provided by an embodiment of the present application;

[0085] Figure 10c is a schematic diagram of a rubber strip on a door plate provided by an embodiment of the present application;

[0086] Figure 10d is a schematic diagram of a mask diagram of a damaged picture provided by an embodiment of the present application;

[0087] Figure 10e is a structural schematic diagram of a double-branch attention model provided by an embodiment of the present application;

[0088] Figure 11 is a schematic diagram of various damaged identifications on a floor provided by an embodiment of the present application;

[0089] Figure 12 is a principle schematic diagram of a damaged matching method provided by an embodiment of the present application;

[0090] Figure 13 is a structural schematic diagram of an image processing device provided by an embodiment of the present application;

[0091] Figure 14 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0092] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is made in connection with the drawings, in which the same reference numerals represent the same elements throughout the several figures, and the illustrative embodiments are not intended to be exhaustive or to be handed down as a scientific treatise. Rather, the purpose of this description is to convey some embodiments of the application to others to allow others to make and use the application.

[0093] The embodiment of the present application provides an image processing scheme, which can be used for intelligently matching a damaged picture and a maintenance item, can save the matching time between the damaged picture and the maintenance item, and thus improves the matching efficiency of the maintenance item and the damaged picture. The general principle of the scheme is as follows:

[0094] When it is needed to match the maintenance item and the damaged picture of a target box body, firstly, m maintenance items and n damaged pictures of the target box body can be acquired; then, the damage information reflected by each maintenance item in the m maintenance items and the damage information reflected by each damaged picture in the n damaged pictures can be acquired; finally, the m maintenance items and the n damaged pictures are matched based on the damage information reflected by each maintenance item and the damage information reflected by each damaged picture, and a plurality of damage maintenance relationship groups can be obtained. Each damage maintenance relationship group includes one maintenance item and at least one damaged picture, and the damage information reflected by the maintenance item and the damage information reflected by the damaged picture in each damage maintenance relationship group are matched. It can be seen that, in the embodiment of the present application, the intelligent damage matching between the maintenance item and the damaged picture can be realized based on the damage information reflected by the maintenance item and the damage information reflected by the damaged picture. Compared with the manual damage matching processing of the damaged picture and the maintenance item, the present application does not need manual intervention, realizes the intelligent matching between the maintenance item and the damaged picture, and thus can improve the matching efficiency and accuracy.

[0095] Next, the image processing scheme mentioned above will be introduced in combination with the technical terms related to the present application.

[0096] In a possible implementation, the image processing scheme of the present application can be combined with deep learning technology in the field of artificial intelligence. For example, deep learning technology can be used to obtain damage information reflected by the damage picture and to obtain damage information reflected by the maintenance item. The so-called artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, large function testing technology, operation / interaction systems, mechatronics, etc. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0097] In a possible implementation, the image processing scheme can be executed by a computer device. The computer device can be a terminal device or a server, which is not limited. The terminal device mentioned herein can include, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palm computer, a mobile internet device (MID), a smart voice interaction device, a vehicle-mounted terminal, a roadside device, an aircraft, a wearable device, a smart home appliance, or a wearable device with image processing function such as a smart watch, a smart bracelet, a pedometer, etc. The server can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms, etc.

[0098] Further, the computer device can be located in the blockchain network or outside the blockchain network, and no limitation is made in this regard. The so-called blockchain network is a network composed of a peer-to-peer network (P2P network) and a blockchain, and the blockchain refers to a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. In essence, it is a decentralized database, and is a series of data blocks (or blocks) associated using cryptographic methods. When the computer device is located in the blockchain network or is in communication connection with the blockchain network, the computer device can upload internal data (such as m maintenance entries, n damaged pictures, damaged information reflected by each maintenance entry, damaged information reflected by each damaged picture, and damaged maintenance relationship groups) to the blockchain of the blockchain network for storage, so as to prevent the internal data of the computer device from being tampered with, thereby improving the security of the internal data.

[0099] In another possible implementation, the image processing scheme provided by the embodiments of the present application can be executed by a terminal device and a server together. The terminal device and the server can be directly or indirectly connected through wired or wireless communication. In this case, the terminal device and the server can constitute an image processing system, as shown in Figure 1 Figure 1 is a structural schematic diagram of an image processing system provided by the embodiments of the present application. In executing the image processing scheme, the terminal device can be responsible for acquiring m maintenance entries and n damaged pictures of a target box body, and for sending the acquired data to the server. Correspondingly, the server can be responsible for acquiring damaged information reflected by each maintenance entry in the m maintenance entries and damaged information reflected by each damaged picture in the n damaged pictures after receiving the data (m maintenance entries and n damaged pictures). Then, the server performs damaged matching on the m maintenance entries and the n damaged pictures based on the damaged information reflected by each maintenance entry and the damaged information reflected by each damaged picture, and obtains a plurality of damaged maintenance relationship groups. Optionally, the server can send the obtained plurality of damaged maintenance relationship groups to the terminal device, and the terminal device can output and display the maintenance entries and damaged pictures included in the damaged maintenance relationship groups.

[0100] ​It should be understood that the above merely illustrates the steps performed by the terminal device and the server, and does not limit the scope of the work. For example, in other embodiments, the terminal device may be responsible for obtaining the damage information reflected in each of the m maintenance entries and the damage information reflected in each of the n damage images. In this case, the terminal device can send the data of m maintenance entries, n damage images, the damage information reflected in each maintenance entry, and the damage information reflected in each damage image to the server, so that the server can perform the operation of determining multiple damage repair relationship groups based on this data. For example, in other embodiments, after the terminal device sends m maintenance items and n damage images to the server, the server can be responsible for obtaining the damage information reflected in each of the m maintenance items and the damage information reflected in each of the n damage images. Then, the server sends the damage information reflected in each maintenance item and the damage information reflected in each damage image to the terminal device. The terminal device uses the damage information reflected in each maintenance item and the damage information reflected in each damage image to perform damage matching on the m maintenance items and the n damage images, thereby obtaining multiple damage repair relationship groups, and so on.

[0101] Furthermore, it can be Figure 1 The provided image processing system is deployed on blockchain nodes. For example, servers and terminal devices can both be considered blockchain nodes, collectively forming a blockchain network. Therefore, the image processing workflow for multiple damage repair relationship groups defined in this application can be executed on the blockchain. This ensures fairness and impartiality in the image processing workflow, while also making it traceable, thereby enhancing the security of the image processing workflow.

[0102] It should be noted that, in executing the image processing scheme of this application, the damage information reflected in each of the m maintenance entries and the damage information reflected in each of the n damaged images are obtained. Based on the damage information reflected in each maintenance entry and each damaged image, the m maintenance entries and n damaged images are matched for damage, resulting in multiple damage repair relationship groups. These processes all involve large-scale computation and require significant computing power and storage space. Therefore, in one possible implementation of this application, a computer device can obtain sufficient computing power and storage space through cloud computing technology to execute the determination of multiple damage repair relationship groups involved in this application. Cloud computing is a computing model that distributes computing tasks across a resource pool composed of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." Resources in the "cloud" appear infinitely scalable to users and can be accessed at any time, used on demand, expanded at any time, and paid for based on usage.

[0103] Based on the above description of the image processing scheme, this application proposes an image processing method. This image processing method can be executed by the aforementioned computer device, or by the aforementioned terminal device and server jointly. For ease of explanation, the following description will use the execution of this image processing method by a computer device as an example; see [link to documentation]. Figure 2 As shown, the image processing method may include steps S201 to S204:

[0104] S201: Obtain m maintenance entries and n damage images of the target enclosure.

[0105] In this context, "target container" refers to any one of the containers requiring verification, which can be a shipping container, and m and n are both positive integers. In this embodiment, "maintenance entry" refers to text used to record relevant information about damage to the target container during repair and inspection. This information may include the damaged component on the target container, the side and area of ​​the damaged component, the type of damage, and repair information. The maintenance entry can be determined after the target container enters the yard and is inspected by personnel, identifying various types of damage that could affect subsequent transportation and use.

[0106] In practice, the maintenance entries for the target enclosure can be stored in a table. Please refer to [link / reference]. Figure 3a This is a schematic diagram of a maintenance entry provided in an embodiment of this application. Figure 3a In this example, assuming the target enclosure includes 5 maintenance entries, where 301 represents one of them, and taking maintenance entry 301 as an example, the relevant information regarding damage in a maintenance entry can include component code, location code, damage code, repair code, etc. Figure 3a In the table shown, the “CMP” column stores the component code for each maintenance item, the “LOC” column stores the location code for each maintenance item, the “DMG” column stores the damage code for each maintenance item, the “REP” column stores the repair code for each maintenance item, the “Length” column stores the length information of the damage in the target enclosure, the “Width” column stores the width information of the damage in the target enclosure, and so on.

[0107] It should be understood that the target box is a standardized steel box by specification, which is assembled by corrugated plates, corner columns, corner pieces and other standardized parts. Each part has a corresponding code for representation, which is called a part code. Each part code can consist of three capital letters, which is used to uniquely represent a certain type of part on the target box, such as the part code of the corner piece CFG, the part code of the side plate PAA, the part code of the front plate FAA, the part code of the corner column CPA, the part code of the door lock lever LBR, and the part code of the door handle LBH, etc. During transportation and use, the target box may be damaged due to impact or rain and other accidents, such as broken holes, deformation, rust, dirt and other types of damage, which affect subsequent use. The damage code is a standard code used to indicate different types of damage, which consists of two capital letters, such as the damage code of the deformation BT.

[0108] The position code is used to identify the position code of the damaged target box. Each face and area of the target box is also standardized. According to the face and area of the damage, a four-character position code can be obtained, such as RX3N, where R represents the right face, X represents the middle area of the right face, and 3N represents the third area from the door frame side. Therefore, the position code is used to identify a rough area where a damage occurs. The box inspector will also write the position code next to the damage location with chalk to indicate the specific location, and then take a photo for record. Please refer to Figure 3b , Figure 3b is a schematic diagram provided by an embodiment of the present application for dividing each face of the target box. As shown in Figure 3b , taking the perspective of the rear door of the target box as an example, the target box can be divided into seven faces, each face corresponding to a face identifier, which can be a letter. For example: D (rear door), F (front), L (left), R (right), T (top), U (bottom), B (floor inside the box). For each face (taking the right face R as an example), the face can be further divided into multiple areas. Please refer to Figure 3c , Figure 3c is a schematic diagram provided by an embodiment of the present application for dividing each area of the face. As shown in Figure 3cAs shown, the face can be divided into five longitudinal areas H, T, X, B, G from top to bottom; and divided into five transverse areas 1, 2, 3, 4, 5 from left to right. For example, the position code is RT1N, R represents the identity of the face (right face), T represents the longitudinal area identity, 1 represents the transverse area identity, and N refers to the filling code. In addition, for cross-area, for transverse cross-area, the starting area and the ending area are identified, such as transverse cross-area 1, 2, 3, the position code is RT13. For longitudinal cross-area, if crossing 2 longitudinal areas, the larger area is identified first; if crossing 3 longitudinal areas, the middle area X is identified. It should be noted that if crossing 4 longitudinal areas, such as crossing H, T, X, B, the longitudinal area H needs to be identified separately. Taking the right face R as an example, the identification is RH13 and RX13 (assuming crossing 1 and 3 transverse areas). In addition, if crossing 5 longitudinal areas, such as crossing H, T, X, B, G, the longitudinal areas H and G need to be identified separately. Taking the right face R as an example, the identification is RH13, RB13 and RX13 (assuming crossing 1 and 3 transverse areas).

[0109] The damage picture refers to a picture obtained by using a shooting device (such as a camera, a mobile phone camera, etc.) to shoot a damaged part on the target box, wherein the damage picture can also include relevant mark information (such as handwritten damage position code) handwritten by the box inspection personnel during the box inspection process. There are many types of damage on the target box, such as corrosion (CO), deformation (BT), missing parts (MS), component wear (WT) and breakage (BR) and the like. Please refer to Figure 4 , Figure 4 is a schematic diagram of a damage picture of different types of damage provided by an embodiment of the present application. As shown in Figure 4 , Figure 4 shows a damage picture of different types of damage provided by an embodiment of the present application, such as a damage picture of deformation (BT), a damage picture of missing parts (MS), a damage picture of component wear (WT), and a damage picture of breakage (BR). It should be noted that in the present application, the damage picture obtained by the box inspection personnel during the box inspection process of the target box can be stored in a folder, and the folder can include n damage pictures of the target box.

[0110] S202: Obtain the damage information reflected by each maintenance entry in the m maintenance entries, and obtain the damage information reflected by each damage picture in the n damage pictures.

[0111] In a specific implementation, the damage information can include a damage location and damage detail information of the damage occurred on the target box. The damage location can include a face identifier. The computer device can obtain the damage location in each of the m maintenance entries by reading the location code included in each maintenance entry, and can obtain the damage detail information of the maintenance entry by reading the damage code and component code included in each maintenance entry. For example, the damage location reflected by a maintenance entry can be determined by the location code included in the maintenance entry. For example, the location code included in the maintenance entry is RX3N, the first character (R) in the location code can be determined as the face identifier, and the damage location on the target box reflected by the maintenance entry can be determined as "right face" according to the face identifier "R". For another example, the location code included in the maintenance entry is DX3N, the damage location on the target box reflected by the maintenance entry is "D (door panel face)", the damage code included in the maintenance entry is CO (corrosion), and the component code is LBR (door lock rod), and the damage detail information of the target box reflected by the maintenance entry is that the door lock rod is corroded.

[0112] In addition, the computer device can obtain the damage location of the damage picture by damage location recognition processing on each damage picture, and can obtain the damage detail information of the damage picture by damage type recognition and component recognition processing on the damage picture. In a possible implementation, the damage location of each damage picture can be obtained by location code recognition on each damage picture. If the location code recognition is successful, the face identifier in the recognized location code is determined as the face identifier corresponding to each damage picture. If the location code recognition fails, component recognition is performed on each damage picture to determine that each damage picture is obtained by photographing a target component on the target box, and the face identifier corresponding to the target component is determined as the face identifier corresponding to the damage picture.

[0113] S203: Based on the damage information reflected by each maintenance entry and the damage information reflected by each damage picture, the m maintenance entries and the n damage pictures are matched for damage to obtain a plurality of damage repair relationship groups.

[0114] In the embodiments of the present application, each damage maintenance relationship group includes one maintenance entry and at least one damage picture, and the damage information reflected by the maintenance entry in each damage maintenance relationship group matches the damage information reflected by the damage picture. For example, if a damage maintenance relationship group includes maintenance entry 1 and damage picture 1, the damage information reflected by maintenance entry 1 in the damage maintenance relationship group matches the damage information reflected by damage picture 1. For another example, if a damage maintenance relationship group includes maintenance entry 1 and damage picture 1, damage picture 2 and damage picture 3, the damage information reflected by maintenance entry 1 in the damage maintenance relationship group matches the damage information reflected by damage picture 1, damage picture 2 and damage picture 3. For convenience of description, the following description is based on an example in which one maintenance relationship group includes one maintenance entry and one damage picture. For example, the damage information reflected by a maintenance entry is that the door handle on the target cabinet is deformed, and the damage picture matched with the damage information reflected by the maintenance entry is a picture obtained by shooting the deformed door handle on the target cabinet.

[0115] In a possible implementation, in the process of damage matching of the m maintenance entries and the n damage pictures, for any maintenance entry in the m maintenance entries, the maintenance entry can be matched with each damage picture in the n damage pictures one by one until a damage picture matched with the maintenance entry is determined. Alternatively, for any damage picture in the n damage pictures, the damage picture can be matched with each maintenance entry in the m maintenance entries one by one until a maintenance entry matched with the damage picture is determined. Then, in the process of damage matching, if the damage information reflected by a maintenance entry X matches the damage information reflected by a damage picture Y, the maintenance entry X and the damage picture Y are determined as a damage maintenance relationship group.

[0116] In another possible implementation, the damage information can include a damage position and damage detail information of the damage occurred on the target container. In the process of damage matching of the m maintenance entries and the n damage pictures, first, the maintenance entries and the damage pictures having similar damage positions are divided into a set, to obtain W sets to be matched, W being a positive integer; then in a set to be matched, the maintenance entries and the damage pictures having the same damage detail information are determined as a damage maintenance relationship group. Wherein, the damage position can include a face identifier, the maintenance entry of the i th and the damage picture of the j th having similar damage positions means that the face indicated by the face identifier in the damage position of the maintenance entry of the i th belongs to the same face group as the face indicated by the face identifier in the damage position of the damage picture of the j th, i being a positive integer and i≤m, j being a positive integer and j≤n. For example, the face indicated by the face identifier in the damage position of the maintenance entry of the i th is the right face R, and the face indicated by the face identifier in the damage position of the damage picture of the j th is also the right face R, then it can be considered that the maintenance entry of the i th and the damage picture of the j th have similar damage positions. In this way, in the process of damage matching, the maintenance entries and the damage pictures having similar damage positions can be first divided into sets, and then the maintenance entries and the damage pictures included in each set after the set division are matched again, compared with one-by-one matching of all the maintenance entries and the damage pictures, this way of first classification and then matching can reduce the workload, thereby improving the matching efficiency.

[0117] It should be noted that the computer device can also store basic information of the target container, and the basic information of the target container can be stored in the form of a table. Moreover, the plurality of damage maintenance relationship groups can be stored in association with the table of the target container, and subsequently, when the user needs to obtain the plurality of damage maintenance relationship groups of the target container, the table of the target container can be obtained. Wherein, the table of the target container can be as shown in Table 1:

[0118] Table 1. Table of the target container

[0119]

[0120]

[0121] As shown in the table of Table 1, some basic information of the target container such as container owner and size can be displayed, for example, the identification of the target container (such as container number EISU9304637, the reason for entering the container as “heavy container emptying” and the date, bill of lading number and the like information).

[0122] In addition, in the process of damage matching of the damage picture and the maintenance entry, the computer device can also output the matching result of the damage picture and the maintenance entry. Please refer to Figure 5a ,Figure 5a is a matching result diagram of a damaged picture and a maintenance item provided by an embodiment of the present application. The folders 1-5 correspond to the maintenance items in the maintenance item list shown in the foregoing Figure 3a , and the sequence number column in the maintenance item list shown in the foregoing Figure 3a , and the damaged picture matched with the maintenance item 310; for example, the folder 2 includes the maintenance item 302 with the sequence number 2 and the damaged picture matched with the maintenance item 302. The damaged pictures that are not matched with the maintenance items are generally the retained pictures such as the nameplate picture (501), the box number picture (502), the large angle inner box picture (503) and the outer box picture (504) of the target box, as shown in Figure 5b Figure 5b is a diagram of a retained picture provided by an embodiment of the present application.

[0123] In the embodiment of the present application, after the m maintenance items and the n damaged pictures of the target box are obtained, m and n are positive integers; the damage information reflected by each maintenance item in the m maintenance items can be obtained, and the damage information reflected by each damaged picture in the n damaged pictures can be obtained; and the m maintenance items and the n damaged pictures are matched based on the damage information reflected by each maintenance item and the damage information reflected by each damaged picture, to obtain a plurality of damaged maintenance relationship groups. Each damaged maintenance relationship group includes one maintenance item and at least one damaged picture, and the damage information reflected by the maintenance item in each damaged maintenance relationship group and the damage information reflected by the damaged picture are matched. It can be seen that, compared with manual matching, the intelligent damaged matching between the maintenance items and the damaged pictures can be realized based on the damage information reflected by the maintenance items and the damage information reflected by the damaged pictures in the present application, so that the efficiency and accuracy of matching the maintenance items and the damaged pictures can be improved.

[0124] It should be noted that the above embodiment exemplarily lists two implementation manners of the step S203. Next, a second possible implementation manner of the step S203 proposed by the embodiment of the present application is described in detail. Next, the flowchart shown in Figure 6 is described with reference to FIG. 6, and the flowchart can specifically include the following steps S601-S602: Figure 6

[0125] S601: Dividing the maintenance items and the damaged pictures with similar damaged positions into a set to obtain W sets to be matched, and W is a positive integer.

[0126] ​​In a specific implementation, the target box can include seven surfaces, which are a left surface, a right surface, a front surface, a top surface, a door panel surface, an inner floor surface and a bottom floor surface. Each surface corresponds to a surface identifier, and the damage location includes the surface identifier. The left surface, the right surface and the front surface can be classified into a first surface group, the top surface can be classified into a second surface group, the door panel surface can be classified into a third surface group, and the inner floor surface and the bottom floor surface can be classified into a fourth surface group. The i th maintenance item and the j th damage picture have similar damage locations means that the surface indicated by the surface identifier in the damage location of the i th maintenance item and the surface indicated by the surface identifier in the damage location of the j th damage picture belong to the same surface group, i is a positive integer and i≤m, and j is a positive integer and j≤n.

[0127] In a possible implementation, in the process of dividing the maintenance items and the damage pictures having similar damage locations into a set to obtain W sets of to-be-matched sets, the computer device can first determine the damage location of each damage picture. The damage location can include a surface identifier. The damage location of the j th damage picture is obtained by performing damage location identification processing on the j th damage picture. In the process of performing step S601, the computer device can specifically perform steps s11-s13.

[0128] s11, the computer device performs position code identification on the j th damage picture, and if the position code identification is successful, the surface identifier in the identified position code is determined as the surface identifier corresponding to the j th damage picture.

[0129] For example, please refer to Figure 7a and Figure 7b , Figure 7a and Figure 7b are a damage picture position code identification schematic diagram provided by an embodiment of the present application. As shown in Figure 7a , the position code obtained after successfully performing position code identification on the damage picture can be "RX24", and then the surface identifier corresponding to the damage picture is determined according to the first character in the position code, that is, "R". Similarly, as shown in Figure 7b , the position code obtained after successfully performing position code identification on the damage picture can be "DB3N", and then the surface identifier corresponding to the damage picture is determined according to the first character in the position code, that is, "D".

[0130] s12, if the position code identification fails, the computer device performs component identification on the j th damage picture to determine that the j th damage picture is obtained by photographing a target component on the target box.

[0131] The position code recognition failure can include: if the jth damaged picture includes a position code, but the position code in the damaged picture cannot be successfully recognized, the computer device can be considered to fail in position code recognition of the jth damaged picture. In addition, the position code recognition failure can also include that the jth damaged picture does not include a position code or the position code is incomplete (for example, is blocked by water stains, dirt, etc.), so the computer device can also be considered to fail in position code recognition of the jth damaged picture.

[0132] In a possible implementation, the computer device can determine the target component on the target box by performing component recognition on the jth damaged picture, which can include: first, the computer device performs component recognition processing on the jth damaged picture to obtain a component recognition image, and the component recognition image includes image elements corresponding to a plurality of components on the target box. Then, the computer device determines the target component from the plurality of components according to the area of the image element corresponding to each component in the component recognition image.

[0133] For example, referring to Figure 7c , Figure 7c is a schematic diagram of component recognition of a damaged picture provided by an embodiment of the present application. As shown in Figure 7c , the picture 701 is a schematic diagram of the jth damaged picture, and the component recognition image obtained by performing component recognition on the damaged picture 701 is shown in the picture 702. The component recognition image 702 can include image elements corresponding to a plurality of components, and the color of the image element corresponding to each component can be different, that is, the colors of the image elements corresponding to different components are different. For example, the component recognition image 702 includes image elements of the side corrugated plate, the upper cross beam, the corner column, the lower cross beam, the front corrugated plate, and the air window. Further, according to the different image elements, it can be determined that the red color is the side corrugated plate, the pink color is the upper cross beam, the gray color is the corner column, the yellow-green color is the lower cross beam, the green color is the front corrugated plate, and the dark green color is the air window, and so on.

[0134] In a possible implementation, the components on the target box include any one or more of the following: a side plate, a front plate, a corner column, a door lock rod, a door handle, and a bottom cross beam. The computer device can determine the target component from the plurality of components according to the area of the image element corresponding to each component in the component recognition image, which can include:

[0135] ① If the plurality of components includes a bottom cross beam (CMA), and the ratio of the area of the image element corresponding to the bottom cross beam to the total area of the component recognition image reaches a reference threshold, the target component is determined to be the bottom cross beam. For example, referring to Figure 7d , Figure 7d is a schematic diagram of recognizing a bottom cross beam provided by an embodiment of the present application. As shown inFigure 7d As shown, the damaged picture in the picture 703 includes a bottom crossbeam, and the component recognition picture 704 obtained after component recognition on the damaged picture 703, if the pixel number ratio of the bottom crossbeam reaches a certain fixed threshold, it indicates that the ratio of the area of the image element corresponding to the bottom crossbeam in the component recognition picture to the total area of the component recognition picture reaches the reference threshold, then the target component is determined as the bottom crossbeam. That is, it can be determined that the damaged picture 703 is obtained by shooting the bottom crossbeam on the target box body.

[0136] ②If the bottom crossbeam is not included in the plurality of components, the component corresponding to the image element with the largest area in the component recognition picture is determined as the target component. For example, Figure 7c As shown in the component recognition picture, the areas of the image elements of the side corrugated plate, the upper crossbeam, the corner column, the lower crossbeam, the front corrugated plate and the air window in the component recognition picture are obtained, for example, S1, S2, S3, S4, S5, S6 respectively, and then the maximum value is determined from S1, S2, S3, S4, S5, S6, assuming S1, then the target component is determined as the side corrugated plate.

[0137] s13, the computer device determines the face identifier corresponding to the target component as the face identifier corresponding to the jth damaged picture.

[0138] In specific implementation, according to the target component determined by s12, the face identifier corresponding to the target component can be determined as the face identifier of the jth damaged picture. For example, if the determined target component is a "door handle", the face identifier corresponding to the jth damaged picture can be determined as "D"; for another example, if the determined target component is a "front plate", the face identifier corresponding to the jth damaged picture can be determined as "F". It should be noted that if the target component is determined as a bottom crossbeam, the face identifier corresponding to the jth damaged picture can be determined as the face identifier corresponding to any face in the fourth face group.

[0139] In a possible implementation, after the computer device executes steps s11-s13, the computer device can further execute steps s21-s22 in the process of executing step S601:

[0140] s21, Classify the damaged images. After identifying the damaged locations in the images, the n damaged images can be classified based on the location code identification result or the component identification result. This means grouping damaged images whose faces, indicated by the face markers at the damaged locations, belong to the same face group into one set. The target box includes seven faces: left, right, front, top, door panel, inner floor, and bottom floor, which can be represented by the capital letters L, R, F, T, D, B, and U, respectively. Based on this standard and the similarities and differences between different surfaces, the three similar surfaces of the corrugated board—left (L), right (R), and front (F)—can be classified into the first surface group, denoted by "RLF". The top surface (T) is classified into the second surface group, denoted by "T". The door panel surface (D) is classified into the third surface group, denoted by "D". The inner floor surface (B) and the bottom floor surface (U) are classified into the fourth surface group, denoted by "BU". Therefore, based on the results of damage location identification processing for each damaged image, n damaged images can be divided into four categories: RLF, T, D, and BU. Each category corresponds to a surface group, i.e., RLF corresponds to the first surface group, T to the second surface group, D to the third surface group, and BU to the fourth surface group.

[0141] For example, see Figure 8a , Figure 8a This is a schematic diagram illustrating the classification of damaged images provided in an embodiment of this application. For example... Figure 8a As shown, based on the face markers included in the damaged location of each identified damaged image, the... Figure 8a The 22 damaged images shown are divided into three categories: RLF, D, and BU, namely, the first side group, the third side group, and the fourth side group. For another example, please refer to... Figure 8b , Figure 8b This is a schematic diagram of various types of damaged images provided in the embodiments of this application. For example... Figure 8b As shown, Figure 8b The following images are shown: a damaged image (801) of the first face group RLF, a damaged image (802) of the third face group D, a damaged image (803) of the second face group T, and a damaged image (804) of the fourth face group BU provided in this application embodiment.

[0142] s22, Classify the maintenance entries. Next, based on the location codes included in the maintenance entries, the m maintenance entries can be classified. That is, each maintenance entry can be classified according to the first character of its location code. Please refer to [link to relevant documentation]. Figure 8c , Figure 8c This is a schematic diagram illustrating the classification of maintenance entries according to an embodiment of this application. For example... Figure 8cAs shown, each maintenance entry can include a location code (LOC), for example, the location code of maintenance entry 1 (i.e. the maintenance entry corresponding to the serial number 1) is "BR3N", then when classifying the maintenance entry 1, according to the first character "B" in the location code, the maintenance entry 1 can be classified into the BU class, i.e. the fourth face group; for example, the location code of maintenance entry 3 is "LR6N", then when classifying the maintenance entry 3, according to the first character "L" in the location code, the maintenance entry 1 can be classified into the RLF class, i.e. the first face group. In this way, the face group to which each of the m maintenance entries belongs can be determined. It should be noted that since the target box has some damage such as dirt, odor, etc. which does not belong to any specific damage on a face, the location code (LOC) in the maintenance entry issued for these damages can correspond to IXXX, EXXX, etc. Then, since the first character of such maintenance entries is I or E, the face group to which the maintenance entries belong cannot be determined according to I or E, so the maintenance entries can also be classified into the fourth face group.

[0143] Finally, after classifying the m maintenance entries and the n damage pictures respectively, the maintenance entries and the damage pictures with similar damage locations can be classified into a set to obtain a to-be-matched set. For example, if the face indicated by the face identifier in the damage location of the i th maintenance entry is R, and the face indicated by the face identifier in the damage location of the j th damage picture is R, then the i th maintenance entry and the j th damage picture are classified into a to-be-matched set. For example, if the face indicated by the face identifier in the damage location of the i th maintenance entry is L, and the face indicated by the face identifier in the damage location of the j th damage picture is R, then the i th maintenance entry and the j th damage picture are classified into a to-be-matched set. That is, the face indicated by the face identifier in the damage location of the i th maintenance entry and the face indicated by the face identifier in the damage location of the j th damage picture belong to any face in the first face group, and the i th maintenance entry and the j th damage picture can be classified into the same set. In this way, after classifying the m maintenance entries and the n damage pictures, W to-be-matched sets can be obtained.

[0144] It should be noted that the box inspector usually inspects the damage on one face and then inspects the damage on another face during the inspection of the target box, so the maintenance entries on the same face are usually adjacent. See the following table: Figure 8d , Figure 8dFigure 1 is a schematic diagram of maintenance items arranged in order provided by an embodiment of the present application. The broken pictures are also generally in the order of the maintenance items, and due to the similarities and differences of different faces and the non-sequential nature of broken pictures on the same face, when the present application performs collection division on the maintenance items and the broken pictures, the maintenance items and the broken pictures with similar broken locations are divided into the same set, and the maintenance items and the broken pictures with different broken locations are divided into different sets. In addition, in the case that the i th maintenance item and the j th maintenance item have similar broken locations but are not adjacent, the i th maintenance item and the j th maintenance item are divided into different matching sets. For example, maintenance item 1 and maintenance item 8, the face indicated by the face identifier in the broken location of maintenance item 1 is B, and the face indicated by the face identifier in the broken location of maintenance item 8 is also B; but since maintenance item 1 and maintenance item 8 are not adjacent, maintenance item 1 and maintenance item 8 can be divided into different matching sets.

[0145] S602: In one matching set, determine the maintenance item and the broken picture with the same broken detail information as one broken maintenance relationship group.

[0146] In a specific implementation, the j th broken picture belongs to the k th matching set, k is a positive integer and k≤W; the broken detail information of the j th broken picture includes a broken area code and a broken type. Therefore, in the process of executing step S602, the computer device can obtain the broken detail information of the j th broken picture. In the process of obtaining the broken detail information of the j th broken picture by the computer device, the following steps s31-s33 can be specifically executed:

[0147] s31: Determine the target face group corresponding to the k th matching set. The face group corresponding to the k th matching set refers to the face group to which the face indicated by the face identifier in the broken location of any broken picture in the k th matching set belongs.

[0148] For example, the face indicated by the face identifier in the broken location of the broken picture included in the k th matching set is R, and since R belongs to the first face group (RLF), the target face group corresponding to the k th matching set is the first face group; for another example, the face indicated by the face identifier in the broken location of the broken picture included in the k th matching set is T, and since T belongs to the second face group (T), the target face group corresponding to the k th matching set is the second face group.

[0149] s32: Determine the broken area code corresponding to the j th broken picture based on the face identifier corresponding to the j th broken picture and the broken area identifier on the j th broken picture.

[0150] In a specific implementation, the damage position further includes a damage area identifier. After the damage position identification processing on the jth damaged picture, a damage area identifier of the damage occurred on the jth damaged picture can also be obtained. For example, after the damage position identification processing on the jth damaged picture, it is determined that the damage area of the damage occurred on the jth damaged picture is the third corrugation from the door frame side in the middle area of the right face. Then, according to the damage area and in combination with the foregoing description of the area division, it can be determined that the damage area code of the damage occurred on the jth damaged picture can be “RX3N”.

[0151] s33, determining the damage identifier of the jth damaged picture according to the damage feature corresponding to the target face group.

[0152] The damage identifier is used to indicate the damage type of the damage occurred on the target box body. In a specific implementation, the damage feature corresponding to different face groups can be different, that is, the damage type of the damage occurred on the damaged picture can be different, and the damage identifiers corresponding to different types of damage are also different. For example, if the damage is deformation, the corresponding damage identifier can be BT; for another example, if the damage is corrosion, the corresponding damage identifier can be CO; for still another example, if the damage is a break, the corresponding damage identifier can be BR. Therefore, the following describes how to determine the damage identifier of the damaged picture for different face groups:

[0153] (1) If the target face group includes the first face group (RLF) or the second face group (T), the damage feature corresponding to the first face group and the second face group includes: the damage type of the damage occurred on any face is the first type of damage, and the ratio of the damage area of the damage occurred on any face to the total area of any face is less than a first proportion threshold. The so-called first type of damage can generally include but is not limited to the following types of damage: break, corrosion, deformation, etc. For the first type of damage, since the damage area ratio is small, each damaged picture in the to-be-matched set is also associated with an associated picture. Then, the computer device can determine the damage identifier of the jth damaged picture according to the damage feature corresponding to the target face group, which can include:

[0154] ①Obtain the associated picture corresponding to the jth damaged picture.

[0155] For example, the left face, the right face, the front face, and the top face of the corrugated plate often have deformation, break, corrosion, and other damage conditions, and some breaks are not only one place. In order to shoot the tiny break, it needs to be shot at a very close distance, which leads to the inability to accurately identify the component category. Therefore, for the first type of damage corresponding to the target face group, a large picture (damaged picture) and a detailed picture (associated picture) obtained by shooting a plurality of tiny breaks on the corrugated plate in detail are usually shot, please refer to Figure 9a , Figure 9aThis is a schematic diagram of a damaged image and related images provided in an embodiment of this application, such as... Figure 9a As shown, image 901 is a damaged image, and image 902 is a related image of damaged image 901. In practice, identifying the damaged image determines the location of the damage (e.g., left, right, front, or top). It should be noted that the location determined by identifying the damaged image is approximate; that is, identifying the damaged image can at least distinguish whether the damage is on the front, top, or left / right. However, if the identified damaged image is on the left / right, it may not be possible to accurately determine whether the damage occurred on the left or right. In this case, after identifying the component and location code of the first image (damaged image) and matching it with the corresponding maintenance entry, the process continues to match the damage type on subsequent images with minor breaks (related images).

[0156] ②Use the damage identification algorithm corresponding to the first type of damage to identify the damage in the associated image corresponding to the j-th damaged image, and determine the damage marker on the damaged area in the j-th damaged image.

[0157] To determine the type of damage, it is necessary to use the damage identification algorithm corresponding to the first type of damage. Please refer to [link / reference]. Figure 9b , Figure 9b This is a schematic diagram illustrating damage identification of a damaged image provided in an embodiment of this application. For example, if the first type of damage is a tear, the corresponding damage identification algorithm can be a tear identification algorithm, such as... Figure 9b As shown in Figure 903, a breakage identification algorithm can be used to identify related images. Therefore, the breakage marker "BR" can be determined for the broken area in the j-th broken image. For example, if the first type of damage is corrosion, the corresponding breakage identification algorithm can be a corrosion identification algorithm, such as... Figure 9b As shown in Figure 904, a corrosion recognition algorithm can be used to identify associated images. Therefore, the damage marker on the damaged area in the j-th damaged image can be identified as "CO". For example, if the first type of damage is deformation, the corresponding damage recognition algorithm can be a deformation recognition algorithm, such as... Figure 9b As shown in Figure 905, the deformation recognition algorithm can be used to identify the associated images. Therefore, the damage marker "BT" can be determined to be the damage marker on the damaged area in the j-th damaged image.

[0158] In the embodiments of the present application, a plurality of deep learning algorithms (such as a damage identification algorithm corresponding to the first type of damage, a damage identification algorithm corresponding to the second type of damage, etc.) are combined to identify damage information (including damage information of maintenance items and damage information of damage pictures), so that the maintenance items and the damage pictures can be intelligently matched based on the damage information of the maintenance items and the damage information of the damage pictures. For example, it can include detection and identification of position codes, component codes and various types of damage (such as deformation, breakage, rust, dirt, odor, etc.) in the damage pictures. Compared with manual screening and identification of position codes in damage pictures, the deep learning algorithm used in the present application has better identification effect and generalization for the damage pictures taken.

[0159] (2) If the target face group includes a third face group (D), the damage characteristics corresponding to the third face group include: the damage type occurring on any face is the first type of damage, the components on the door panel face include adhesive tape and other components, and the component area of the adhesive tape on the door panel face is less than a second proportion threshold. That is to say, the area proportion of the adhesive tape on the door panel face is small. The first proportion threshold and the second proportion threshold can be the same or different. Specifically, the door panel of the target box body can contain various components, such as adhesive tape, door handle, door lock rod, large and small brackets, and retainer, etc. For example, as shown in FIG. 1, the door panel of the target box body can contain various components, such as adhesive tape, door handle, door lock rod, large and small brackets, and retainer, etc. Figure 10a , Figure 10a is a schematic diagram of a door panel of a target box body provided by the embodiments of the present application. Each component can correspond to a component code, so when matching the components on the door panel with the maintenance items, the damage picture corresponding to the maintenance item can be determined according to the identification result of the position code and the component identification result in the damage picture. In addition, the same component can be in a plurality of different damage pictures, so the pixel proportion of the component in the damage picture is not less than a certain threshold (i.e., the second proportion threshold).

[0160] In a possible implementation, the jth damage picture is composed of a red channel image, a blue channel image and a green channel image; the door panel face includes a left door and a right door, and the damage area identifier corresponding to the jth damage picture includes a left door identifier and a right door identifier. The computer device determines the damage area identifier of the damage occurring on the jth damage picture, which can include: first, replacing the blue channel image in the jth damage picture with the component identification image of the jth damage picture. Then, inputting the replaced jth damage picture into the classification network model for classification processing to determine whether the jth damage picture is taken for the left door or the right door. Finally, adding the left door identifier or the right door identifier to the damage area identifier corresponding to the jth damage picture.

[0161] In a specific implementation, when the position code recognition of the jth damaged picture fails (including that the position code cannot be recognized or that there is no handwritten position code in the jth damaged picture), the classification network model is first called to recognize the left door or the right door in the jth damaged picture. Further, based on the result obtained after recognition, the maintenance items of the corresponding door lock rod and other components on the door plate are matched, and therefore the distinction between the left door and the right door is particularly important for the matching between the damaged picture and the maintenance item, and thus a classification network model for recognizing the left door and the right door is designed.

[0162] Since the component categories and distributions on the left door and the right door are generally similar, the components or areas on the two sides of the door plate can be used for distinction, such as the visual clues of the angle column and the hinge plate on the left side of the left door and the angle column and the hinge plate on the right side of the right door. In order to fully utilize these information to better distinguish whether the door plate is a left door or a right door, the input structure of the conventional classification algorithm can be improved to obtain the classification network model provided in the embodiments of the present application. In the process of using the classification network model, the B (Blue) channel image in the three-channel image of the jth damaged picture is replaced by the component recognition image after component recognition, and in this way, the clues of other components and areas can be explicitly increased. Further, the structure of the classification network model provided in the embodiments of the present application can adopt the structure of a residual network (Residual Neural Network, ResNet) (such as the structure of the network resnet18). Through the recognition of the classification neural network model, the photographed door plate picture (the jth damaged picture) can be classified into four categories, i.e., the left door, the right door, the door plate (unable to distinguish the left door and the right door), and both doors (including the left door and the right door). Please refer to Figure 10b , Figure 10b which is a structural schematic diagram of the classification network model provided in the embodiments of the present application. As shown in Figure 10b , the B (Blue) channel image in the jth damaged picture is replaced by the component recognition image of the jth damaged picture, and then the replaced jth damaged picture is input into the classification network model for recognition processing, so as to obtain the classification result of the jth damaged picture. The classification network model provided in the embodiments of the present application can include a feature extraction (resent banckbone) module and a classification (fc, 2048 and fc, 4) module. It should be noted that the classification network model provided in the embodiments of the present application can adopt the structure of other neural networks (such as a convolutional neural network) in addition to the residual network structure, which is not limited in the embodiments of the present application.

[0163] In a possible implementation, the computer device determines the damage identifier of the jth damaged picture according to the damage characteristics corresponding to the target face group, which can include the following steps:

[0164] ①The first recognition result is obtained by performing adhesive tape damage recognition on the jth damaged picture, and the first recognition result is used to indicate whether the adhesive tape is damaged or not damaged.

[0165] In a specific implementation, the components on the door panel surface include adhesive tapes. The computer device performs adhesive tape damage recognition on the jth damaged picture to obtain the first recognition result, which can include the following steps. First, a double-branch attention model is obtained, the double-branch attention model includes an adhesive tape edge line segmentation and recognition branch and an adhesive tape damage classification branch, the adhesive tape edge line segmentation and recognition branch includes a first feature extraction module, and the adhesive tape damage classification branch includes a second feature extraction module and a classification processing module. Then, the computer device performs mask processing on the jth damaged picture to obtain a mask image, and the mask image only includes information of image elements corresponding to the adhesive tapes. Next, the computer device calls the first feature extraction module to perform feature extraction on the mask image to obtain edge line segmentation features of the adhesive tapes, and calls the second feature extraction module to perform feature extraction on the mask image to obtain damage classification features of the adhesive tapes. Finally, the computer device performs splicing processing on the edge line segmentation features and the damage classification features, and calls the classification processing module to perform classification processing based on the splicing processing result to obtain the first recognition result.

[0166] In the foregoing manner, the embodiment of the present application further designs a method for classifying the door panel as a left door or a right door in the left-right door recognition algorithm, thereby improving the accuracy of classifying the door panel as a left door or a right door. The method includes replacing the blue channel image of the input damaged picture with a component recognition image of the component recognition, and explicitly adding a clue of different adjacent regions and components of the left door and the right door, thereby improving the accuracy of position matching of the components on the door.

[0167] Since the adhesive tapes around the door panel can be damaged, and when the top edge adhesive tape of the door panel is damaged, the proportion of the adhesive tape in the damaged picture is small, which makes it difficult to classify the proportion and damage of the adhesive tape. Please refer to Figure 10c , Figure 10c is a schematic diagram of an adhesive tape on a door panel, as shown in Figure 10c , the adhesive tape on the door panel is located at the dashed box s10, and it can be seen that the proportion of the adhesive tape in the damaged picture is small. In addition, the input of the double-branch attention model is a mask image obtained by performing mask processing on the jth damaged picture, wherein the mask image only includes information of image elements corresponding to the adhesive tapes. This means that first, the regions other than the adhesive tapes in the component recognition image of the jth damaged picture can be set to 0, so that only the relevant information of the adhesive tapes is retained in the component recognition image of the jth damaged picture, as shown in Figure 10d , Figure 10dis a schematic diagram of a mask picture of a broken picture provided by an embodiment of the present application. Secondly, since the normal adhesive tape edge is smooth and straight, and the broken adhesive tape edge is usually different in shape and uneven, based on this visual feature, an embodiment of the present application designs a network structure of double branches and fusion edge line segmentation attention (attention) features (i.e. a double-branch attention model), which further improves the recognition accuracy of adhesive tape breakage.

[0168] Therefore, for the case that the adhesive tape occupies a small proportion in the broken picture, in order to more accurately identify whether the adhesive tape is broken, an embodiment of the present application provides a double-branch attention model for whether the adhesive tape is broken. Please refer to Figure 10e , Figure 10e is a structural schematic diagram of a double-branch attention model provided by an embodiment of the present application. The double-branch attention model shown in Figure 10e may be a high resolution network (hrnet) model, wherein the so-called hrnet model is a network structure that can simultaneously retain high resolution and low resolution feature maps to better achieve detailed segmentation results. As shown in Figure 10e The double-branch attention model can include an adhesive tape edge line segmentation and identification branch 100 and an adhesive tape breakage classification branch 200. The adhesive tape edge line segmentation and identification branch 100 can include a first feature extraction module 1001, and the adhesive tape breakage classification branch 200 can include a second feature extraction module 2001 and a classification processing module 2002. The structures of the first feature extraction module 1001 and the second feature extraction module 2001 can be the same or different. Moreover, the first feature extraction module 1001 and the second feature extraction module 2001 can be composed of multiple convolution layers (Conv layers). It should be noted that the first identification result obtained by calling the double-branch attention model to identify the adhesive tape breakage of the jth broken picture can not only indicate whether the adhesive tape is broken or not, but also include the segmentation result of the adhesive tape edge line, for example, the segmentation result of the adhesive tape edge line indicating whether the edge of the adhesive tape is smooth or broken.

[0169] In summary, based on the double-branch attention model provided by an embodiment of the present application, the segmentation branch of the adhesive tape edge line and the binary classification branch of whether the adhesive tape is broken are designed, wherein the adhesive tape edge line segmentation branch can promote the network to learn the features of the adhesive tape edge and splice them to the middle feature map of the adhesive tape breakage classification branch. The attention mechanism can further explicitly increase the features of the smooth or broken adhesive tape edge to the classification branch of whether the adhesive tape is broken, so as to further improve the classification accuracy of adhesive tape breakage.

[0170] ② adopt the damage identification algorithm corresponding to the first type of damage to identify the damage of other components on the jth damaged picture to obtain a second identification result, and the second identification result is used to indicate whether damage of other components occurs on the jth damaged picture.

[0171] In a specific implementation, for damage identification of other components (such as a door handle, a door lock rod, a large bracket, a small bracket, a retainer, and the like) on the door panel, the embodiments of the present application can still adopt the damage identification algorithm corresponding to the first type of damage (such as a damage opening identification algorithm, a rust identification algorithm, a deformation identification algorithm, and the like) to identify the damage of other components to obtain a second identification result of other components. For example, the second identification result can include that the door handle deforms or the door lock rod rusts, and the like.

[0172] ③ determine the damage identifier corresponding to the jth damaged picture according to the first identification result and the second identification result. For example, if it is identified by the above method that the adhesive tape in the jth damaged picture has a damage opening, it can be determined that the damage identifier corresponding to the jth damaged picture is the damage identifier of the adhesive tape damage, which is "CO"; for another example, if it is identified by the above method that the door handle in the jth damaged picture deforms, it can be determined that the damage identifier corresponding to the jth damaged picture is (BT).

[0173] (3) If the target face group includes the fourth face group (BU), and the damage characteristics corresponding to the fourth face group include that the damage category occurring on any face in the fourth face group includes one or more of the first type of damage and the second type of damage. Then, the computer device can determine the damage identifier of the jth damaged picture according to the damage characteristics corresponding to the target face group, which can include:

[0174] ① respectively adopt the damage identification algorithm corresponding to the first type of damage and the damage identification algorithm corresponding to the second type of damage to identify the damage of the jth damaged picture to obtain a third identification result and a fourth identification result; the third identification result is used to indicate whether the damage belonging to the first type of damage exists on the jth damaged picture, and the fourth identification result is used to indicate whether the damage belonging to the second type of damage exists on the jth damaged picture.

[0175] In a specific implementation, in addition to the first type of damage (such as a damage opening, deformation, rust, and the like) on the floor (including an inner floor and a bottom floor), there are more dirt, oil stains, and odors (which can be referred to as the second type of damage). Then, for the first type of damage, the damage identification algorithm corresponding to the first type of damage (such as a damage opening identification algorithm, a rust identification algorithm, a deformation identification algorithm, and the like) can be adopted to identify the damage of the jth damaged picture. Please refer to Figure 11 , Figure 11 is a schematic diagram of various types of damage on a floor provided by the embodiments of the present application. As shown in Figure 11As shown, dirt identification can be performed on the picture 1101 by the damage identification algorithm corresponding to the second type of damage (such as the dirt identification algorithm); oil stain identification can be performed on the picture 1102 by the damage identification algorithm corresponding to the second type of damage (such as the oil stain identification algorithm); and dirt identification can be performed on the picture 1103 by the damage identification algorithm corresponding to the first type of damage (such as the damage identification algorithm). It should be noted that in the case where the damaged picture has an odor, when the target box is being tested by the box tester, if there is an odor on the surface (such as the floor surface), the box tester can mark the odor on the surface as "OR", which means that when the jth damaged picture is identified, if the "OR" mark in the damaged picture is identified, it can be determined that the damage corresponding to the damaged picture is an odor damage. The damage identification algorithm corresponding to the first type of damage and the damage identification algorithm corresponding to the second type of damage can use the network structure model of resnet18 for classification identification.

[0176] 2. According to the third identification result and the fourth identification result, the damage mark corresponding to the jth damaged picture is determined. For example, if it is identified by the above method that the jth damaged picture has a breach, it can be determined that the damage mark corresponding to the jth damaged picture is (BR); for another example, if it is identified by the above method that the jth damaged picture has an odor, it can be determined that the damage mark corresponding to the jth damaged picture is (OR).

[0177] Finally, based on the above description, the complete process of the damage matching method provided by the embodiments of the present application is briefly described in combination with the flowchart shown in the following Figure 12 Figure 12 Figure 12 is a principle diagram of a damage matching method provided by the embodiments of the present application. As shown in Figure 12 ​​As shown, first, m maintenance entries and n damage pictures of the target container can be acquired, and then when damage matching is performed on the maintenance entries and the damage pictures, the m maintenance entries and the n damage pictures can be set-partitioned to obtain a plurality of to-be-matched sets, which may, for example, include a corrugated plate set, a door plate and component set, a floor and bottom beam set, and a dirt and odor set. One to-be-matched set here can correspond to one face group described above, for example, the corrugated plate set can correspond to the first and second face groups described above; the door plate and component set can correspond to the third face group described above; the floor and bottom beam set can correspond to the fourth face group described above, and so on. Then, based on the maintenance entries and the damage pictures in each to-be-matched set, accurate matching between each damage picture and maintenance entry is respectively performed (such as damage matching in the corrugated plate set, damage matching in the door plate and component set, damage matching in the floor and bottom beam set, and damage matching in the dirt and odor set), until a damage picture uniquely matched with each maintenance entry is determined, thereby obtaining one damage maintenance relationship group. In this way, a plurality of damage maintenance relationship groups about the target container can be obtained.

[0178] In the embodiments of the present application, in the process of inspecting the target container, thousands of damage pictures corresponding to each target container need to be matched with maintenance entries. Through joint position code recognition, component recognition, and various damage recognition (such as damage identification, deformation identification, and rust identification) algorithms, the maintenance entries and the damage pictures can be intelligently matched, greatly saving manpower and time, and thus improving the efficiency of damage matching. Further, according to the direct similarity and relevance of the maintenance entries and the damage pictures, a coarse-to-fine fine matching strategy is designed. First, a rough classification is performed according to the position code and the component code; then the range is further narrowed down, and more detailed position code specific area, component category, and damage type information are accurately matched for each category. Compared with matching each damage picture and maintenance entry one by one, the embodiments of the present application further improve the efficiency of damage matching.

[0179] Please refer to Figure 13 , Figure 13 is a structural schematic diagram of an image processing device provided by an embodiment of the present application. The image processing device 1300 can be applied to a computer device in the corresponding method embodiments described above. The image processing device 1300 can be a computer program (including program code) running in the computer device, for example, the image processing device 1300 is an application software; the device can be used to execute the corresponding steps in the method provided by the embodiments of the present application. The image processing device 1300 can include:

[0180] The acquisition unit 1301 is configured to acquire m maintenance entries and n damage pictures of a target box body, where m and n are positive integers.

[0181] The acquisition unit 1301 is further configured to acquire damage information reflected by each maintenance entry in the m maintenance entries, and acquire damage information reflected by each damage picture in the n damage pictures.

[0182] The processing unit 1302 is configured to perform damage matching on the m maintenance entries and the n damage pictures based on the damage information reflected by each maintenance entry and the damage information reflected by each damage picture, to obtain a plurality of damage maintenance relationship groups, each of which includes one maintenance entry and at least one damage picture, and the damage information reflected by the maintenance entry in each damage maintenance relationship group matches the damage information reflected by the damage picture.

[0183] In a possible implementation, the damage information includes a damage position and damage detail information of damage occurring on the target box body.

[0184] The processing unit 1302 is configured to perform the following operations when performing damage matching on the m maintenance entries and the n damage pictures based on the damage information reflected by each maintenance entry and the damage information reflected by each damage picture to obtain the plurality of damage maintenance relationship groups.

[0185] The maintenance entries and the damage pictures having similar damage positions are divided into a set to obtain W to-be-matched sets, where W is a positive integer.

[0186] In one to-be-matched set, the maintenance entries and the damage pictures having the same damage detail information are determined as one damage maintenance relationship group.

[0187] In a possible implementation, the target box body includes a left face, a right face, a front face, a top face, a door panel face, an inner floor face, and a bottom floor face, each of which corresponds to a face identifier; the left face, the right face, and the front face are divided into a first face group, the top face is divided into a second face group, the door panel face is divided into a third face group, and the inner floor face and the bottom floor face are divided into a fourth face group.

[0188] The damage position includes the face identifier, and the i th maintenance entry and the j th damage picture have similar damage positions means that the face indicated by the face identifier in the damage position of the i th maintenance entry and the face indicated by the face identifier in the damage position of the j th damage picture belong to the same face group, i is a positive integer and i≤m, and j is a positive integer and j≤n.

[0189] In a possible implementation, the damage position of the j th damage picture is obtained by performing damage position identification processing on the j th damage picture, and the processing unit 1302 is further configured to perform the following operations:

[0190] performing position code recognition on the jth damaged picture, and if the position code recognition is successful, determining a face identifier in the recognized position code as the face identifier corresponding to the jth damaged picture;

[0191] if the position code recognition fails, performing component recognition on the jth damaged picture, and determining that the jth damaged picture is obtained by photographing a target component on the target box body;

[0192] determining a face identifier corresponding to the target component as the face identifier corresponding to the jth damaged picture.

[0193] In a possible implementation, the processing unit 1302 performs component recognition on the jth damaged picture, and determines that the jth damaged picture is obtained by photographing a target component on the target box body, for performing the following operations:

[0194] performing component recognition processing on the jth damaged picture to obtain a component recognition image, and the component recognition image includes image elements corresponding to multiple components on the target box body;

[0195] determining the target component from the multiple components according to an area occupied by each component in the component recognition image.

[0196] In a possible implementation, the components on the target box body include any one or more of the following: a side plate, a front plate, a corner column, a door lock rod, a door handle, and a bottom cross beam.

[0197] When the processing unit 1302 determines the target component from the multiple components according to an area occupied by each component in the component recognition image, the processing unit 1302 is configured to perform the following operations:

[0198] if the multiple components include the bottom cross beam, and a ratio of the area occupied by the image element corresponding to the bottom cross beam in the component recognition image to a total area of the component recognition image reaches a reference threshold, the target component is determined to be the bottom cross beam.

[0199] if the multiple components do not include the bottom cross beam, a component corresponding to an image element with the largest area in the component recognition image is determined to be the target component.

[0200] In a possible implementation, the jth damaged picture belongs to a kth to-be-matched set, k is a positive integer and k≤W; and the damaged detail information of the jth damaged picture includes a damaged area code and a damaged type.

[0201] When the processing unit 1302 obtains the damaged detail information of the jth damaged picture, the processing unit 1302 is configured to perform the following operations:

[0202] determining a target face group corresponding to the kth to-be-matched set, the face group corresponding to the kth to-be-matched set being a face group to which a face indicated by a face identifier in a damaged position of any one damaged picture in the kth to-be-matched set belongs;

[0203] determining a damaged area code corresponding to the jth damaged picture based on the face identifier corresponding to the jth damaged picture and the damaged area identifier of the damaged area on the jth damaged picture;

[0204] determining the damaged identifier of the jth damaged picture according to the damaged feature corresponding to the target face group.

[0205] In a possible implementation, the target face group includes a first face group or a second face group, and the damaged feature corresponding to the first face group or the second face group includes: the damaged type of any face on which damage occurs is a first type of damage, and the ratio of the damaged area on any face to the total area of the face is less than a first proportion threshold;

[0206] When determining the damaged identifier of the jth damaged picture according to the damaged feature corresponding to the target face group, the processing unit 1302 is configured to perform the following operation:

[0207] obtaining an associated picture corresponding to the jth damaged picture;

[0208] performing damage identification on the associated picture corresponding to the jth damaged picture by using a damage identification algorithm corresponding to the first type of damage to determine the damaged identifier of the damaged area on the jth damaged picture.

[0209] In a possible implementation, the target face group includes a third face group, and the damaged feature corresponding to the third face group includes: the damaged type of any face on which damage occurs is the first type of damage, the components on the door panel face include a rubber strip and other components, and the area of the rubber strip on the door panel face is less than a second proportion threshold;

[0210] When determining the damaged identifier of the jth damaged picture according to the damaged feature corresponding to the target face group, the processing unit 1302 is configured to perform the following operation:

[0211] performing rubber strip damage identification on the jth damaged picture to obtain a first identification result, the first identification result being used to indicate whether the rubber strip is damaged or undamaged;

[0212] performing other component damage identification on the jth damaged picture by using a damage identification algorithm corresponding to the first type of damage to obtain a second identification result, the second identification result being used to indicate whether other components on the jth damaged picture are damaged or undamaged;

[0213] determining the damaged identifier corresponding to the jth damaged picture according to the first identification result and the second identification result.

[0214] In a possible implementation, the jth damaged picture is composed of a red channel picture, a blue channel picture and a green channel picture; the door panel surface includes a left door and a right door, and the damaged area identifier corresponding to the jth damaged picture includes a left door identifier and a right door identifier;

[0215] The processing unit 1302 is configured to perform the following operation when determining the damaged area identifier on which damage occurs in the jth damaged picture:

[0216] replace the blue channel picture in the jth damaged picture with a component identification picture of the jth damaged picture;

[0217] input the replaced jth damaged picture into the classification network model for classification processing, and determine whether the jth damaged picture is obtained by photographing the left door or the right door;

[0218] add the left door identifier or the right door identifier to the damaged area identifier corresponding to the jth damaged picture.

[0219] In a possible implementation, the components on the door panel surface include a rubber strip;

[0220] The processing unit 1302 is configured to perform the following operation when performing rubber strip damage identification on the jth damaged picture to obtain a first identification result:

[0221] obtain a double-branch attention model, the double-branch attention model including a rubber strip edge line segmentation and identification branch and a rubber strip damage classification branch, the rubber strip edge line segmentation and identification branch including a first feature extraction module, and the rubber strip damage classification branch including a second feature extraction module and a classification processing module;

[0222] perform mask processing on the jth damaged picture to obtain a mask picture, the mask picture including only information of image elements corresponding to the rubber strip;

[0223] call the first feature extraction module to perform feature extraction on the mask picture to obtain edge line segmentation features of the rubber strip, and call the second feature extraction module to perform feature extraction on the mask picture to obtain damage classification features of the rubber strip;

[0224] perform splicing processing on the edge line segmentation features and the damage classification features, and call the classification processing module to perform classification processing based on a splicing processing result to obtain the first identification result.

[0225] In a possible implementation, the target surface group corresponding to the kth to-be-matched set includes a fourth surface group, and the damage characteristics corresponding to the fourth surface group include that the damage category occurring on any surface in the fourth surface group includes one or more of a first type of damage and a second type of damage;

[0226] The processing unit 1302 is configured to perform the following operations when determining the damage identifier of the jth damage picture according to the damage characteristics corresponding to the target face group:

[0227] The damage recognition algorithm corresponding to the first type of damage and the damage recognition algorithm corresponding to the second type of damage are respectively used to perform damage recognition on the jth damage picture to obtain a third recognition result and a fourth recognition result; the third recognition result is used to indicate whether there is damage belonging to the first type of damage on the jth damage picture, and the fourth recognition result is used to indicate whether there is damage belonging to the second type of damage on the jth damage picture.

[0228] The damage identifier corresponding to the jth damage picture is determined according to the third recognition result and the fourth recognition result.

[0229] In the embodiments of the present application, after m maintenance entries and n damage pictures of a target box body are obtained, m and n are positive integers; damage information reflected by each maintenance entry in the m maintenance entries can be obtained, and damage information reflected by each damage picture in the n damage pictures can be obtained; and the m maintenance entries and the n damage pictures are matched based on the damage information reflected by each maintenance entry and the damage information reflected by each damage picture, to obtain a plurality of damage maintenance relationship groups. Each damage maintenance relationship group includes one maintenance entry and at least one damage picture, and the damage information reflected by the maintenance entry in each damage maintenance relationship group and the damage information reflected by the damage picture match. It can be seen that, compared with manual matching, the maintenance entries and the damage pictures can be intelligently matched based on the damage information reflected by the maintenance entries and the damage information reflected by the damage pictures in the present application, so that the efficiency and accuracy of matching the maintenance entries and the damage pictures can be improved.

[0230] Please refer to Figure 14 , Figure 14 is a structural schematic diagram of a computer device provided by the embodiments of the present application. The computer device 1400 is used to execute the steps performed by the computer device in the corresponding method embodiments described above, and the computer device 1400 includes one or more processors 1410, one or more input devices 1420, one or more output devices 1430 and a memory 1440. The above-mentioned processor 1410, input device 1420, output device 1430 and memory 1440 are connected through a bus 1450. The memory 1440 is used to store a computer program, and the computer program includes program instructions. The processor 1410 is used to call the program instructions stored in the memory 1440 to perform the following operations:

[0231] obtain m maintenance entries and n damage pictures of a target box body, m and n are positive integers;

[0232] obtaining damage information reflected by each of the m maintenance entries, and obtaining damage information reflected by each of the n damaged pictures;

[0233] matching the m maintenance entries and the n damaged pictures based on the damage information reflected by each of the maintenance entries and the damage information reflected by each of the damaged pictures, to obtain a plurality of damaged maintenance relationship groups, each of which includes one maintenance entry and at least one damaged picture, and the damage information reflected by the maintenance entry in each of the damaged maintenance relationship groups matches the damage information reflected by the damaged picture.

[0234] In a possible implementation, the damage information includes a damage position and damage detail information of the damage on the target box body.

[0235] When the processor 1410 matches the m maintenance entries and the n damaged pictures based on the damage information reflected by each of the maintenance entries and the damage information reflected by each of the damaged pictures to obtain a plurality of damaged maintenance relationship groups, the processor 1410 is configured to perform the following operations:

[0236] dividing the maintenance entries and the damaged pictures having similar damage positions into a set to obtain W to-be-matched sets, W being a positive integer;

[0237] In one to-be-matched set, the maintenance entries and the damaged pictures having the same damage detail information are determined as one damaged maintenance relationship group.

[0238] In a possible implementation, the target box body includes a left face, a right face, a front face, a top face, a door panel face, an inner floor face, and a bottom floor face, each of which corresponds to a face identifier; the left face, the right face, and the front face are divided into a first face group, the top face is divided into a second face group, the door panel face is divided into a third face group, and the inner floor face and the bottom floor face are divided into a fourth face group.

[0239] The damage position includes the face identifier, and the ith maintenance entry and the jth damaged picture have similar damage positions means that the face indicated by the face identifier in the damage position of the ith maintenance entry and the face indicated by the face identifier in the damage position of the jth damaged picture belong to the same face group, i being a positive integer and i≤m, and j being a positive integer and j≤n.

[0240] In a possible implementation, the damage position of the jth damaged picture is obtained by performing damage position recognition processing on the jth damaged picture, and the processor 1410 is further configured to perform the following operations:

[0241] performing position code recognition on the jth damaged picture, and if the position code recognition is successful, determining the face identifier in the recognized position code as the face identifier corresponding to the jth damaged picture;

[0242] If the position code recognition fails, the jth damaged picture is subjected to component recognition to determine that the jth damaged picture is obtained by photographing a target component on the target box body;

[0243] The face identifier corresponding to the target component is determined as the face identifier corresponding to the jth damaged picture.

[0244] In a possible implementation, the processor 1410 subjects the jth damaged picture to component recognition to determine that the jth damaged picture is obtained by photographing a target component on the target box body, and is configured to perform the following operations:

[0245] The jth damaged picture is subjected to component recognition processing to obtain a component recognition image, and the component recognition image includes image elements corresponding to a plurality of components on the target box body;

[0246] The target component is determined from the plurality of components according to an area occupied by each component in the component recognition image.

[0247] In a possible implementation, the components on the target box body include any one or more of the following: a side plate, a front plate, a corner column, a door lock rod, a door handle, and a bottom cross beam.

[0248] The processor 1410 is configured to perform the following operations when determining the target component from the plurality of components according to an area occupied by each component in the component recognition image.

[0249] If the plurality of components include a bottom cross beam, and a ratio of an area occupied by the image element corresponding to the bottom cross beam in the component recognition image to a total area of the component recognition image reaches a reference threshold, the target component is determined to be the bottom cross beam.

[0250] If the plurality of components do not include a bottom cross beam, a component corresponding to an image element with the largest area in the component recognition image is determined to be the target component.

[0251] In a possible implementation, the jth damaged picture belongs to a kth to-be-matched set, k is a positive integer and k≤W; and the damaged detail information of the jth damaged picture includes a damaged area code and a damaged type.

[0252] The processor 1410 is configured to perform the following operations when obtaining the damaged detail information of the jth damaged picture.

[0253] A target face group corresponding to the kth to-be-matched set is determined, and the face group corresponding to the kth to-be-matched set is a face group to which a face indicated by a face identifier in a damaged position of any one damaged picture in the kth to-be-matched set belongs.

[0254] determine the damage area code corresponding to the jth damaged picture based on the face identifier corresponding to the jth damaged picture and the damage area identifier of the damage area on the jth damaged picture;

[0255] determine the damage identifier of the jth damaged picture according to the damage feature corresponding to the target face group.

[0256] In a possible implementation, the target face group includes a first face group or a second face group, and the damage feature corresponding to the first face group and the second face group includes: the damage type of any face on which damage occurs is a first type of damage, and the ratio of the damage area on any face on which damage occurs to the total area of any face is less than a first proportion threshold;

[0257] When determining the damage identifier of the jth damaged picture according to the damage feature corresponding to the target face group, the processor 1410 is configured to perform the following operations:

[0258] obtain an associated picture corresponding to the jth damaged picture;

[0259] perform damage identification on the associated picture corresponding to the jth damaged picture by using a damage identification algorithm corresponding to the first type of damage to determine the damage identifier of the damage area on which damage occurs in the jth damaged picture.

[0260] In a possible implementation, the target face group includes a third face group, and the damage feature corresponding to the third face group includes: the damage type of any face on which damage occurs is a first type of damage, the components on the door panel face include a tape and other components, and the component area of the tape on the door panel face is less than a second proportion threshold;

[0261] When determining the damage identifier of the jth damaged picture according to the damage feature corresponding to the target face group, the processor 1410 is configured to perform the following operations:

[0262] perform tape damage identification on the jth damaged picture to obtain a first identification result, and the first identification result is used to indicate whether the tape is damaged or not damaged;

[0263] perform other component damage identification on the jth damaged picture by using a damage identification algorithm corresponding to the first type of damage to obtain a second identification result, and the second identification result is used to indicate whether other components on the jth damaged picture are damaged or not damaged;

[0264] determine the damage identifier corresponding to the jth damaged picture according to the first identification result and the second identification result.

[0265] In a possible implementation, the jth damaged picture is composed of a red channel picture, a blue channel picture, and a green channel picture; the door panel face includes a left door and a right door, and the damage area identifier corresponding to the jth damaged picture includes a left door identifier and a right door identifier;

[0266] The processor 1410 is configured to perform the following operations when determining the damage area identifier of the jth damaged picture on which damage occurs:

[0267] replace the blue channel picture in the jth damaged picture with a component identification picture of the jth damaged picture;

[0268] input the replaced jth damaged picture into the classification network model for classification processing to determine whether the jth damaged picture is obtained by photographing the left door or the right door;

[0269] add the left door identifier or the right door identifier to the damage area identifier corresponding to the jth damaged picture.

[0270] In a possible implementation, the components on the door panel surface include a rubber strip;

[0271] The processor 1410 is configured to perform the following operations when performing rubber strip damage identification on the jth damaged picture to obtain a first identification result:

[0272] obtain a double-branch attention model, the double-branch attention model including a rubber strip edge line segmentation and identification branch and a rubber strip damage classification branch, the rubber strip edge line segmentation and identification branch including a first feature extraction module, and the rubber strip damage classification branch including a second feature extraction module and a classification processing module;

[0273] perform mask processing on the jth damaged picture to obtain a mask picture, the mask picture including only information of image elements corresponding to the rubber strip;

[0274] call the first feature extraction module to perform feature extraction on the mask picture to obtain edge line segmentation features of the rubber strip, and call the second feature extraction module to perform feature extraction on the mask picture to obtain damage classification features of the rubber strip;

[0275] perform splicing processing on the edge line segmentation features and the damage classification features, and call the classification processing module to perform classification processing based on a splicing processing result to obtain the first identification result.

[0276] In a possible implementation, the target face group corresponding to the kth to-be-matched set includes a fourth face group, and the damage characteristics corresponding to the fourth face group include that the damage category occurring on any face in the fourth face group includes one or more of a first type of damage and a second type of damage.

[0277] The processor 1410 is configured to perform the following operations when determining the damage identifier of the jth damaged picture according to the damage characteristics corresponding to the target face group:

[0278] The jth damage picture is subjected to damage identification by using the first type of damage corresponding damage identification algorithm and the second type of damage corresponding damage identification algorithm respectively, to obtain a third identification result and a fourth identification result; the third identification result is used to indicate whether there is damage belonging to the first type of damage on the jth damage picture, and the fourth identification result is used to indicate whether there is damage belonging to the second type of damage on the jth damage picture;

[0279] The damage identifier corresponding to the jth damage picture is determined according to the third identification result and the fourth identification result.

[0280] In the embodiment of the present application, after the m maintenance entries and the n damage pictures of the target box body are obtained, m and n are positive integers; the damage information reflected by each maintenance entry in the m maintenance entries can be obtained, and the damage information reflected by each damage picture in the n damage pictures can be obtained; and the m maintenance entries and the n damage pictures are subjected to damage matching based on the damage information reflected by each maintenance entry and the damage information reflected by each damage picture, to obtain a plurality of damage maintenance relationship groups. Each damage maintenance relationship group includes one maintenance entry and at least one damage picture, and the damage information reflected by the maintenance entry in each damage maintenance relationship group and the damage information reflected by the damage picture match. It can be seen that, compared with manual matching, the damage information reflected by the maintenance entry and the damage information reflected by the damage picture can be used to realize intelligent damage matching between the maintenance entry and the damage picture in the present application, so that the efficiency and accuracy of matching the maintenance entry and the damage picture can be improved.

[0281] In addition, it should be noted here that the embodiment of the present application also provides a computer storage medium, and the computer storage medium stores a computer program, and the computer program includes program instructions. When the processor executes the above-mentioned program instructions, the method in the corresponding embodiment described above can be executed, and therefore, the description will not be repeated here. For technical details not disclosed in the computer storage medium embodiments of the present application, please refer to the description of the method embodiments of the present application. As an example, the program instructions can be deployed on one computer device, or executed on multiple computer devices located in one place, or executed on multiple computer devices distributed in multiple places and interconnected through a communication network.

[0282] According to one aspect of the present application, a computer program product or computer program is provided, which includes a computer program stored in a computer readable storage medium. The processor of the computer device reads the computer program from the computer readable storage medium, and the processor executes the computer program, so that the computer device can execute the method in the corresponding embodiment described above, and therefore, the description will not be repeated here.

[0283] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The above-mentioned program can be stored in a computer readable storage medium. When the program is executed, the program can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), or the like.

[0284] The above disclosure is merely preferred embodiments of the present application and is not intended to limit the scope of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope of the present application.

Claims

1. An image processing method, characterized in that, The method includes: Obtain m maintenance entries and n damage images of the target enclosure, where m and n are both positive integers; the target enclosure includes a left side, right side, front side, top side, door panel side, inner floor surface, and bottom floor surface, each side corresponding to a face identifier; the left side, right side, and front side are assigned to the first face group, the top side to the second face group, the door panel side to the third face group, and the inner floor surface and bottom floor surface to the fourth face group; Obtain the damage information reflected in each of the m maintenance entries, and obtain the damage information reflected in each of the n damage images; the damage information includes the damage location and damage details on the target box, and the damage location includes a surface marker; Maintenance entries and damaged images with similar damage locations are grouped into a set, resulting in W sets to be matched, where W is a positive integer; where the i-th maintenance entry and the j-th damaged image have similar damage locations means that the face indicated by the face marker in the damage location of the i-th maintenance entry belongs to the same face group as the face indicated by the face marker in the damage location of the j-th damaged image, where i is a positive integer and i≤m, and j is a positive integer and j≤n; In a set to be matched, maintenance items with the same damage details and damage images are identified as a damage repair relationship group; Each damage repair relationship group includes one maintenance entry and at least one damage image. The damage information reflected by the maintenance entry and the damage information reflected by the damage image in each damage repair relationship group are matched.

2. The method as described in claim 1, characterized in that, The damage location of the j-th damaged image is obtained by performing damage location identification processing on the j-th damaged image, and the method further includes: The j-th damaged image is subjected to location code recognition. If the location code recognition is successful, the face identifier in the recognized location code is determined as the face identifier corresponding to the j-th damaged image. If the location code recognition fails, then component recognition is performed on the j-th damaged image to determine that the j-th damaged image was obtained by taking a picture of the target component on the target box. The face identifier corresponding to the target component is determined as the face identifier corresponding to the j-th damaged image.

3. The method as described in claim 2, characterized in that, The step of identifying components in the j-th damaged image to determine that the j-th damaged image is obtained by photographing a target component on the target box includes: The j-th damaged image is subjected to component identification processing to obtain a component identification image, which includes image elements corresponding to multiple components on the target box. The target component is determined from the plurality of components based on the area occupied by the image element corresponding to each component in the component recognition image.

4. The method as described in claim 3, characterized in that, The components on the target enclosure include any one or more of the following: side panels, front panels, corner posts, door lock rods, door handles, and bottom crossbeams; the process of determining the target component from the plurality of components based on the area occupied by the image element corresponding to each component in the component recognition image includes: If the plurality of components includes a bottom crossbeam, and the ratio of the area occupied by the image element corresponding to the bottom crossbeam in the component recognition image to the total area of ​​the component recognition image reaches a reference threshold, then the target component is determined to be the bottom crossbeam. If the bottom crossbeam is not included among the multiple components, then the component corresponding to the image element with the largest area in the component identification image is determined as the target component.

5. The method as described in claim 2, characterized in that, The j-th damaged image belongs to the k-th set to be matched, where k is a positive integer and k≤W; the damage details of the j-th damaged image include the damaged area code and the damage type; Obtain the damage details information of the j-th damaged image, including: Determine the target face group corresponding to the kth set to be matched. The face group corresponding to the kth set to be matched refers to the face group to which the face indicated by the face identifier at the damaged position of any damaged image in the kth set to be matched belongs. The damaged area code corresponding to the j-th damaged image is determined based on the face identifier corresponding to the j-th damaged image and the damaged area identifier on the j-th damaged image. Based on the damage characteristics corresponding to the target surface group, the damage identifier of the j-th damaged image is determined.

6. The method as described in claim 5, characterized in that, The target surface group includes the first surface group or the second surface group. The damage characteristics corresponding to the first surface group and the second surface group include: the damage type occurring on any surface is the first type of damage, and the ratio of the area of ​​the damaged area occurring on any surface to the total area of ​​the surface is less than a first ratio threshold. The step of determining the damage identifier of the j-th damaged image based on the damage characteristics corresponding to the target surface group includes: Obtain the associated image corresponding to the j-th damaged image; The damage identification algorithm corresponding to the first type of damage is used to identify the damage in the associated image corresponding to the j-th damaged image, and the damage markers on the damaged area in the j-th damaged image are determined.

7. The method as described in claim 5, characterized in that, The target surface group includes the third surface group, and the damage characteristics corresponding to the third surface group include: the damage type occurring on any surface is the first type of damage, the components on the door panel include adhesive strips and other components, and the area of ​​the adhesive strips on the door panel is less than the second proportion threshold. The step of determining the damage identifier of the j-th damaged image based on the damage characteristics corresponding to the target surface group includes: The j-th damaged image is subjected to adhesive strip damage identification to obtain a first identification result, which is used to indicate whether the adhesive strip is damaged or not. The damage identification algorithm corresponding to the first type of damage is used to identify damage to other components in the j-th damaged image to obtain a second identification result. The second identification result is used to indicate whether other components in the j-th damaged image are damaged or not. The damage identifier corresponding to the j-th damaged image is determined based on the first recognition result and the second recognition result.

8. The method as described in claim 7, characterized in that, The j-th damaged image is composed of a red channel image, a blue channel image, and a green channel image; the door panel includes a left door and a right door, and the damaged area markings corresponding to the j-th damaged image include left door markings and right door markings; Determining the identifier of the damaged area on the j-th damaged image includes: Replace the blue channel image in the j-th damaged image with the component identification image of the j-th damaged image; The replaced j-th damaged image is input into the classification network model for classification processing to determine whether the j-th damaged image was taken from the left door or the right door; Add the left door identifier or the right door identifier to the damaged area identifier corresponding to the j-th damaged image.

9. The method as described in claim 8, characterized in that, The components on the door panel include adhesive strips; the process of identifying adhesive strip damage in the j-th damaged image to obtain a first identification result includes: A dual-branch attention model is obtained, which includes a rubber strip edge line segmentation and recognition branch and a rubber strip damage classification branch. The rubber strip edge line segmentation and recognition branch includes a first feature extraction module, and the rubber strip damage classification branch includes a second feature extraction module and a classification processing module. A mask image is obtained by performing a masking process on the j-th damaged image. The mask image only includes information about the image elements corresponding to the adhesive strip. The first feature extraction module is invoked to extract features from the mask image to obtain the edge line segmentation features of the adhesive strip, and the second feature extraction module is invoked to extract features from the mask image to obtain the damage classification features of the adhesive strip; The edge line segmentation features and the damage classification features are spliced ​​together, and the classification processing module is called to perform classification processing based on the splicing result to obtain the first recognition result.

10. The method as described in claim 5, characterized in that, The target surface group corresponding to the kth set to be matched includes the fourth surface group, and the damage characteristics corresponding to the fourth surface group include: the damage category of any surface in the fourth surface group includes one or more of the first type of damage and the second type of damage; The step of determining the damage identifier of the j-th damaged image based on the damage characteristics corresponding to the target surface group includes: The damage identification algorithms corresponding to the first type of damage and the second type of damage are respectively used to identify the damage of the j-th damaged image, resulting in a third identification result and a fourth identification result. The third identification result is used to indicate whether the j-th damaged image has or does not have damage belonging to the first type of damage, and the fourth identification result is used to indicate whether the j-th damaged image has or does not have damage belonging to the second type of damage. The damage identifier corresponding to the j-th damaged image is determined based on the third and fourth recognition results.

11. An image processing apparatus, characterized in that, The device includes: The acquisition unit is used to acquire m maintenance items and n damage images of the target enclosure, where m and n are both positive integers; the target enclosure includes a left side, right side, front side, top side, door panel side, inner floor surface, and bottom floor surface, each side corresponding to a face identifier; the left side, right side, and front side are divided into a first face group, the top side into a second face group, the door panel side into a third face group, and the inner floor surface and bottom floor surface into a fourth face group; The acquisition unit is further configured to acquire the damage information reflected in each of the m maintenance entries, and to acquire the damage information reflected in each of the n damage images; the damage information includes the damage location and damage details on the target box, and the damage location includes a surface marker; The processing unit is used to group maintenance items and damaged images with similar damage locations into a set, resulting in W sets to be matched, where W is a positive integer; wherein, the i-th maintenance item and the j-th damaged image have similar damage locations, meaning that the face indicated by the face marker in the damage location of the i-th maintenance item belongs to the same face group as the face indicated by the face marker in the damage location of the j-th damaged image, where i is a positive integer and i≤m, and j is a positive integer and j≤n; The processing unit is also used to identify maintenance items and damage images with the same damage details as a damage repair relationship group in a set to be matched; Each damage repair relationship group includes one maintenance entry and at least one damage image. The damage information reflected by the maintenance entry and the damage information reflected by the damage image in each damage repair relationship group are matched.

12. A computer device, characterized in that, include: Storage devices and processors; A memory, wherein one or more computer programs are stored; A processor for loading one or more computer programs to implement the image processing method as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in any one of claims 1-10.

14. A computer program product, characterized in that, The computer program product includes a computer program adapted to be loaded by a processor and execute the image processing method as described in any one of claims 1-10.

Images