Container seal number identification method and related device

Abstract

The application discloses a container lead seal number identification method and related equipment, and relates to the field of container lead seal number identification; in the shooting process, after the container truck is determined to be stable, the camera is controlled to take screenshots, and target lead seal to-be-inspected area images corresponding to each camera are acquired. Then, all the locks in each target lead seal to-be-inspected area image are detected, lock arrangement codes are generated, and the codes are used to determine the actual positions of the lead seal locks on the rear container door of the container by using a preset relationship mapping table. Subsequently, corresponding camera control instructions are generated, the camera is controlled to take enlarged pictures of the positions, and high-resolution lead seal enlarged pictures are acquired. Finally, lead seal number identification results are extracted from the lead seal enlarged pictures, the identification results corresponding to two cameras and belonging to the same lead seal are cross-compared and optimized, and the final lead seal number identification result is determined. Thus, the application realizes the automatic and accurate identification of the lead seal number.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of container seal number identification, in particular to a container seal number identification method and related equipment. BACKGROUND

[0002] With the development of global economy, international trade is increasingly frequent, and the frequency of sea transportation operations, as an important transportation mode, continues to rise. The traditional port business relying on manual work has been difficult to meet the demand of modern port operation with high frequency and high timeliness due to high cost and low efficiency, and intelligent transformation is imperative. After loading goods, the container needs to be applied with a lead seal with a unique identification at the lock catch of the container door to ensure safety during transportation. When the truck passes through the gate, the actual seal number needs to be checked with the manifest information to determine whether the container has been opened illegally and to protect the integrity of the goods. However, due to the small size of the lead seal and the small font of the seal number, it needs to be shot at close range to be identified. The traditional manual verification method has been unable to meet the current high-intensity gate operation, and there are problems such as low efficiency and easy to make mistakes. This identification task faces multiple technical problems: first, the position of the lead seal on the container is not fixed, which is difficult to accurately position; second, the seal number needs to be accurately extracted from the lead seal image containing interference information; third, the system needs to adapt to different vehicle types, box types and lead seal layout, and has self-adaptive ability; fourth, in a single operation, two cameras may shoot multiple lead seals, and lead seal matching and unification across different angles need to be realized; fifth, when the lead seal numbers identified by the two cameras are inconsistent, how to effectively verify and integrate them; sixth, for lead seals with high identification difficulty, multi-model collaborative identification needs to be introduced, and how to select the optimal decision when the multi-model results conflict is also a key problem to improve the identification accuracy. Therefore, an automatic, efficient and accurate lead seal number identification and verification method is urgently needed to support the unmanned and intelligent operation of the gate. SUMMARY

[0003] In order to solve the multiple technical problems faced by the current identification task and realize efficient and accurate identification of the lead seal number, the present application proposes a container seal number identification method, which comprises:

[0004] Adjusting the shooting angle of the first camera arranged on the left side of the gate and the shooting angle of the second camera arranged on the right side of the gate, so that the shooting area of the first camera is a set first effective detection area, and the shooting area of the second camera is a set second effective detection area;

[0005] Based on the starting signal of the operation, the first camera and the second camera are adjusted to the corresponding first preset point respectively, and the continuous shooting of images is started until the preset shooting cutoff time point is reached. During the shooting process, for each camera, the appearance state of the container rear door in the corresponding effective detection area in the image collected by the camera is detected in real time, and whether the truck has stopped stably is judged according to the stability of the continuous multiple frame detection results;

[0006] When it is judged that the current truck has been stopped, the first camera and the second camera are controlled to take screenshots, and target seal detection area images corresponding to the cameras are obtained through the screenshots;

[0007] For each target seal detection area image, all the locks contained therein are detected, and a lock arrangement code is generated based on the detected lock positions. The actual positions of the seals with locks on the rear container door are obtained by using the lock arrangement code and a preset relationship mapping table;

[0008] For the actual position of each seal with lock on the rear container door, a corresponding camera control instruction is generated, and the corresponding camera is controlled to take a magnified image of the seal through the instruction;

[0009] The seal number recognition result in the seal magnified image is obtained, and the seal number recognition results of each camera corresponding to the same seal are cross-compared and optimized to determine the target seal number recognition result of the corresponding seal.

[0010] Further, during the shooting process, for each camera, the appearance state of the container rear door in the corresponding effective detection area in the image collected by the camera is detected in real time, and whether the current truck has been stopped is judged according to the stability of the continuous multiple frame detection results; Specifically:

[0011] During the shooting process, for each camera, it is detected in real time whether there is a container rear door located in the corresponding effective detection area in the image collected by the camera. If there is, the coordinates of the rear door in the image and the corresponding time stamp are recorded in the effective rear door list corresponding to the camera, and the effective rear door count value corresponding to the camera is increased by 1. Whether the current truck has been stopped is judged based on the effective rear door list corresponding to each camera and the count value of the effective rear door.

[0012] Further, the detection of whether there is a container rear door located in the corresponding effective detection area in the image collected by the camera is specifically: the coordinates of the container rear door in the image are obtained to form a detection matrix; the coincidence degree of the detection matrix and the corresponding effective detection area is calculated, and if the coincidence degree is greater than a preset value, it is determined that there is a container rear door located in the corresponding effective detection area in the image.

[0013] Further, the judgment of whether the current truck has been stopped based on the effective rear door list corresponding to each camera and the count value of the effective rear door is specifically:

[0014] For each camera, when the count value of the effective tailgate corresponding to the camera is greater than a preset threshold, the coincidence degree of the tailgate between adjacent two frames is calculated according to the tailgate coordinates of the consecutive frames in the effective tailgate list and the corresponding time stamps, and the coincidence degree determination threshold is dynamically adjusted according to the time interval of adjacent frames; the frame number whose coincidence degree is greater than the corresponding coincidence degree determination threshold is counted as the available frame number, and if the available frame number is greater than a preset threshold, it is determined that the truck has stopped stably under the camera view angle;

[0015] When the available frame numbers of the left and right cameras are both greater than a preset threshold, it is determined that the current truck has stopped stably as a whole.

[0016] Further, when it is determined that the current truck has stopped stably, the first camera and the second camera are controlled to take screenshots, and the target seal inspection area images corresponding to each camera are obtained through the screenshots; specifically:

[0017] For each screenshot, the container tailgate area is identified, and the position information of each area is stored in the corresponding tailgate position set; the set is traversed, the largest area is selected as the main tailgate, the position of the main tailgate is determined, and the seal inspection area corresponding to the camera is determined; the camera is adjusted to the corresponding second preset point, and the seal inspection area is zoomed in and photographed to obtain the target seal inspection area image corresponding to the camera.

[0018] Further, for each target seal inspection area image, all the lockers contained therein are detected, and a locker arrangement code is generated based on the detected locker positions; the actual positions of the lockers with seals on the container tailgate are obtained by using the locker arrangement code and a preset relationship mapping table; specifically:

[0019] For each target seal inspection area image, all the lockers contained therein are detected, and the positions of the lockers with seals are stored in the locker set with seals; at the same time, the rightmost preset number of locker positions are selected from all the detected lockers and arranged in order from left to right to form an initial locker set;

[0020] The arrangement type of each locker in the initial locker set is divided according to the longitudinal position, and a locker arrangement code is generated based on the arrangement type;

[0021] By matching the positions of each locker in the locker set with seals with the locker positions in the initial locker set, the arrangement serial number of the lockers with seals in the initial locker set is determined;

[0022] Based on the determined arrangement serial number and the locker arrangement code, the actual positions of the lockers with seals on the container tailgate are obtained by using the preset relationship mapping table;

[0023] In the preset relationship mapping table, for each possible lock arrangement code, the actual physical position of each arrangement number corresponding lock on the container rear door is preset.

[0024] Further, the arrangement type is divided according to the longitudinal position of each lock in the initial lock set, and the lock arrangement code is generated based on the arrangement type; specifically:

[0025] The longitudinal coordinates of each lock in the initial lock set are obtained, the mean value of all lock longitudinal coordinates is calculated as a set threshold, the locks with longitudinal coordinates less than the set threshold are divided into the first row, and the locks with longitudinal coordinates greater than or equal to the threshold are divided into the second row; wherein the first row is above the second row; the first row lock is marked as 0, and the second row lock is marked as 1; a four-bit binary code is formed as a lock arrangement code according to the arrangement order from left to right in the initial lock set.

[0026] Further, the seal number identification result in the seal magnified image is obtained, and the seal number identification results corresponding to the same seal of each camera are cross-compared and optimized to determine the target seal number identification result of the corresponding seal; specifically:

[0027] The seal magnified image is detected by a deep learning neural network model to identify the ordinary seal and / or the customs seal present in the image, and to determine the position of each type of seal in the image; the position of the ordinary seal includes a character region provided with seal number information and a corresponding two-dimensional code region; the position of the customs seal only includes a character region provided with seal number information;

[0028] Based on the detection result corresponding to each seal, the two-dimensional code region and the character region provided with seal number information in the ordinary seal are cut out from the seal magnified image respectively to obtain a two-dimensional code local image and a seal local image; for the customs seal, only the character region is cut out as a seal local image;

[0029] According to the combination of the seal type in the seal magnified image, a differentiated identification strategy is adopted to obtain the identification result of the ordinary seal number and / or the customs seal number, wherein the ordinary seal is preferentially identified using the two-dimensional code local image, and if the identification fails, a multi-modal identification is performed in combination with the seal local image;

[0030] For the ordinary seal, the ordinary seal number identification result corresponding to the first camera is compared with the ordinary seal number identification result corresponding to the corresponding second camera according to the first cross-comparison rule to determine the target ordinary seal number identification result.

[0031] For the customs seal, the customs seal number identification result corresponding to the first camera is compared with the customs seal number identification result corresponding to the corresponding second camera according to the second cross-comparison rule to determine the target customs seal number identification result.

[0032] Further, the combination of the lead seal type in the magnified lead seal image is used to adopt a differentiated recognition strategy to obtain the recognition result of the ordinary lead seal number and / or the customs lead seal number; specifically:

[0033] According to the detection result, the type of the lead seal contained in the image is determined;

[0034] When there is only an ordinary lead seal, the two-dimensional code local map of the ordinary lead seal is preferentially recognized, and if the recognition is successful, it is taken as the ordinary lead seal number recognition result, otherwise, the multi-modal recognition method is used to obtain the ordinary lead seal number recognition result based on the lead seal local map of the ordinary lead seal;

[0035] When there is only a customs lead seal, the customs lead seal number recognition result is obtained based on the lead seal local map of the customs lead seal;

[0036] When there is both an ordinary lead seal and a customs lead seal, the customs lead seal number recognition result is obtained first, and then the ordinary lead seal number recognition result is obtained, and the customs lead seal number recognition result is used to eliminate interference and assist correction in the process of obtaining the ordinary lead seal number recognition result.

[0037] Further, the two-dimensional code local map of the ordinary lead seal is preferentially recognized, and if the recognition is successful, it is taken as the ordinary lead seal number recognition result, specifically:

[0038] The two-dimensional code local map is decoded using a two-dimensional code scanning tool, and if the decoding is successful and a non-empty string is obtained, and the string meets the preset lead seal number format rule, it is determined that the two-dimensional code content is valid, and it is taken as the ordinary lead seal number recognition result under the corresponding camera, and the result is marked as the two-dimensional code recognition source, and the recognition confidence is set to 1.

[0039] Further, if the decoding fails, the result is empty or does not meet the preset lead seal number format rule, the multi-modal recognition method is used to obtain the ordinary lead seal number recognition result based on the lead seal local map of the ordinary lead seal.

[0040] Further, the multi-modal recognition method is used to obtain the ordinary lead seal number recognition result based on the lead seal local map of the ordinary lead seal, specifically:

[0041] An OCR recognition model is used to recognize the character region in the lead seal local map, and a plurality of candidate texts are obtained; a first candidate set is constructed based on each candidate text, and each element in the set includes: a candidate text, a recognition confidence corresponding to the text, and position information of the text in the lead seal local map;

[0042] According to the spatial position priority and text features of each candidate text in the first candidate set in the lead seal local map, combined with the recognition confidence, the multiple candidate results are sorted and optimized to determine the first lead seal number recognition result under the corresponding camera;

[0043] According to the semantic features of each candidate text in the first candidate set, the first lead seal number recognition result is semantically completed or format corrected to obtain a second lead seal number recognition result;

[0044] The lead seal number recognition is performed on the lead seal local map by one or more third-party large models to obtain the lead seal number recognition result corresponding to each model;

[0045] The second lead seal number recognition result and the lead seal number recognition result output by each third-party large model are combined to form the general lead seal number recognition result under the corresponding camera.

[0046] Further, according to the spatial position priority and text features of each candidate text in the first candidate set in the lead seal local map, combined with the recognition confidence, the multiple candidate results are sorted and optimized to determine the first lead seal number recognition result under the corresponding camera; specifically:

[0047] All elements in the first candidate set are traversed, and the minimum value of the vertical coordinates of the candidate texts corresponding to the elements is recorded. The candidate text corresponding to the element with the minimum minimum value of the vertical coordinates is selected as the first candidate general lead seal number;

[0048] If the first candidate set contains only one element, the candidate text corresponding to the element is taken as the first lead seal number recognition result under the corresponding camera;

[0049] If the first candidate set contains two or more elements, the candidate text corresponding to the element with the second minimum minimum value of the vertical coordinates is selected as the second candidate general lead seal number, and the following optimization operation is performed:

[0050] The text lengths of the first candidate general lead seal number and the second candidate general lead seal number are compared, and the one with the longer text length is selected as the first lead seal number recognition result under the corresponding camera;

[0051] When the text lengths of the two are the same, the recognition confidence of each is compared, and the candidate text with the higher recognition confidence is selected as the first lead seal number recognition result under the corresponding camera.

[0052] Further, the lead seal local map of the customs seal is obtained by the customs seal number determination method to obtain the customs seal number recognition result, specifically:

[0053] The local image of the lead seal is subjected to character recognition using an OCR recognition model to obtain a plurality of candidate texts; a second candidate set is constructed based on the candidate texts, each element in the set containing a candidate text, a recognition confidence corresponding to the text, and position information of the text in the local image of the lead seal;

[0054] A candidate text with the longest length is selected from the second candidate set as a third lead seal number recognition result;

[0055] The local image of the lead seal is subjected to character recognition using an OCR recognition model to obtain a plurality of candidate texts; a second candidate set is constructed based on the candidate texts, each element in the set containing a candidate text, a recognition confidence corresponding to the text, and position information of the text in the local image of the lead seal;

[0056] The third lead seal number recognition result is added as a preferred option to the lead seal number recognition results output by the third-party large models in sequence to form an ordered multi-modal candidate set;

[0057] A customs lead seal number recognition result corresponding to the camera is obtained in the multi-modal candidate set based on a preferred rule.

[0058] Further, the customs lead seal number recognition result corresponding to the camera is obtained in the multi-modal candidate set based on a preferred rule, specifically:

[0059] Elements with a null value in the multi-modal candidate set are removed to obtain an effective candidate set;

[0060] The elements in the effective candidate set are preferred based on the length of the text:

[0061] The length of the characters of each element is calculated, and the maximum length value is recorded;

[0062] If the element with the maximum length is unique, it is taken as the customs lead seal number recognition result corresponding to the camera; if there are multiple elements with the maximum length, the element ranked first in sequence is selected as the customs lead seal number recognition result corresponding to the camera.

[0063] Further, for the ordinary lead seal, the ordinary lead seal number recognition result corresponding to the first camera is compared with the ordinary lead seal number recognition result corresponding to the second camera according to a first cross-comparison rule to determine the target ordinary lead seal number recognition result; specifically:

[0064] If only one camera corresponding to the same physical seal has a common seal number identification result based on the successful identification of the local image of the two-dimensional code, the identification result is taken as the target common seal number identification result; if the common seal number identification results of the same physical seal corresponding to the two cameras are both obtained based on the successful identification of the local image of the two-dimensional code, the two results are compared to see if they are consistent: if consistent, the result is taken as the target common seal number identification result; if inconsistent, the decoding quality scores of the two two-dimensional code identification results are further compared, and the one with a higher decoding quality score is selected as the target common seal number identification result;

[0065] If neither of the two cameras successfully identifies through the local image of the two-dimensional code, then:

[0066] The second seal number identification result in the common seal number identification result corresponding to the first camera is extracted as the first selected object, and the second seal number identification result in the common seal number identification result corresponding to the second camera is extracted as the second selected object; the first selected object and the second selected object correspond to the same physical seal;

[0067] The text lengths of the first selected object and the second selected object are compared, and the longer one is selected as the preferred object; when the lengths are the same, the recognition confidence of the corresponding object is compared, and the one with a higher recognition confidence is selected as the preferred object; after the preferred object is determined, it and the seal number identification results of all third-party large models corresponding to the first camera and the second camera together form a multi-source fusion identification result set of the common seal, i.e., the target common seal number identification result.

[0068] Further, for the customs seal, the customs seal number identification result corresponding to the first camera and the corresponding customs seal number identification result corresponding to the second camera are compared according to the second cross-comparison rule to determine the target customs seal number identification result; specifically:

[0069] The text lengths of the customs seal number identification result corresponding to the first camera and the customs seal number identification result corresponding to the second camera are compared, wherein the two identification results correspond to the same physical seal; the longer identification result is selected as the target customs seal number identification result;

[0070] When the text lengths of the two are the same, the recognition confidence of the corresponding identification result is compared, and the identification result with a higher recognition confidence is selected as the target customs seal number identification result.

[0071] Further, the first preset point is a wide-angle shooting pose called by the corresponding camera in the initial detection stage, used to obtain the overall image of the container and locate the rear container door of the container;

[0072] The second preset point is a high-definition shooting pose parameter set or called by the corresponding camera based on the position of the region in the screenshot after determining the sealed area to be detected, including an adaptive cloud platform fine-tuning angle and a high zoom lens parameter, used for optical zoom and focusing shooting of the sealed area to be detected, to obtain a high-resolution sealed local image, i.e., a target sealed area to be detected image.

[0073] To solve the above technical problems, the embodiment of the present application also provides an electronic device, comprising a processor and a memory storing programs, the programs comprising instructions which, when executed by the processor, cause the processor to perform the method described above.

[0074] To solve the above technical problems, the embodiment of the present application also provides a non-transitory machine-readable medium storing computer instructions for causing the computer to perform the method described above.

[0075] Compared with the prior art, the present application has at least the following beneficial effects:

[0076] (1) Based on the operation starting signal, the first camera and the second camera are adjusted to the respective first preset point, and the continuous image shooting is started. In the shooting process, the appearance state of the container rear door in the corresponding effective detection area in the image collected by each camera is detected in real time, and whether the truck has stopped is judged according to the stability of the continuous multi-frame detection results, so that the subsequent operation is ensured when the vehicle is stable. After stopping, the camera is controlled to take a screenshot to obtain the target sealed area to be detected image corresponding to each camera. Then, all the locks in each target sealed area to be detected image are detected to generate a lock arrangement code, and the actual position of each sealed lock on the container rear door is determined by using the code and a preset relationship mapping table. Subsequently, corresponding camera control instructions are generated to control the camera to shoot the positions to obtain high-resolution sealed zoom images. Finally, the sealed number recognition result is extracted from the sealed zoom image, and the recognition results corresponding to the two cameras belonging to the same sealed are cross-compared and optimized to determine the final sealed number recognition result. In this way, the present application not only realizes the automatic and accurate recognition of the sealed number, but also greatly reduces the need for manual intervention, and improves the efficiency and reliability of the port gate operation.

[0077] (2) The present application realizes dynamic determination of the truck stopping state by detecting the appearance state of the container rear door in the effective detection area in the image in real time, and judging the position stability based on the coordinate and timestamp information of the continuous multiple frames. This method avoids the limitations of relying on fixed positions or manual intervention for triggering shooting, and can accurately capture the opportunity when the vehicle is completely stopped under complex working conditions where the sealed position is not fixed, thereby ensuring the stability and reliability of subsequent image acquisition, and providing a key prerequisite for accurate positioning of the sealed.

[0078] (3) The application dynamically determines the actual positions of each lead seal on the rear container door by recognizing all the locks in the image and generating a lock arrangement code, and combining a preset relationship mapping table. This method does not rely on fixed templates or prior layouts, and can flexibly adapt to changes in different box types and lead seal installation positions, realizing self-adaptive recognition ability for diversified container structures and improving the versatility and deployment flexibility of the scheme.

[0079] (4) According to the combination of lead seal types in the lead seal magnified image, the application adopts a differentiated recognition strategy: for ordinary lead seals, preferentially use the two-dimensional code local image for recognition (two-dimensional code information standard, strong anti-interference), and when recognition fails or there is no two-dimensional code, combine the character area image for multi-modal recognition; for customs lead seals, directly perform high-precision text recognition based on the character area. This strategy fully utilizes the high reliability advantage of two-dimensional code recognition, significantly reduces the misrecognition rate of ordinary lead seal numbers; at the same time, special character recognition path is adopted for customs lead seals without two-dimensional code, avoiding redundant processing and improving the overall recognition efficiency and accuracy. By dynamically switching the recognition path according to the type, accurate, efficient and adaptive recognition of different types of lead seals is realized.

[0080] (5) In the multi-modal candidate set, the application adopts the optimization rule of "length priority, order second": first, eliminate null values, then select the candidate result with the longest character length; if there are multiple equal length results, select the element with the higher order. This rule takes "text integrity" as the core criterion, avoiding the bias that may be caused by directly relying on recognition confidence (such as different model confidence scales), ensuring recognition accuracy while improving the certainty of decision logic.

[0081] (6) The application introduces a cross-comparison rule for the recognition results of the first camera and the second camera of the same lead seal: preferentially compare the text length and select the more complete result; when the lengths are the same, further compare the recognition confidence and select the result with higher confidence as the target recognition result. This hierarchical optimization strategy effectively solves the conflict problem of dual-view recognition results, improves the fault tolerance capability under abnormal conditions such as local occlusion and uneven illumination, and ensures the high reliability and stability of the final output result.

[0082] (7) After acquiring images of the target seal inspection area from two cameras, this invention detects all the latches in each target seal inspection area image and determines the actual physical position of each seal on the container based on the latch arrangement code and a preset relationship mapping table. Then, the identification results of the same physical seal from different camera perspectives are associated and unified. Specifically, the identification result of the second seal number in the ordinary seal number identification result corresponding to the first camera is extracted as the first candidate object, and the identification result of the second seal number in the ordinary seal number identification result corresponding to the second camera is extracted as the second candidate object; the first candidate object and the second candidate object correspond to the same physical seal. This method realizes cross-view seal entity matching, avoids repeated identification or misalignment caused by differences in perspective, and ensures the integrity and consistency of identification results under multiple perspectives. Attached Figure Description

[0083] Figure 1 This is a flowchart of a container seal number identification method according to an embodiment of the present invention;

[0084] Figure 2 This is a flowchart of a method for generating container seal number recognition results based on multi-source fusion according to an embodiment of the present invention;

[0085] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0086] Example 1

[0087] The following are specific embodiments of the present invention, which are described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.

[0088] To address the multiple technical challenges currently faced in the identification task and achieve efficient and accurate identification of seal numbers, such as... Figure 1 As shown, this invention proposes a method for identifying container seal numbers, comprising:

[0089] Adjust the shooting angles of the first camera set on the left side of the gate and the second camera set on the right side of the gate so that the shooting area of ​​the first camera is the set first effective detection area and the shooting area of ​​the second camera is the set second effective detection area.

[0090] Based on the start signal of the operation, the first camera and the second camera are adjusted to the corresponding first preset point and start to continuously shoot images until the preset shooting end time (20 seconds) is reached. During the shooting process, for each camera, the appearance status of the container rear door in the corresponding effective detection area in the image it collects is detected in real time, and the stability of the detection results of multiple consecutive frames is used to determine whether the current container truck has stopped.

[0091] The first preset point is a wide-angle shooting pose called by the corresponding camera in the initial detection stage, used to obtain an overall image of the container and locate the rear container door of the container;

[0092] In the shooting process, for each camera, the occurrence state of the rear container door of the container in the corresponding effective detection area in the image collected by the camera is detected in real time, and whether the current truck has stopped stably is judged according to the stability of the continuous multiple frame detection results; specifically:

[0093] In the shooting process, for each camera, whether the rear container door of the container located in the corresponding effective detection area exists in the image collected by the camera is detected in real time, if it exists, the coordinates of the rear container door in the image and the corresponding time stamp are recorded in the effective rear container door list corresponding to the camera, and the effective rear container door count value corresponding to the camera is added by 1 (the initial value of the effective rear container door count value is 0); whether the current truck has stopped stably is judged based on the effective rear container door list corresponding to each camera and the count value of the effective rear container door.

[0094] The detection of whether the rear container door of the container located in the corresponding effective detection area exists in the image collected by the camera is specifically: obtaining the coordinates of the rear container door of the container in the image to form a to-be-detected matrix; calculating the coincidence degree of the to-be-detected matrix and the corresponding effective detection area, if the coincidence degree is greater than a preset value, it is determined that the rear container door of the container located in the corresponding effective detection area exists in the image.

[0095] The judgment of whether the current truck has stopped stably based on the effective rear container door list corresponding to each camera and the count value of the effective rear container door is specifically:

[0096] For each camera, when the count value of the effective rear container door corresponding to the camera is greater than a preset threshold (in this embodiment, the threshold is equal to 15), the coincidence degree of the rear container door between adjacent two frames is calculated according to the rear container door coordinates and the corresponding time stamp of the continuous frames in the effective rear container door list of the camera, and the coincidence degree determination threshold is dynamically adjusted according to the time interval of adjacent frames; the number of frames with a coincidence degree greater than the corresponding coincidence degree determination threshold is counted as the number of available frames, if the number of available frames is greater than a preset threshold, it is determined that the truck has stopped stably in the view angle of the camera;

[0097] When the number of available frames of the left and right cameras is greater than the preset threshold, it is determined that the current truck has stopped stably.

[0098] It should be noted that in the process of judging whether the truck is stable, first, whether the container rear door located in the corresponding effective detection area exists in the image collected by the truck is detected in real time. If it exists, the coordinates of the rear door in the image and its corresponding time stamp are recorded in the effective rear door list corresponding to the camera. However, in actual operation, there may be a situation that the container rear door is not detected in some frames, resulting in a long time interval between adjacent effective frames. In order to deal with this situation, the coincidence degree determination threshold is dynamically adjusted according to the time interval between adjacent two frames. Specifically, when the time interval between adjacent frames is short, a higher coincidence degree threshold is allowed to be set, because the image changes less in a short time; when the time interval between adjacent frames is long, the coincidence degree threshold is reduced to adapt to the possible slight displacement or shaking. Through this dynamic adjustment mechanism, the stability judgment under different shooting frequencies and environmental conditions can be more flexible, ensuring that the truck can be accurately judged to be stable even in the case of frame interruption.

[0099] The present application realizes dynamic judgment of the truck stopping state by detecting the appearance state of the container rear door in the effective detection area in the image in real time, and judging the position stability based on the coordinate and time stamp information of continuous multiple frames. This method avoids the limitation of relying on fixed position or manual intervention to trigger shooting, and can accurately capture the moment when the vehicle completely stops under complex working conditions where the seal position is not fixed, thereby ensuring the stability and reliability of subsequent image acquisition, and providing a key prerequisite for accurate positioning of the seal.

[0100] When it is judged that the current truck has stopped, the first camera and the second camera are controlled to take screenshots, and the target seal detection area image corresponding to each camera is obtained through the screenshots;

[0101] When it is judged that the current truck has stopped, the first camera and the second camera are controlled to take screenshots, and the target seal detection area image corresponding to each camera is obtained through the screenshots; specifically:

[0102] For each screenshot, the container rear door area in the screenshot is identified, and the position information of each area is stored in the corresponding rear door position set. The set is traversed, the largest area is selected as the main rear door, and the seal detection area corresponding to the camera is determined according to the position of the main rear door. The camera is adjusted to the corresponding second preset point, and the seal detection area is zoomed in and shot to obtain the target seal detection area image corresponding to the camera.

[0103] It needs to be explained that there may be multiple rear doors in the screenshot, so it is necessary to traverse the set to determine the main container rear door, i.e. the main rear door. In this way, the system can accurately identify and select the most prominent container rear door in the image, thereby ensuring that the subsequent lead seal detection area is more accurate and reliable.

[0104] In addition, the lead seal detection area corresponding to the camera is determined according to the position of the main rear door, specifically:

[0105] The lead seal detection area corresponding to the camera is determined according to the position of the main rear door. The system first calculates the center position of the main rear door, and defines a lead seal detection area based on the center point. This detection area is located in the center part of the main rear door, ensuring that it covers the possible lead seal position. Specifically, the system will select the midpoint of the left and right boundaries of the main rear door as the left boundary of the detection area, and the midpoint of the upper and lower boundaries as the upper boundary of the detection area, while keeping the right boundary and lower boundary of the detection area consistent with the right boundary and lower boundary of the main rear door (in simple terms: the left boundary and upper boundary of the detection area are the midpoints of the left and right boundaries and the upper and lower boundaries of the main rear door, and the right boundary and lower boundary are consistent with the right boundary and lower boundary of the main rear door). In this way, the system can accurately locate and enlarge the area where the lead seal is located, ensuring accurate shooting and identification of the lead seal.

[0106] The second preset point is a high-definition shooting pose parameter set or called based on the position of the lead seal detection area in the screenshot after the camera determines the lead seal detection area. It contains the appropriate gimbal fine-tuning angle and high zoom lens parameters, which are used for optical zoom and focus shooting of the lead seal detection area to obtain a high-resolution lead seal local image, i.e. the target lead seal detection area image.

[0107] For each target lead seal detection area image, detect all the locks it contains, and generate a lock arrangement code based on the detected lock position. Use the lock arrangement code and the preset relationship mapping table to obtain the actual position of each lead seal lock on the container rear door.

[0108] In the present application, the specific position of the container rear door or lock is accurately described by four coordinate values. These four coordinates usually form a rectangular region, representing the bounding box of the target object (such as the rear door or lead seal) in the image. Specifically, the four coordinates include the horizontal and vertical coordinates of the upper left corner (x1, y1) and the horizontal and vertical coordinates of the lower right corner (x2, y2), or the left boundary x min , the upper boundary y min , the right boundary x max and the lower boundary y maxFor example, when determining the position of the container rear door, the coordinate values are used to generate a to-be-inspected matrix, and the coincidence degree with the corresponding effective detection area is calculated; when detecting the lock and the lead seal, the coordinate values are also used to generate a lock arrangement code, and the shooting angle and magnification of the camera are adjusted accordingly. In this way, the specific position of the target object in the image can be accurately defined, facilitating subsequent processing steps (such as zoom shooting, etc.). In this way, the overall flexibility and operability of the system are improved.

[0109] For each target lead seal to-be-inspected area image, all the locks contained therein are detected, and a lock arrangement code is generated based on the detected lock positions. The lock arrangement code and a preset relationship mapping table are used to obtain the actual positions of the lead-sealed locks on the container rear door; specifically:

[0110] For each target lead seal to-be-inspected area image, all the locks contained therein are detected, and the positions of the lead-sealed locks are stored in a lead-sealed lock set. At the same time, the rightmost preset number (4) of lock positions are selected from all the detected locks and arranged in order from left to right to form a primary lock set.

[0111] On the container rear door, the locks are usually symmetrically distributed, and the lead seal is usually installed on the rightmost lock position, especially near the door handle side. Therefore, selecting the rightmost preset number (such as 4) of locks from all the detected locks as the key analysis area can cover the actual installation position of most lead seals. This strategy effectively reduces the detection range, improves the processing efficiency, and enhances the specificity and reliability of the lead seal position judgment, which is suitable for various box types and lock arrangement scenarios.

[0112] The arrangement type is divided according to the longitudinal positions of the locks in the primary lock set, and a lock arrangement code is generated based on the arrangement type;

[0113] The arrangement type is divided according to the longitudinal positions of the locks in the primary lock set, and a lock arrangement code is generated based on the arrangement type; specifically:

[0114] The longitudinal coordinates of each lock in the primary lock set are obtained, the mean value of all the lock longitudinal coordinates is calculated as a set threshold, and the locks with longitudinal coordinates less than the set threshold are divided into a first row, and the locks with longitudinal coordinates greater than or equal to the threshold are divided into a second row; wherein the first row is located above the second row; the first row of locks is marked as 0, and the second row of locks is marked as 1, and a four-bit binary code is formed as the lock arrangement code according to the arrangement order from left to right in the primary lock set.

[0115] By matching the positions of the locks in the lead-sealed lock set with the positions of the locks in the primary lock set, the arrangement number of the lead-sealed locks in the primary lock set is determined;

[0116] Based on the determined arrangement number and the lock arrangement code, the actual position of each lead-sealed lock on the container rear door is obtained by using a preset relationship mapping table.

[0117] In the preset relationship mapping table, for each possible lock arrangement code, the actual physical position of each arrangement number on the container rear door is preset respectively.

[0118] The actual position of each lead-sealed lock on the container rear door is described in detail as follows:

[0119] For each target lead-sealed detection area image, the system first detects all the locks in the image and stores the position information of the lead-sealed lock in the lead-sealed lock set. At the same time, the rightmost preset number (for example, 4) of lock positions are selected from all the detected locks and arranged in order from left to right to form the initial lock set.

[0120] Next, the arrangement type is divided according to the longitudinal coordinates of each lock in the initial lock set, and the lock arrangement code is generated. The specific steps are as follows:

[0121] The mean value of the longitudinal coordinates of each lock in the initial lock set is calculated as a threshold.

[0122] Locks with longitudinal coordinates less than the threshold are classified as the first row (upper), marked as 0; locks with longitudinal coordinates greater than or equal to the threshold are classified as the second row (lower), marked as 1.

[0123] According to the above rules, a four-bit binary code, i.e. the lock arrangement code, is formed in order from left to right in the initial lock set.

[0124] For example, consider the following three typical lock arrangement codes and their corresponding physical position mapping relationship table:

[0125]

[0126] Among them:

[0127] 1001: indicates that the four rightmost locks are in the second row, the first row, the first row, and the second row from left to right. According to the physical position mapping relationship table, if the lead-sealed lock is located on the leftmost side, its actual physical position is 2; if it is located on the left middle position, its actual physical position is 1; if it is located on the right middle position, its actual physical position is 3; if it is located on the rightmost side, its actual physical position is 4.

[0128] 1010: represents the four rightmost locks from left to right in turn as the second row, the first row, the second row, and the first row. According to the physical position mapping table, if the lead-sealed lock is located at the leftmost side, its actual physical position is 2; the actual physical position of the left middle position is 1; the actual physical position of the right middle position is 4; and the actual physical position of the rightmost side is 3.

[0129] 0110: represents the four rightmost locks from left to right in turn as the first row, the second row, the second row, and the first row. According to the physical position mapping table, if the lead-sealed lock is located at the leftmost side, its actual physical position is 0; the actual physical position of the left middle position is also 0; the actual physical position of the right middle position is 2; and the actual physical position of the rightmost side is 1.

[0130] By matching the positions of the lead-sealed locks in the lead-sealed lock set with the positions of the locks in the initial lock set, it is determined which locks in the initial lock set are lead-sealed, and their arrangement numbers in the set are recorded. Then, based on the determined arrangement numbers and the lock arrangement code, the actual physical positions of the lead-sealed locks on the rear door of the container are found using a preset relationship mapping table. This mapping table pre-sets the specific positions of the locks corresponding to each arrangement number on the rear door of the container for each possible lock arrangement code. In this way, the exact position of each lead-sealed lock on the container can be accurately located.

[0131] In addition, it should be noted that the number corresponding to the actual physical position is defined in advance by the wharf party, which means that the specific position of each lock on the rear door of the container (such as the numbers 1, 2, 3, 4, etc.) is pre-set according to the operation specifications and standards of the wharf. This pre-definition ensures that the system can accurately correspond the recognized lead-sealed locks to the specific physical positions, thereby achieving unified and standardized management.

[0132] The present application dynamically determines the actual positions of the lead-sealed locks on the rear door of the container by recognizing all the locks in the image and generating a lock arrangement code, in combination with a preset relationship mapping table. This method does not rely on fixed templates or prior layouts, and can flexibly adapt to changes in different container types and lead-seal installation positions, achieving self-adaptive recognition capability for diversified container structures and improving the versatility and deployment flexibility of the scheme.

[0133] For the actual position of each lead-sealed lock on the rear door of the container, a corresponding camera control instruction is generated, which controls the corresponding camera to take a magnified photo of the corresponding lead seal; the lead seal magnified photo may contain ordinary lead seals or customs lead seals, or both types of lead seals.

[0134] The seal number recognition result in the seal zoom-in image is obtained, and the seal number recognition results corresponding to the same seal of each camera are cross-compared and optimized to determine the target seal number recognition result of the corresponding seal.

[0135] The seal number recognition result in the seal zoom-in image is obtained, and the seal number recognition results corresponding to the same seal of each camera are cross-compared and optimized to determine the target seal number recognition result of the corresponding seal; specifically:

[0136] The seal zoom-in image is detected by a deep learning neural network model to recognize ordinary seals and / or customs seals present in the image and determine the positions of each type of seal in the image; the position of the ordinary seal includes a character region provided with seal number information and a corresponding two-dimensional code region; the position of the customs seal only includes a character region provided with seal number information;

[0137] Based on the detection result corresponding to each seal, the two-dimensional code region and the character region provided with seal number information in the ordinary seal are cut out from the seal zoom-in image to obtain a two-dimensional code local image and a seal local image; for the customs seal, only the character region is cut out as a seal local image;

[0138] According to the combination of seal types in the seal zoom-in image, a differentiated recognition strategy is used to obtain the recognition result of the ordinary seal number and / or the customs seal number, wherein the two-dimensional code local image is used to recognize the ordinary seal first, and if the recognition fails, a multi-modal recognition is performed in combination with the seal local image;

[0139] According to the combination of seal types in the seal zoom-in image, a differentiated recognition strategy is used to obtain the recognition result of the ordinary seal number and / or the customs seal number; specifically:

[0140] The type of seal contained in the image is determined according to the detection result;

[0141] When there is only an ordinary seal, the two-dimensional code local image of the ordinary seal is used to recognize first, and if the recognition is successful, the ordinary seal number recognition result is obtained, otherwise, a multi-modal recognition method is used to obtain the ordinary seal number recognition result based on the seal local image of the ordinary seal;

[0142] The two-dimensional code local image of the ordinary seal is used to recognize first, and if the recognition is successful, the ordinary seal number recognition result is obtained, specifically:

[0143] The two-dimensional code local image is decoded using a two-dimensional code scanning tool, if the decoding is successful and a non-empty string is obtained, and the string meets the preset seal number format rule, it is determined as valid two-dimensional code content, which is taken as the ordinary seal number recognition result under the corresponding camera, and is marked as the two-dimensional code recognition source, and its recognition confidence is set to 1.

[0144] The otherwise lead seal local map based on the common lead seal adopts a multi-modal recognition method to obtain a common lead seal number recognition result. Specifically, if the decoding fails, the result is empty, or it does not conform to the preset lead seal number format rule, a multi-modal recognition method is used to obtain a common lead seal number recognition result based on the lead seal local map of the common lead seal.

[0145] The otherwise lead seal local map based on the common lead seal adopts a multi-modal recognition method to obtain a common lead seal number recognition result. Specifically, if the decoding fails, the result is empty, or it does not conform to the preset lead seal number format rule, a multi-modal recognition method is used to obtain a common lead seal number recognition result based on the lead seal local map of the common lead seal.

[0146] A character region in the lead seal local map is identified using an OCR recognition model to obtain a plurality of candidate texts. A first candidate set is constructed based on each candidate text, and each element in the set includes: a candidate text, a recognition confidence corresponding to the text, and position information of the text in the lead seal local map.

[0147] According to the spatial position priority and text features of each candidate text in the first candidate set, combined with the recognition confidence, the plurality of candidate results are sorted and optimized to determine the first lead seal number recognition result under the corresponding camera.

[0148] It should be noted that if the OCR fails to recognize valid text (i.e., text containing correct format) conforming to the preset format, both the first lead seal number recognition result and the second lead seal number recognition result are marked as null. Valid text generally refers to text that conforms to specific format rules (such as character length, specific character combination, etc.) and has a high recognition confidence.

[0149] The otherwise lead seal local map based on the common lead seal adopts a multi-modal recognition method to obtain a common lead seal number recognition result. Specifically, if the decoding fails, the result is empty, or it does not conform to the preset lead seal number format rule, a multi-modal recognition method is used to obtain a common lead seal number recognition result based on the lead seal local map of the common lead seal.

[0150] All elements in the first candidate set are traversed, and the minimum value of the vertical coordinates of the candidate texts corresponding to each element is recorded. The candidate text corresponding to the element with the smallest minimum value of the vertical coordinates is selected as the first common lead seal number to be selected.

[0151] In the practical application scenario of container seals, the seal number is usually located at the upper or middle part of the lock. Since the origin of the image coordinate system is generally set at the top left corner of the image, a smaller vertical coordinate value indicates a closer position to the top of the image. Therefore, the vertical coordinate of the seal number character is relatively low, close to the top of the image. By selecting the candidate text with the smallest minimum vertical coordinate, texts appearing at higher positions in the image, which may belong to background noise or other irrelevant information, can be effectively filtered out, thereby improving the accuracy of the recognition result. This method takes advantage of the physical characteristics of the seal number position, ensuring that the recognition process prioritizes the text most likely to be the correct seal number, reducing the likelihood of misidentification, and improving overall recognition efficiency.

[0152] If the first candidate set contains only one element, the candidate text corresponding to the element is taken as the first seal number recognition result under the corresponding camera;

[0153] If the first candidate set contains two or more elements, the candidate text corresponding to the element with the second smallest minimum vertical coordinate is selected as the second candidate ordinary seal number, and the following preferred operations are performed:

[0154] Compare the text lengths of the first and second candidate ordinary seal numbers, and select the one with the longer text length as the first seal number recognition result under the corresponding camera;

[0155] When the text lengths of the two are the same, compare their corresponding recognition confidence, and select the candidate text with higher recognition confidence as the first seal number recognition result under the corresponding camera.

[0156] According to the semantic features of each candidate text in the first candidate set, the first seal number recognition result is semantically completed or modified in format to obtain the second seal number recognition result;

[0157] It needs to be explained that in order to further improve the quality of the recognition result, the system will check and process the semantic features of each candidate text. Specifically, the system will traverse all candidate texts (i.e. all elements in the first candidate set) and check whether each candidate text contains specific keywords such as “NOS”, “AKKON” or “PIL”. If these specific keywords are found in the candidate text, the corresponding identifier needs to be added before the first seal number recognition result. In this way, the system can semantically complete or modify the format of the first seal number recognition result, ensuring that the final output of the second seal number recognition result is not only accurate, but also contains necessary context information, improving the integrity and reliability of the overall recognition result.

[0158] Perform seal number recognition on the seal local image through one or more third-party large models to obtain seal number recognition results corresponding to each model;

[0159] The second lead seal number recognition result and the lead seal number recognition result output by each third-party large model are combined to form a general lead seal number recognition result under the corresponding camera.

[0160] In this embodiment, the third-party large models include MiniCPM-Llama3-V2.5-int4 and Deepseek.

[0161] When only the customs seal exists, the lead seal local image based on the customs seal is used to obtain a customs seal number recognition result by a customs seal number determination method;

[0162] The customs seal number recognition result is obtained by the customs seal number determination method based on the lead seal local image of the customs seal, and specifically:

[0163] An OCR recognition model is used to perform character recognition on the lead seal local image to obtain a plurality of candidate texts. A second candidate set is constructed based on the candidate texts, and each element in the set includes a candidate text, an identification confidence corresponding to the text, and position information of the text in the lead seal local image.

[0164] The candidate text with the longest length is selected from the second candidate set as a third lead seal number recognition result. It needs to be particularly pointed out that if the OCR fails to recognize a valid text (i.e., a text containing a correct format) that meets the preset format, the third lead seal number recognition result will be marked as null. A valid text generally refers to a text that meets specific format rules (such as character length, specific character combination, etc.) and has a high recognition confidence. In this way, the system can ensure that only truly valid content will be used as a candidate text, avoiding errors caused by misrecognition or invalid recognition.

[0165] The lead seal local image is sent to one or more third-party large models for recognition to obtain a lead seal number recognition result output by each model;

[0166] The third lead seal number recognition result is used as a preferred option, and the lead seal number recognition results output by each third-party large model are sequentially added (specifically added based on the preset priority of the third-party large model) to form an ordered multi-modal candidate set.

[0167] The customs seal number recognition result under the corresponding camera is obtained in the multi-modal candidate set based on a preference rule.

[0168] The customs seal number recognition result under the corresponding camera is obtained in the multi-modal candidate set based on a preference rule, and specifically:

[0169] Elements with a null value in the multi-modal candidate set are removed to obtain a valid candidate set.

[0170] If the effective candidate set is empty, the customs seal number recognition result under the corresponding camera is set as empty; otherwise, the effective candidate set is optimized based on the text length of each element in the effective candidate set:

[0171] The character length of each element is calculated, and the maximum length value is recorded;

[0172] If the element with the maximum length is unique, it is taken as the customs seal number recognition result under the corresponding camera; if there are multiple elements with the maximum length, the element with the first sequence is selected as the customs seal number recognition result under the corresponding camera.

[0173] The present application adopts the optimization rule of "length priority, sequence as secondary" in the multi-modal candidate set: first, eliminate null values, then select the candidate result with the longest character length; if there are multiple results with the same length, select the element with the higher sequence. This rule takes "text integrity" as the core criterion, avoiding the deviation that may be caused by directly relying on the recognition confidence (such as different model confidence scales), ensuring the recognition accuracy while improving the certainty of decision logic.

[0174] When both ordinary seals and customs seals exist, the customs seal number recognition result is obtained first, then the ordinary seal number recognition result is obtained, and the customs seal number recognition result is used for interference elimination and auxiliary correction in the process of obtaining the ordinary seal number recognition result.

[0175] Specifically, the system will traverse all elements in the first candidate set, first delete the same content as the customs seal number recognition result, and then record the minimum vertical coordinate value of each element corresponding to the candidate text. In this way, the system can effectively eliminate the interference of the customs seal number on the ordinary seal number recognition, ensure that the final output of the ordinary seal number recognition result is not only accurate, but also avoids the misrecognition problem caused by the similarity or overlap of the two kinds of seals, and improves the reliability and accuracy of the overall recognition result.

[0176] According to the combination of seal types in the seal magnification image, the present application adopts a differentiated recognition strategy: for ordinary seals, preferentially use the local image of the two-dimensional code for recognition (two-dimensional code information standard, strong anti-interference), and when recognition fails or there is no two-dimensional code, combine the character area image for multi-modal recognition; for customs seals, directly perform high-precision text recognition based on the character area. This strategy fully utilizes the high reliability advantage of two-dimensional code recognition, significantly reduces the misrecognition rate of ordinary seal numbers; at the same time, a special character recognition path is adopted for customs seals without two-dimensional codes, avoiding redundant processing and improving the overall recognition efficiency and accuracy. By dynamically switching the recognition path according to the type, precise, efficient and adaptive recognition of different types of seals is realized.

[0177] For the common lead seal, the common lead seal number recognition result corresponding to the first camera is compared with the common lead seal number recognition result corresponding to the second camera according to the first cross comparison rule to determine the target common lead seal number recognition result;

[0178] For the common lead seal, the common lead seal number recognition result corresponding to the first camera is compared with the common lead seal number recognition result corresponding to the second camera according to the first cross comparison rule to determine the target common lead seal number recognition result; specifically:

[0179] If, for the same physical lead seal, only one camera corresponding to the common lead seal number recognition result is successfully obtained based on the local two-dimensional code image recognition, the recognition result is taken as the target common lead seal number recognition result; if the common lead seal number recognition results of the same physical lead seal corresponding to the two cameras are both obtained based on the successful recognition of the local two-dimensional code image, whether the two results are consistent is compared: if consistent, the result is taken as the target common lead seal number recognition result; if inconsistent, the decoding quality scores of the two two-dimensional code recognition results are further compared, and the one with higher decoding quality score is selected as the target common lead seal number recognition result; if the decoding quality scores are also consistent, the recognition result corresponding to the first camera is selected as the target common lead seal number recognition result;

[0180] In the present application, the local two-dimensional code image is decoded using a two-dimensional code scanning tool, and the decoding process generates multiple indicators reflecting image quality and decoding reliability at the same time as the output recognition result, including edge sharpness, image blur, and error correction code usage degree. These indicators can be integrated by the internal algorithm of the two-dimensional code scanning tool into a quantitative value, i.e. the "decoding quality score". The higher the score, the better the image quality and the more reliable the decoding result. When both cameras successfully recognize the two-dimensional code but the results are inconsistent, the system selects the recognition result corresponding to the one with higher decoding quality score as the final target result, thereby ensuring that the most reliable lead seal number information can still be obtained in the case of multi-view recognition conflict. This scoring mechanism belongs to the conventional output capability of two-dimensional code decoding technology and can be realized without additional development.

[0181] If both cameras fail to successfully recognize the local two-dimensional code image, then:

[0182] The second lead seal number recognition result in the common lead seal number recognition result corresponding to the first camera is extracted as the first selected object, and the second lead seal number recognition result in the common lead seal number recognition result corresponding to the second camera is extracted as the second selected object; the first selected object and the second selected object correspond to the same physical lead seal;

[0183] It is checked whether the two selected objects are empty:

[0184] If one of the two candidate objects is empty, the result of the other party is selected as the preferred object;

[0185] If both parties are empty, the preferred object is also empty;

[0186] If both parties are not empty, then: compare the text length of the first candidate object and the second candidate object, and select the longer one as the preferred object; when the length is the same, compare the corresponding recognition confidence, and select the one with higher recognition confidence as the preferred object;

[0187] It should be noted here that in some cases, the recognition confidence of the two candidate objects may also be exactly the same. In order to make a decision in this case, the system selects the first candidate object corresponding to the first camera from the two candidate objects as the final preferred object. This strategy ensures that the system can still make a clear choice in the case of all evaluation indicators being the same, avoiding processing delays or errors due to inability to make a decision.

[0188] After determining the preferred object, it is combined with the seal number recognition results of all third-party large models corresponding to the first camera and the second camera to form a multi-source fusion recognition result set of the ordinary seal, i.e., the target ordinary seal number recognition result.

[0189] After the two cameras respectively acquire the corresponding target seal detection area images, the actual physical position of each seal on the container is determined by detecting all the locks in each target seal detection area image and based on a preset relationship mapping table of lock arrangement coding. Then, the recognition results belonging to the same physical seal under different camera perspectives are associated and unified. Specifically, the second seal number recognition result in the ordinary seal number recognition result corresponding to the first camera is extracted as the first candidate object, and the second seal number recognition result in the ordinary seal number recognition result corresponding to the second camera is extracted as the second candidate object; the first candidate object and the second candidate object correspond to the same physical seal. This method realizes cross-perspective seal entity matching, avoids repeated recognition or mispositioned association due to perspective differences, and ensures the integrity and consistency of the recognition results under multiple perspectives.

[0190] For the customs seal, the customs seal number recognition result corresponding to the first camera and the corresponding customs seal number recognition result corresponding to the second camera are compared according to the second cross-comparison rule to determine the target customs seal number recognition result.

[0191] The first camera corresponding customs seal number recognition result and the corresponding second camera corresponding customs seal number recognition result are compared according to the second cross-comparison rule to determine the target customs seal number recognition result; specifically:

[0192] Check whether the customs seal number recognition result corresponding to the first camera and the customs seal number recognition result corresponding to the second camera are empty:

[0193] If both are empty, the target customs seal number recognition result is also empty;

[0194] If only one is empty, the recognition result of the other is selected as the target customs seal number recognition result;

[0195] If both are not empty, then: compare the text length of the customs seal number recognition result corresponding to the first camera and the customs seal number recognition result corresponding to the second camera, wherein the two recognition results correspond to the same physical seal; select the longer recognition result as the target customs seal number recognition result;

[0196] When the text length of both is the same, compare the corresponding recognition confidence, and select the recognition result with higher recognition confidence as the target customs seal number recognition result. When the recognition confidence is also the same, select the recognition result corresponding to the first camera as the target customs seal number recognition result.

[0197] The present application introduces a cross-comparison rule for the recognition results of the first camera and the second camera of the same seal: preferentially compare the text length and select the more complete result; when the length is the same, further compare the recognition confidence and select the result with higher confidence as the target recognition result. This hierarchical optimization strategy effectively solves the conflict problem of dual-view recognition results, improves the fault tolerance capability under abnormal conditions such as local occlusion and uneven illumination, and ensures the high reliability and stability of the final output result.

[0198] Based on the job start signal, the first camera and the second camera are adjusted to the respective first preset point, and continuous image shooting is started. During shooting, the appearance state of the container rear door in the corresponding effective detection area in the image collected by each camera is detected in real time, and whether the truck has stopped is judged according to the stability of the continuous multi-frame detection result, ensuring that the subsequent operation is performed when the vehicle is stable. After stopping, the camera is controlled to take a screenshot to obtain the target seal detection area image corresponding to each camera. Then, all the locks in each target seal detection area image are detected, the lock arrangement code is generated, and the actual position of each seal lock on the container rear door is determined by using the code and the preset relationship mapping table. Subsequently, the corresponding camera control instruction is generated to control the camera to shoot the positions at a larger scale to obtain high-resolution seal zoom images. Finally, the seal number recognition result is extracted from the seal zoom images, and the recognition results corresponding to the same seal from the two cameras are cross-compared and optimized to determine the final seal number recognition result. In this way, the present application not only realizes the automatic and accurate recognition of the seal number, but also greatly reduces the need for manual intervention, improves the efficiency and reliability of the port gate operation.

[0199] Example 2

[0200] To address the multiple technical challenges currently faced in the identification task and achieve efficient and accurate identification of seal numbers, such as... Figure 2 As shown, this invention also proposes a method for generating container seal number identification results based on multi-source fusion, including:

[0201] Obtain magnified images of the seal from a first camera and a second camera, both from two set viewpoints. The magnified images of the seal include ordinary seals and / or customs seals. For each included seal, the corresponding two magnified images of the seal respectively contain image information of the seal taken from different viewpoints.

[0202] It should be noted that the enlarged images of the lead seals are partial images obtained by high-magnification optical magnification of a single lead-sealed latch on the rear door of the container. Each enlarged image corresponds to a specific latch location, focusing on the lead seal installed on that latch.

[0203] The magnified image of the seals is detected using a deep learning neural network model to identify ordinary seals and / or customs seals in the image, and to determine the location of each type of seal in the image; the location of the ordinary seal includes a character area containing the seal number information and a corresponding QR code area; the location of the customs seal only includes a character area containing the seal number information.

[0204] Based on the detection results corresponding to each lead seal, the QR code area and the character area containing the lead seal number information in the ordinary lead seal are extracted from the enlarged image of the lead seal to obtain a partial image of the QR code and a partial image of the lead seal; for customs lead seals, only the character area is extracted as a partial image of the lead seal.

[0205] Based on the combination of seal types in the enlarged image of the seal, the identification results of ordinary seal numbers and / or customs seal numbers are obtained by using a differentiated identification strategy with the corresponding partial QR code image and / or partial seal image.

[0206] The method involves using a differentiated identification strategy based on the combination of seal types in the enlarged image of the seal, and employing corresponding partial images of QR codes and / or seals to obtain the identification results of ordinary seal numbers and / or customs seal numbers; specifically:

[0207] Determine the type of lead seal contained in the image based on the detection results;

[0208] When only a regular lead seal exists, identification is performed based on a partial image of the QR code on the regular lead seal. If the identification is successful, it is taken as the identification result of the regular lead seal number. Otherwise, a multimodal identification method is used to obtain the identification result of the regular lead seal number based on the partial image of the regular lead seal.

[0209] The priority is based on the local two-dimensional code of the general seal to identify, if the identification is successful, it is regarded as the general seal number identification result, specifically:

[0210] The local two-dimensional code is decoded by using a two-dimensional code scanning tool, if the decoding is successful and a non-empty string is obtained, and the string meets the preset seal number format rule, it is determined as valid two-dimensional code content, which is regarded as the general seal number identification result under the corresponding camera, and the result is marked as the two-dimensional code identification source, and the identification confidence is set to 1.

[0211] Otherwise, the general seal number identification result is obtained based on the seal local map of the general seal by using a multi-modal identification method, specifically: if the decoding fails, the result is empty or does not meet the preset seal number format rule, the general seal number identification result is obtained based on the seal local map of the general seal by using a multi-modal identification method.

[0212] The general seal number identification result is obtained based on the seal local map of the general seal by using a multi-modal identification method, specifically:

[0213] The character region in the seal local map is identified by using an OCR identification model to obtain a plurality of candidate texts; a first candidate set is constructed based on each candidate text, each element in the set includes: a candidate text, an identification confidence corresponding to the text, and position information of the text in the seal local map;

[0214] In the present application, the position of the text in the seal local map or the position of the seal in the image is accurately described by four coordinate values. The four coordinates usually form a rectangular region, which represents the bounding box of the target object (such as text or seal) in the image. Specifically, the four coordinates include the horizontal and vertical coordinates of the upper left corner (x1, y1) and the horizontal and vertical coordinates of the lower right corner (x2, y2), or respectively represent the left boundary x min , the upper boundary y min , the right boundary x max and the lower boundary y max .

[0215] According to the spatial position priority and text features of each candidate text in the first candidate set in the seal local map, combined with the identification confidence, the plurality of candidate results are sorted and optimized to determine the first seal number identification result under the corresponding camera;

[0216] It should be noted that if the OCR fails to identify valid text (i.e. text containing correct format) meeting the preset format, both the first seal number identification result and the second seal number identification result are marked as null value. Valid text usually refers to text meeting specific format rules (such as character length, specific character combination, etc.) and having high identification confidence.

[0217] The first seal number recognition result under the corresponding camera is determined by sorting and optimizing the multiple candidate results according to the spatial position priority and the text features of each candidate text in the first candidate set in combination with the recognition confidence, specifically as follows:

[0218] All elements in the first candidate set are traversed, and the minimum value of the ordinate of the candidate text corresponding to each element is recorded. The candidate text corresponding to the element with the minimum minimum value of the ordinate is selected as the first ordinary seal number to be selected.

[0219] In the actual application scenario of the container seal, the seal number is usually located in the upper or middle part of the lock. Since the origin of the image coordinate system is generally set at the upper left corner of the image, the smaller the ordinate value is, the closer the position is to the top of the image. Therefore, the ordinate of the seal number character is relatively low, close to the top of the image. By selecting the candidate text with the minimum minimum value of the ordinate, texts appearing at a higher position in the image, which may belong to background noise or other irrelevant information, can be effectively filtered out, thereby improving the accuracy of the recognition result. This method utilizes the physical characteristics of the seal number position, ensures that the recognition process can give priority to the text that is most likely to be the correct seal number, reduces the possibility of misrecognition, and improves the overall recognition efficiency.

[0220] If the first candidate set contains only one element, the candidate text corresponding to the element is taken as the first seal number recognition result under the corresponding camera.

[0221] If the first candidate set contains two or more elements, the candidate text corresponding to the element with the second minimum minimum value of the ordinate is selected as the second ordinary seal number to be selected, and the following optimization operation is performed:

[0222] The text length of the first ordinary seal number to be selected and the second ordinary seal number to be selected is compared, and the one with the longer text length is selected as the first seal number recognition result under the corresponding camera.

[0223] When the text lengths of the two are the same, the recognition confidence corresponding to each is compared, and the candidate text with the higher recognition confidence is selected as the first seal number recognition result under the corresponding camera.

[0224] According to the semantic features of each candidate text in the first candidate set, the first seal number recognition result is semantically completed or modified in format to obtain the second seal number recognition result.

[0225] It needs to be explained that, in order to further improve the quality of the recognition result, the system will check and process the semantic features of each candidate text. Specifically, the system will traverse all candidate texts (i.e. all elements in the first candidate set) and check whether each candidate text contains specific keywords such as "NOS", "AKKON" or "PIL", etc. If these specific keywords are found to exist in the candidate text, the corresponding identifier needs to be added before the first seal number recognition result. In this way, the system can perform semantic completion or format correction on the first seal number recognition result, ensuring that the final output of the second seal number recognition result is not only accurate but also contains necessary context information, improving the integrity and reliability of the overall recognition result.

[0226] performing seal number recognition on the seal local image by one or more third-party large models to obtain a seal number recognition result corresponding to each model;

[0227] combining the second seal number recognition result and the seal number recognition results output by each third-party large model to obtain a common seal number recognition result under the corresponding camera.

[0228] In this embodiment, the third-party large models include MiniCPM-Llama3-V2.5-int4 and Deepseek.

[0229] When there is only a customs seal, a seal local image based on the customs seal is used to obtain a customs seal number recognition result by a customs seal number determination method;

[0230] The seal local image based on the customs seal is used to obtain a customs seal number recognition result by a customs seal number determination method, specifically:

[0231] A character recognition model is used to perform character recognition on the seal local image to obtain a plurality of candidate texts; a second candidate set is constructed based on the candidate texts, each element in the set including: a candidate text, an identification confidence corresponding to the text, and position information of the text in the seal local image;

[0232] The candidate text with the longest length is selected from the second candidate set as a third seal number recognition result; it needs to be particularly noted that if the OCR fails to recognize a valid text (i.e. a text containing a correct format) that meets the preset format, the third seal number recognition result will be marked as null. A valid text usually refers to a text that meets specific format rules (such as character length, specific character combination, etc.) and has a high recognition confidence. In this way, the system can ensure that only truly valid content will be used as a candidate text, avoiding errors caused by misidentification or invalid identification.

[0233] sending the local image of the lead seal to one or more third-party large models for recognition to obtain lead seal number recognition results output by each model;

[0234] The third lead seal number recognition result is taken as a first choice, and subsequent lead seal number recognition results output by each third-party large model are sequentially added (specifically added based on a preset priority of the third-party large model) to form an ordered multi-modal candidate set.

[0235] Based on the preferred rule, the customs lead seal number recognition result under the corresponding camera is obtained from the multi-modal candidate set.

[0236] The preferred rule-based customs lead seal number recognition result under the corresponding camera is obtained from the multi-modal candidate set, specifically:

[0237] Elements with a null value in the multi-modal candidate set are removed to obtain an effective candidate set.

[0238] If the effective candidate set is empty, the customs lead seal number recognition result under the corresponding camera is set to null; otherwise, the text length of each element in the effective candidate set is preferred:

[0239] The character length of each element is calculated, and the maximum length value is recorded.

[0240] If the element with the maximum length is unique, it is taken as the customs lead seal number recognition result under the corresponding camera; if there are multiple elements with the maximum length, the element with the first order is selected as the customs lead seal number recognition result under the corresponding camera.

[0241] In the multi-modal candidate set, the preferred rule of "length priority, order as secondary" is adopted: first, remove the null value, then select the candidate result with the longest character length; if there are multiple results with the same length, select the element with the higher order. This rule takes "text integrity" as the core criterion, avoiding the bias that may be caused by directly relying on the recognition confidence (such as different model confidence scales), ensuring the recognition accuracy while improving the certainty of the decision logic.

[0242] When both ordinary lead seals and customs lead seals exist, the customs lead seal number recognition result is obtained first, then the ordinary lead seal number recognition result is obtained, and the customs lead seal number recognition result is used for interference elimination and auxiliary correction in the process of obtaining the ordinary lead seal number recognition result.

[0243] Specifically, the system traverses all elements in the first candidate set, deletes the same content as the customs seal number recognition result, and then records the minimum value of the vertical coordinates of the candidate text corresponding to each element. In this way, the system can effectively exclude the interference of the customs seal number on the recognition of the ordinary seal number, ensure that the final output of the ordinary seal number recognition result is not only accurate, but also avoids the misrecognition problem caused by the similarity or overlap of the two seal numbers, and improves the reliability and accuracy of the overall recognition result.

[0244] According to the combination of the seal types in the seal magnified image, the application adopts a differentiated recognition strategy: for the ordinary seal, the two-dimensional code local image is preferentially used for recognition (two-dimensional code information standard, strong anti-interference), and when the recognition fails or there is no two-dimensional code, multi-modal recognition is performed in combination with the character region image; for the customs seal, high-precision text recognition is directly performed based on the character region. This strategy fully utilizes the high reliability advantage of two-dimensional code recognition, significantly reduces the misrecognition rate of the ordinary seal number; at the same time, a special character recognition path is adopted for the customs seal without two-dimensional code, avoiding redundant processing and improving the overall recognition efficiency and accuracy. By dynamically switching the recognition path according to the type, accurate, efficient and adaptive recognition of different types of seals is realized.

[0245] For the ordinary seal, the ordinary seal number recognition result corresponding to the first camera is compared with the ordinary seal number recognition result corresponding to the corresponding second camera according to the first cross-comparison rule to determine the target ordinary seal number recognition result.

[0246] For the ordinary seal, the ordinary seal number recognition result corresponding to the first camera is compared with the ordinary seal number recognition result corresponding to the corresponding second camera according to the first cross-comparison rule to determine the target ordinary seal number recognition result. Specifically,

[0247] If, for the same physical seal, only one camera corresponding to the ordinary seal number recognition result is obtained based on the successful recognition of the two-dimensional code local image, then the recognition result is taken as the target ordinary seal number recognition result; if the ordinary seal number recognition results of the same physical seal corresponding to the two cameras are both obtained based on the successful recognition of the two-dimensional code local image, then the two results are compared: if they are consistent, then the result is taken as the target ordinary seal number recognition result; if they are inconsistent, then the decoding quality scores of the two two-dimensional code recognition results are further compared, and the one with the higher decoding quality score is selected as the target ordinary seal number recognition result; if the decoding quality scores are also consistent, then the recognition result corresponding to the first camera is selected as the target ordinary seal number recognition result;

[0248] In the present application, the two-dimensional code local map is decoded using a two-dimensional code scanning tool. The decoding process generates multiple indicators reflecting the image quality and decoding reliability, including edge sharpness, image blurring, and error correction code usage, while outputting the recognition result. These indicators can be integrated by the two-dimensional code scanning tool's internal algorithm into a quantitative value, i.e., a "decoding quality score." The higher the score, the better the image quality and the more reliable the decoding result. When both cameras successfully recognize the two-dimensional code but the results are inconsistent, the system compares the decoding quality scores of each other and selects the recognition result corresponding to the higher score as the final target result, ensuring that the most reliable seal number information can be obtained even in the case of multi-angle recognition conflict. This scoring mechanism is a routine output capability of two-dimensional code decoding technology and can be implemented without additional development.

[0249] If neither of the two cameras successfully recognizes the two-dimensional code local map, then:

[0250] Extract the second seal number recognition result in the first camera's corresponding ordinary seal number recognition result as the first candidate object, and extract the second seal number recognition result in the second camera's corresponding ordinary seal number recognition result as the second candidate object. The first candidate object and the second candidate object correspond to the same physical seal.

[0251] Check if both candidate objects are empty:

[0252] If one of the candidate objects is empty, select the result of the other as the preferred object.

[0253] If both are empty, the preferred object is also empty.

[0254] If neither is empty, then: compare the text length of the first candidate object and the second candidate object, and select the longer one as the preferred object; when the lengths are the same, compare their corresponding recognition confidence, and select the one with higher recognition confidence as the preferred object.

[0255] It should be noted that in some cases, the recognition confidence of the two candidate objects may be exactly the same. In order to make a decision in such a case, the system selects the first camera's corresponding candidate object, i.e., the first candidate object, as the final preferred object. This strategy ensures that the system can still make a clear choice in the case of all evaluation indicators being the same, avoiding processing delays or errors due to the inability to make a decision.

[0256] After determining the preferred object, it is combined with the seal number recognition results of all third-party large models corresponding to the first camera and the second camera to form a multi-source fusion recognition result set of the ordinary seal, i.e., the target ordinary seal number recognition result.

[0257] The customs seal number recognition result corresponding to the first camera is compared with the customs seal number recognition result corresponding to the second camera according to a second cross comparison rule to determine a target customs seal number recognition result.

[0258] The customs seal number recognition result corresponding to the first camera is compared with the customs seal number recognition result corresponding to the second camera according to a second cross comparison rule to determine a target customs seal number recognition result.

[0259] It is checked whether the customs seal number recognition result corresponding to the first camera and the customs seal number recognition result corresponding to the second camera are empty.

[0260] If both are empty, the target customs seal number recognition result is also empty.

[0261] If only one is empty, the recognition result of the other is selected as the target customs seal number recognition result.

[0262] If both are not empty, the text length of the customs seal number recognition result corresponding to the first camera is compared with the text length of the customs seal number recognition result corresponding to the second camera, wherein the two recognition results correspond to the same physical seal; the recognition result with the longer length is selected as the target customs seal number recognition result.

[0263] When the text lengths of the two are the same, the corresponding recognition confidence is compared, and the recognition result with the higher recognition confidence is selected as the target customs seal number recognition result. When the recognition confidences are also the same, the recognition result corresponding to the first camera is selected as the target customs seal number recognition result.

[0264] The present application introduces a cross comparison rule for the recognition results of the first camera and the second camera for the same seal: the text length is compared first, and the more complete result is selected; when the lengths are the same, the recognition confidence is further compared, and the result with the higher recognition confidence is selected as the target recognition result. This hierarchical optimization strategy effectively solves the problem of conflict between double-view recognition results, improves the fault tolerance capability under abnormal conditions such as local occlusion and uneven illumination, and ensures the high reliability and stability of the final output result.

[0265] The application obtains a lead seal magnified image photographed by a first camera and a second camera from two set visual angles, detects and locates common lead seals and / or customs lead seals in the image in combination with a deep learning model, and adopts a differential region matting strategy for different types of lead seals: for common lead seals, both the two-dimensional code local image and the lead seal local image provided with lead seal number information are extracted; and for customs lead seals, only the lead seal local image provided with lead seal number information is extracted. On this basis, according to the combination of different lead seal types, the corresponding two-dimensional code local image and / or lead seal local image is used to adopt a differential recognition strategy to obtain the lead seal number recognition result. Further, the recognition results of the same physical lead seal under double visual angles are cross-compared and optimized, effectively improving the accuracy and reliability of the recognition result. The method realizes high-precision automatic recognition of lead seal numbers in a complex scene, is especially suitable for container gate scenes where common lead seals and customs lead seals coexist and recognition is easily disturbed, and significantly improves the intelligent level and robustness of the recognition system.

[0266] The embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory in communication connection with the at least one processor. The memory stores a computer program capable of being executed by the at least one processor, and the computer program is used for causing the electronic device to execute the method of the embodiment of the present application when executed by the at least one processor.

[0267] The embodiment of the present application also provides a non-transient machine readable medium storing a computer program, wherein the computer program is used for causing a computer to execute the method of the embodiment of the present application when executed by a processor of the computer.

[0268] The embodiment of the present application also provides a computer program product, comprising a computer program, wherein the computer program is used for causing a computer to execute the method of the embodiment of the present application when executed by a processor of the computer.

[0269] Reference Figure 3 A block diagram of an electronic device that can serve as a server or client of the embodiment of the present application will now be described, which is an example of a hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent a wide variety of digital electronic computing devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computing devices. The electronic device can also represent a wide variety of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0270] As Figure 3As shown, the electronic device includes a computing unit 401 that can perform various appropriate actions and processes in accordance with a computer program stored in a read only memory (ROM) 402 or a computer program loaded into a random access memory (RAM) 403 from a storage unit 408. In the RAM 403, various programs and data required for operation of the electronic device can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0271] A plurality of components in the electronic device are connected to the I / O interface 405, including an input unit 406, an output unit 407, a storage unit 408, and a communication unit 409. The input unit 406 can be any type of device that can input information to the electronic device, and can receive inputted numerical or character information, and generate key signal inputs related to user settings and / or function controls of the electronic device. The output unit 407 can be any type of device that can present information, and can include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 408 can include, but is not limited to, a magnetic disk, an optical disk. The communication unit 409 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0272] The computing unit 401 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a CPU, a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 401 performs the methods and processes described above. For example, in some embodiments, the method embodiments of the present application can be implemented as a computer program tangibly embodied in a machine-readable medium, such as the storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device via the ROM 402 and / or the communication unit 409. In some embodiments, the computing unit 401 can be configured to perform the above-described methods by any other appropriate means, such as by means of firmware.

[0273] A computer program for implementing the method of the embodiments of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed, implements the functions / operations specified in the flowcharts and / or block diagrams. The computer program can be executed in whole on the machine, partially on the machine, partially on the machine as a stand-alone software package, and partially on a remote machine or a server.

[0274] In the context of the embodiments of the present application, a machine-readable medium can be a tangible medium that can contain or store the program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium can include, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more of an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0275] It should be noted that all directional directions (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative positional relationship, movement condition, etc. between components in a certain specific posture (as shown in the drawings), and if the specific posture changes, the directional directions will also change accordingly.

[0276] In addition, the description such as "first", "second", "one" and the like in the present application is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise specifically limited.

[0277] In the present application, unless otherwise explicitly specified and limited, the terms "connection", "fixation" and the like should be understood in a broad sense, for example, "fixation" can be fixed connection, or detachable connection, or integral; can be mechanical connection, or electrical connection; can be directly connected, or indirectly connected through intermediate medium, can be internal communication of two elements or interaction relationship of two elements, unless otherwise explicitly limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0278] In addition, the technical solutions among various embodiments of the present application can be combined with each other, but it must be based on the fact that a person skilled in the art can realize it, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor is it within the protection scope required by the present application.

Claims

1. A method of identifying a seal number of a container, characterized by, include: Adjust the shooting angles of the first camera set on the left side of the gate and the second camera set on the right side of the gate so that the shooting area of ​​the first camera is the set first effective detection area and the shooting area of ​​the second camera is the set second effective detection area. Based on the start signal of the operation, the first camera and the second camera are adjusted to their respective first preset positions and start to continuously capture images until the preset shooting end time is reached. During the shooting process, for each camera, the appearance status of the container rear door in the corresponding effective detection area in the image it captures is detected in real time, and the stability of the detection results of multiple consecutive frames is used to determine whether the current container truck has come to a complete stop. When it is determined that the current container truck has come to a complete stop, control the first camera and the second camera to take screenshots, and obtain the image of the target lead seal inspection area corresponding to each camera through the screenshots; For each target lead seal inspection area image, detect all the latches it contains, and generate a latch layout code based on the detected latch positions. Use the latch layout code and a preset relationship mapping table to obtain the actual position of each lead seal latch on the container rear door. For each target lead-sealed inspection area image, all latches contained therein are detected, and a latch layout code is generated based on the detected latch positions. Using a preset mapping table between the latch layout code and a pre-defined relationship table, the actual position of each latch with lead seal on the container's rear door is obtained; specifically: For each target lead seal inspection area image, detect all the locks it contains, and store the lock positions with lead seals into the lead seal lock set; at the same time, select the rightmost preset number of lock positions from all detected locks, and arrange them in order from left to right to form the initial row lock set. The arrangement type is divided according to the vertical position of each latch in the initial set of latches, and a latch arrangement code is generated based on the arrangement type; The sequence number of the lead-locked buckles in the initial row of buckles is determined by matching the position of each buckle in the set of lead-locked buckles with the position of the buckles in the initial row of buckles. Based on the determined sequence number and latch arrangement code, the actual position of each lead-locked latch on the rear door of the container is obtained using a preset relationship mapping table; In the preset relationship mapping table, for each possible latch arrangement code, the actual physical position of the latch corresponding to each arrangement number on the container rear door is preset; For each lead seal buckle on the rear door of the container, a corresponding camera control command is generated. The command is used to control the corresponding camera to take a magnified picture of the lead seal. Obtain the seal number recognition result from the enlarged image of the seal, and cross-compare and optimize the seal number recognition results of each camera corresponding to the same seal to determine the target seal number recognition result of the corresponding seal.

2. A method of identifying a seal number of a container according to claim 1, wherein, During the shooting process, for each camera, the presence status of the container rear door within the corresponding effective detection area in the captured images is detected in real time, and the stability of the detection results across multiple consecutive frames is used to determine whether the truck has come to a complete stop; specifically: During the shooting process, for each camera, it is detected in real time whether there is a container rear door located within the corresponding valid detection area in the image it captures. If there is, the coordinates of the rear door in the image and its corresponding timestamp are recorded in the list of valid rear doors for that camera, and the count value of the valid rear door for that camera is incremented by 1. Based on the list of valid rear doors for each camera and the count value of the valid rear doors, it is determined whether the current container truck has come to a complete stop.

3. A method of identifying a seal number of a container according to claim 2, wherein, The detection of whether a container rear door exists in the acquired image within the corresponding valid detection area specifically involves: obtaining the coordinates of the container rear door in the image to form a detection matrix; calculating the overlap between the detection matrix and the corresponding valid detection area; and determining that a container rear door exists in the image within the corresponding valid detection area if the overlap is greater than a preset value.

4. A method of identifying a seal number of a container according to claim 3, wherein, The method of determining whether the current container truck has come to a complete stop based on the list of valid rear doors corresponding to each camera and the count value of valid rear doors is as follows: For each camera, when the count of the effective rear doors corresponding to that camera is greater than a preset threshold, the overlap of the rear doors between two adjacent frames is calculated based on the coordinates of the rear doors in consecutive frames in the list of effective rear doors and the corresponding timestamps. The overlap determination threshold is dynamically adjusted according to the time interval between adjacent frames. The number of frames with an overlap greater than the corresponding overlap determination threshold is counted as the number of usable frames. If the number of usable frames is greater than the preset threshold, it is determined that the truck has come to a stop in the view of that camera. When the number of available frames from both the left and right cameras is greater than the preset threshold, it is determined that the truck has come to a complete stop.

5. The method of claim 1, wherein, When it is determined that the current container truck has come to a complete stop, the first and second cameras are controlled to take screenshots, and the images of the target lead seal inspection area corresponding to each camera are obtained through the screenshots; specifically: For each screenshot, identify the container rear door area and store the location information of each area into the corresponding rear door location set; traverse the set, select the area with the largest area as the main rear door, and determine the lead seal inspection area corresponding to the camera based on the location of the main rear door; Adjust the camera to the corresponding second preset point and magnify the area to be inspected for the lead seal to obtain the image of the target lead seal area to be inspected corresponding to the camera.

6. The method for identifying container seal numbers according to claim 1, characterized in that, The process involves classifying the layout type based on the vertical position of each latch in the initial latch set, and generating a latch layout code based on the layout type; specifically: Obtain the vertical coordinate of each lock in the initial row of locks. Calculate the mean of the vertical coordinates of all locks as a set threshold. Locks with vertical coordinates less than the set threshold are divided into the first row, and locks with vertical coordinates greater than or equal to the threshold are divided into the second row. The first row is located above the second row. Mark the locks in the first row as 0 and the locks in the second row as 1. Form a four-bit binary code according to the left-to-right arrangement order of the locks in the initial row as the lock arrangement code.

7. The method for identifying container seal numbers according to claim 1, characterized in that, The process involves obtaining the seal number identification result from the magnified image of the seal, and cross-comparing and optimizing the seal number identification results corresponding to the same seal from each camera to determine the target seal number identification result for the corresponding seal; specifically: The magnified image of the seals is detected using a deep learning neural network model to identify ordinary seals and / or customs seals in the image, and to determine the location of each type of seal in the image; the location of the ordinary seal includes a character area containing the seal number information and a corresponding QR code area; the location of the customs seal only includes a character area containing the seal number information. Based on the detection results corresponding to each lead seal, the QR code area and the character area containing the lead seal number information in the ordinary lead seal are extracted from the enlarged image of the lead seal to obtain a partial image of the QR code and a partial image of the lead seal; for customs lead seals, only the character area is extracted as a partial image of the lead seal. Based on the combination of seal types in the enlarged image of the seal, a differentiated identification strategy is adopted to obtain the identification results of ordinary seal numbers and / or customs seal numbers. Among them, ordinary seals are identified first using the partial image of the QR code. If the identification fails, multimodal identification is performed in combination with the partial image of the seal. For ordinary lead seals, the identification result of the ordinary lead seal number corresponding to the first camera is compared with the identification result of the ordinary lead seal number corresponding to the second camera according to the first cross comparison rule to determine the target ordinary lead seal number identification result; For customs seals, the customs seal number recognition result corresponding to the first camera is compared with the customs seal number recognition result corresponding to the second camera according to the second cross-comparison rule to determine the target customs seal number recognition result.

8. A method for identifying container seal numbers according to claim 7, characterized in that, The method involves using a differentiated identification strategy to obtain the identification results of ordinary seal numbers and / or customs seal numbers based on the combination of seal types in the enlarged image of the seal; specifically: Determine the type of lead seal contained in the image based on the detection results; When only a regular lead seal exists, identification is performed based on a partial image of the QR code on the regular lead seal. If the identification is successful, it is taken as the identification result of the regular lead seal number. Otherwise, a multimodal identification method is used to obtain the identification result of the regular lead seal number based on the partial image of the regular lead seal. When only a customs seal exists, the customs seal number identification result is obtained based on a partial image of the customs seal using the customs seal number determination method. When both ordinary lead seals and customs lead seals exist, the identification result of the customs lead seal number is obtained first, followed by the identification result of the ordinary lead seal number. During the process of obtaining the ordinary lead seal number identification result, the identification result of the customs lead seal number is used for interference elimination and auxiliary correction.

9. A method for identifying container seal numbers according to claim 8, characterized in that, The identification is based primarily on a partial image of the QR code on the ordinary lead seal. If the identification is successful, it is taken as the identification result of the ordinary lead seal number. Specifically: Use a QR code scanning tool to decode the partial image of the QR code. If the decoding is successful and a non-empty string is obtained, and the string conforms to the preset seal number format rules, it is determined to be valid QR code content. It is then used as the ordinary seal number recognition result under the corresponding camera, and the result is marked as the source of QR code recognition. At the same time, its recognition confidence is set to 1.

10. A method for identifying container seal numbers according to claim 9, characterized in that, Otherwise, the method of obtaining the ordinary seal number recognition result based on the partial image of the ordinary seal using a multimodal recognition method is as follows: if decoding fails, the result is empty, or it does not conform to the preset seal number format rules, then the method of obtaining the ordinary seal number recognition result is based on the partial image of the ordinary seal using a multimodal recognition method.

11. A method for identifying container seal numbers according to claim 10, characterized in that, The partial image of the ordinary lead seal based on the lead seal uses a multimodal recognition method to obtain the ordinary lead seal number recognition result, specifically: Identify character regions in a partial image of a lead seal to obtain multiple candidate texts; construct a first candidate set based on each candidate text, where each element contains: the candidate text, the recognition confidence score corresponding to the text, and the position information of the text in the partial image of the lead seal; Based on the spatial position priority and text features of each candidate text in the first candidate set in the partial image of the lead seal, and combined with the recognition confidence, multiple candidate results are sorted and optimized to determine the first lead seal number recognition result under the corresponding camera. Based on the semantic features of each candidate text in the first candidate set, the first seal recognition result is semantically completed or formatted to obtain the second seal recognition result. The seal number is identified by using one or more third-party large models to identify the seal number in the partial image of the seal, and the seal number identification results corresponding to each model are obtained. The second lead seal identification result is combined with the lead seal identification results output by each third-party large model to form the ordinary lead seal identification result under the corresponding camera.

12. The method for identifying container seal numbers according to claim 11, characterized in that, The process involves ranking and optimizing multiple candidate results based on the spatial position priority and text features of each candidate text in the partial image of the lead seal, combined with recognition confidence, to determine the recognition result of the first lead seal number under the corresponding camera; specifically: Iterate through all elements in the first candidate set, record the minimum ordinate of the candidate text corresponding to each element, and select the candidate text corresponding to the element with the smallest minimum ordinate as the first candidate ordinary lead seal number. If the first candidate set contains only one element, then the candidate text corresponding to that element is taken as the first seal recognition result under the corresponding camera. If the first candidate set contains two or more elements, the candidate text corresponding to the element with the second smallest minimum vertical coordinate is selected as the second candidate ordinary seal number, and the following optimization operation is performed: Compare the text lengths of the first candidate ordinary lead seal number and the second candidate ordinary lead seal number, and select the one with the longer text length as the recognition result of the first lead seal number under the corresponding camera. When the two texts have the same length, their corresponding recognition confidence scores are compared, and the candidate text with the higher recognition confidence score is selected as the first seal recognition result under the corresponding camera.

13. A method for identifying container seal numbers according to claim 12, characterized in that, The partial image of the customs seal, based on the customs seal, is used to obtain the customs seal number identification result through the customs seal number determination method, specifically as follows: Character recognition is performed on a partial image of the lead seal to obtain multiple candidate texts; a second candidate set is constructed based on each candidate text, and each element in the set contains: the candidate text, the recognition confidence score corresponding to the text, and the position information of the text in the partial image of the lead seal; The candidate text with the longest text length from the second candidate set is selected as the third lead seal recognition result; The partial image of the lead seal is sent to one or more third-party large models for recognition, and the lead seal number recognition results output by each model are obtained; The third lead seal identification result is used as the first option, and the lead seal identification results output by each third-party large model are added in sequence to form an ordered multimodal candidate set; Based on the optimization rules, the customs seal number recognition results under the corresponding camera are obtained from the multimodal candidate set.

14. A method for identifying container seal numbers according to claim 13, characterized in that, The process of obtaining the customs seal number recognition result from the multimodal candidate set based on the optimization rules is as follows: Eliminate elements with empty values ​​from the multimodal candidate set to obtain the effective candidate set; Optimization is based on the text length of each element in the valid candidate set: Calculate the character length of each element and record the maximum length value; If the element with the longest length is unique, it will be used as the recognition result of the customs seal number under the corresponding camera; if there are multiple elements with the longest length, the element that ranks first in sequence will be selected as the recognition result of the customs seal number under the corresponding camera.

15. A method for identifying container seal numbers according to claim 14, characterized in that, For ordinary lead seals, the identification result of the ordinary lead seal number corresponding to the first camera is compared with the identification result of the ordinary lead seal number corresponding to the second camera according to the first cross-comparison rule to determine the target ordinary lead seal number identification result; specifically: If, for the same physical seal, only one of the first and second cameras successfully identifies the ordinary seal number based on a partial image of the QR code, then that identification result is taken as the target ordinary seal number identification result. If both cameras successfully identify the ordinary seal number for the same physical seal based on a partial image of the QR code, then the two results are compared: if they are consistent, that result is taken as the target ordinary seal number identification result; if they are inconsistent, then the decoding quality scores of the two QR code identification results are further compared, and the one with the higher decoding quality score is selected as the target ordinary seal number identification result. If neither camera successfully recognizes the QR code using the partial image, then: The identification result of the second lead seal number in the ordinary lead seal number identification result of the first camera is extracted as the first candidate object, and the identification result of the second lead seal number in the ordinary lead seal number identification result of the second camera is extracted as the second candidate object; the first candidate object and the second candidate object correspond to the same physical lead seal; Compare the text lengths of the first and second candidate objects, and select the longer one as the preferred object; when the lengths are the same, compare their corresponding recognition confidence scores, and select the one with the higher recognition confidence score as the preferred object; after determining the preferred object, combine it with the lead seal number recognition results of all third-party large models corresponding to the first and second cameras to form the multi-source fusion recognition result set of the ordinary lead seal, that is, the target ordinary lead seal number recognition result.

16. A method for identifying container seal numbers according to claim 15, characterized in that, Regarding the customs seal, the identification result of the customs seal number corresponding to the first camera is compared with the identification result of the customs seal number corresponding to the second camera according to the second cross-comparison rule to determine the target customs seal number identification result; specifically: Compare the text lengths of the customs seal number recognition results corresponding to the first camera and the customs seal number recognition results corresponding to the second camera, where the two recognition results correspond to the same physical seal; select the recognition result with the longer text length as the target customs seal number recognition result. When the two texts have the same length, their corresponding recognition confidence scores are compared, and the recognition result with the higher recognition confidence score is selected as the target customs seal number recognition result.

17. A method for identifying container seal numbers according to claim 5, characterized in that, The first preset point is the wide-angle shooting pose called by the corresponding camera in the initial detection stage, which is used to acquire the overall image of the container and locate the rear door of the container; The second preset point is a high-definition shooting pose parameter set or called by the corresponding camera based on the position of the area in the screenshot after the lead seal inspection area is determined. It includes the adapted gimbal fine adjustment angle and high zoom lens parameters, which are used to optically magnify and focus the lead seal inspection area to obtain a high-resolution local image of the lead seal, i.e., the target lead seal inspection area image.

18. An electronic device comprising: A processor and a memory storing a program, characterized in that the program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 17.

19. A non-transitory machine-readable medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 17.

Images