Method and system for correlating X-ray analysis and visible light images in high-speed transmission

Abstract

The present invention provides a method and system for associating X-ray analysis with visible light images in high-speed transmission, the method comprising the steps of: acquiring an X-ray image of a package in real time, performing abnormal analysis on the package in the X-ray image, and if an abnormal package is identified, acquiring the position information of the abnormal package in the corresponding X-ray image; acquiring the visible light image of the package in real time and caching it, and numbering the visible light images in the cache according to the time sequence of the visible light image capture; while identifying the abnormal package, calculating the first number of the abnormal package in the cache according to the position information of the abnormal package in the corresponding X-ray image, the relative position of the X-ray machine and the camera, the average speed of the high-speed cross belt, the system processing time, and the center distance between two adjacent pallets, and associating the visible light image corresponding to the first number in the cache with the abnormal package. The present invention realizes the precise association of abnormal packages with visible light images.

Description

Technical Field

[0001] The present invention relates to the technical field of logistics industry, and in particular to a method and system for associating X-ray analysis with visible light images in high-speed transmission. Background Art

[0002] In the modern logistics industry, fast and safe inspection of packages is particularly important. Traditional package inspection systems usually use X-ray equipment to perform security checks inside packages. When an abnormal package is identified, it needs to be associated with the visible light image of the abnormal package in order to intercept and process the package.

[0003] However, there is a common problem in the prior art that the abnormal packages obtained by X-ray analysis cannot be accurately matched with the visible light images, which leads to the need for manual intervention in practical applications, greatly reducing the efficiency of logistics operations. Summary of the invention

[0004] In order to solve the above problems in the prior art, the present invention provides a method and system for associating X-ray analysis with visible light images in high-speed transmission, which is used to achieve accurate association between abnormal packages and visible light images and improve logistics detection efficiency.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] In a first aspect, the present invention provides a method for associating X-ray analysis with visible light images during high-speed transmission, which is applied to a high-speed cross belt that transmits packages through a tray and drives the packages through an X-ray machine and a camera, and the method comprises the steps of:

[0007] S1. Acquire an X-ray image of a package in real time, perform abnormal analysis on the package in the X-ray image, and if an abnormal package is identified, obtain location information of the abnormal package in the corresponding X-ray image;

[0008] S2, acquiring and caching visible light images of the package in real time, and numbering the visible light images in the cache according to the time sequence of capturing the visible light images;

[0009] S3. When the abnormal package is identified, the first number of the abnormal package in the cache is calculated based on the position information of the abnormal package in the corresponding X-ray image, the relative position of the X-ray machine and the camera, the average speed of the high-speed cross belt, the system processing time and the center distance between two adjacent pallets, and the visible light image corresponding to the first number in the cache is associated with the abnormal package.

[0010] The beneficial effect of the present invention is that by calculating the time point when the abnormal package passes through the visible light camera, the cache number of the abnormal package in the visible light image is obtained, so as to realize the accurate association between the abnormal package and the visible light image and improve the efficiency of logistics detection. In addition, the average speed of the high-speed cross belt is added in the calculation, so that the accurate association of the present invention can adapt to the speed and movement direction of the high-speed cross belt, thereby further improving the accuracy.

[0011] Optionally, the calculation formula of the first number N in step S3 is:

[0012] ;

[0013] Where N is rounded to an integer; D 摄像机 is the distance from the camera to the reference point, D X光机 is the distance from the X-ray machine to the reference point. The two use the moving direction of the high-speed cross belt as the positive coordinate axis and the side position where the starting point of the high-speed cross belt is located as the reference point; norm L is the horizontal scale coordinate of the abnormal package in the X-ray image, with the direction of high-speed cross-band movement as the positive direction; X光机 is the actual length of the high-speed cross-belt occupied by the X-ray machine display screen; v is the average speed of the high-speed cross-belt in the first time interval; t 分析 is the processing time of the X-ray image; t 校准 It is the calibration time performed by the user on site, and the default value is 0; L 托盘间距 It is the center distance between two adjacent pallets.

[0014] Optionally, the first time interval is related to the number of visible light images in the cache.

[0015] Optionally, the method further comprises the steps of:

[0016] The triggering time interval of the tray passing through the camera is calculated by acquiring the instantaneous speed of the high-speed cross belt, and the number in the cache is adjusted according to the triggering time interval.

[0017] Optionally, the step of calculating the triggering time interval of the tray passing through the camera by acquiring the instantaneous speed of the high-speed cross belt comprises:

[0018] Calculate the trigger time interval t when the tray passes the camera 平均 for:

[0019] ;

[0020] Where, Q is the number of instantaneous speed measurements in the second time interval; L 托盘间距 is the center distance between two adjacent pallets; v(n) is the nth instantaneous speed of the high-speed cross belt.

[0021] According to the above description, the present invention calculates the trigger time interval of the tray passing through the camera in the current time period to adjust the captured visible light image, so as to ensure that the order of subsequent visible light images will not be disordered when multiple shots or missed shots occur in the visible light image, thereby realizing the automatic error correction function.

[0022] Optionally, performing abnormality analysis on the package in the X-ray image in step S1 includes the following:

[0023] The X-ray image is input into an improved YOLOv5 model, and the improved YOLOv5 model performs abnormal analysis on the package in the X-ray image, wherein the improved YOLOv5 model is obtained by introducing edge information into the YOLOv5 model.

[0024] Optionally, the process of performing abnormal analysis on the package in the X-ray image by the improved YOLOv5 model is as follows:

[0025] Obtain an input feature map F according to the X-ray image;

[0026] Use the Laplacian operator to perform edge detection on the input feature map F and extract the edge feature map F edge ;

[0027] For the edge feature map F edge To average:

[0028] ;

[0029] Among them, W edge is the edge feature map F edge The average edge weight of ; C is the number of channels, c is the serial number of each channel; H is the height of the edge feature map, i is the serial number of each pixel in the height direction of the edge feature map; W is the width of the edge feature map, j is the serial number of each pixel in the width direction of the edge feature map;

[0030] Using the average edge weight W edge Perform global reweighting on the input feature map F:

[0031] ;

[0032] Among them, F' is the weighted feature map, is the field of real numbers;

[0033] The weighted feature map F' is input into the Backbone part of the improved YOLOv5 model, and the edge enhancement features are integrated in the Head part to generate multiple detection boxes, and then the precise detection boxes and confidence scores are obtained through non-maximum suppression. The improved YOLOv5 model introduces a new loss function for balancing the edge enhancement features and the target box loss during training. The precise detection box is the abnormal object in the X-ray image.

[0034] According to the above description, by enhancing the sensitivity of the recognition model to edge information, the edge position of the object can be more accurately located, boundary blur and false detection can be reduced, and the detection accuracy can be improved.

[0035] Optionally, the X-ray image includes a plurality of packages, and each package appears on a plurality of X-ray images. Then, obtaining the position information of the abnormal package in the corresponding X-ray image in step S1 includes:

[0036] From multiple X-ray images that identify the same abnormal object, an X-ray image with the highest confidence score is selected as the image to be associated, and the position information of the abnormal package in the image to be associated is obtained.

[0037] Optionally, the position information of the abnormal package in the corresponding X-ray image is a 10,000-percent coordinate.

[0038] In a second aspect, the present invention provides a system for correlating X-ray analysis and visible light images during high-speed transmission, comprising:

[0039] An X-ray machine is used to obtain an X-ray image of the package in real time, perform abnormal analysis on the package in the X-ray image, and if an abnormal package is identified, obtain the position information of the abnormal package in the corresponding X-ray image;

[0040] A camera is used to acquire visible light images of the package in real time and cache them, and to number the visible light images in the cache according to the time sequence of capturing the visible light images;

[0041] The associating device is used to calculate the first number of the abnormal package in the cache according to the position information of the abnormal package in the corresponding X-ray image, the relative position of the X-ray machine and the camera, the average speed of the high-speed cross belt, the system processing time and the center distance between two adjacent pallets when the abnormal package is identified, and associate the visible light image corresponding to the first number in the cache with the abnormal package.

[0042] Among them, the technical effects corresponding to the system for associating X-ray analysis and visible light images in high-speed transmission provided by the second aspect refer to the relevant description of the method for associating X-ray analysis and visible light images in high-speed transmission provided by the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the main flow of a method for associating X-ray analysis and visible light images in high-speed transmission according to an embodiment of the present invention;

[0044] Figure 2 The figure is a schematic diagram of the structure of a system for associating X-ray analysis and visible light images in high-speed transmission according to an embodiment of the present invention.

[0045] Description of reference numerals:

[0046] 1. X-ray machine;

[0047] 2. Camera;

[0048] 100, high-speed cross belt; 101, tray; 102, infrared trigger;

[0049] 200. Package. DETAILED DESCRIPTION

[0050] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0051] Embodiment 1

[0052] This embodiment is suitable for application scenarios such as logistics, warehousing and express delivery, where security inspection of the package 200 is required. In particular, in the scenario where an X-ray machine 1 is required to perform security inspection on the package 200 on the logistics line of the high-speed cross belt 100. The existing accuracy of the association between abnormal packages 200 and visible light images is not high. This embodiment accurately associates the abnormal package 200 with the visible light image by calculating the time when the package 200 arrives at the camera 2, and further improves the association accuracy by the average speed of the high-speed cross belt 100. See the following description for details.

[0053] Reference Figure 2 It can be seen that the method of the present invention is applied to the high-speed cross belt 100, which transmits the package 200 through the pallet 101 and drives the package 200 to pass through the X-ray machine 1 and the camera 2. Among them, the X-ray machine 1 will shoot the package 200 to generate an X-ray image for identifying abnormal packages 200; and the camera 2 will capture the package 200 on the pallet 101 through the infrared trigger 102 to generate a visible light image for identifying the information of the package 200. Therefore, the association between the two is very important.

[0054] Please refer to Figure 1 to Figure 2 , a method for associating X-ray analysis with visible light images during high-speed transmission, comprising the steps of:

[0055] S1. Acquire an X-ray image of the package 200 in real time, perform abnormal analysis on the package 200 in the X-ray image, and if an abnormal package 200 is identified, obtain position information of the abnormal package 200 in the corresponding X-ray image.

[0056] In this embodiment, the X-ray image includes multiple packages 200, and each package 200 appears on multiple X-ray images. A specific example is: the X-ray machine 1 shoots at 20 frames per second, so there are 20 frames of X-ray images per second, and there are five packages 200 on each frame of the X-ray image, then the image of a package 200 will appear on multiple frames of X-ray images. Assuming that the time for the package 200 to pass through the X-ray machine 1 is 0.5 seconds, there are 10 frames of X-ray images with the same package 200.

[0057] Therefore, in this embodiment, obtaining the position information of the abnormal package 200 in the corresponding X-ray image in step S1 includes:

[0058] From multiple X-ray images that identify the same abnormal object, an X-ray image with the highest confidence score is selected as the image to be associated, and the position information of the abnormal package 200 in the image to be associated is obtained.

[0059] In other embodiments, the first frame or the last frame in which the abnormal package 200 is identified may be selected.

[0060] In order to better adapt to the characteristics of X-ray images, this embodiment proposes an improved YOLOv5 model, which introduces an attention mechanism that specifically enhances edge detection and contrast analysis. This solution is particularly suitable for X-ray images with a single color but strong edge and contrast features. Specifically, in this embodiment, the abnormal analysis of the package 200 in the X-ray image in step S1 includes the following:

[0061] The X-ray image is input into the improved YOLOv5 model, and the improved YOLOv5 model performs abnormal analysis on the package 200 in the X-ray image. The improved YOLOv5 model is obtained by introducing edge information into the YOLOv5 model.

[0062] In this embodiment, the improved YOLOv5 model performs abnormal analysis on the package 200 in the X-ray image as follows:

[0063] S11, obtaining an input feature map F according to the X-ray image;

[0064] Use the Laplacian operator to perform edge detection on the input feature map F and extract the edge feature map F edge ;

[0065] The above expression is:

[0066] F edge =Laplacian(F).

[0067] S12, edge feature map F edge To average:

[0068] ;

[0069] Among them, W edge is the edge feature map F edge The average edge weight of ; C is the number of channels, c is the serial number of each channel; H is the height of the edge feature map, i is the serial number of each pixel in the height direction of the edge feature map; W is the width of the edge feature map, j is the serial number of each pixel in the width direction of the edge feature map.

[0070] Among them, W edge Represents the average edge strength in the entire feature map.

[0071] S13. Use average edge weight W edge Perform global reweighting on the input feature map F:

[0072] ;

[0073] Among them, F' is the weighted feature map, is the field of real numbers;

[0074] S14. The weighted feature map F' is input into the Backbone part of the improved YOLOv5 model, and the edge enhancement features are integrated in the Head part to generate multiple detection boxes. Then, the accurate detection boxes and confidence scores are obtained through non-maximum suppression. The improved YOLOv5 model introduces a new loss function for balancing the edge enhancement features and the target box loss during training. The accurate detection box is the abnormal object in the X-ray image.

[0075] That is, this embodiment introduces edge enhancement features in the Backbone part of the YOLOv5 model: the weighted feature map F' is input into the Backbone part of the YOLOv5 model. Backbone is responsible for extracting multi-scale features from the input feature map, so the introduction of weighted feature maps can enhance the model's ability to capture edge information.

[0076] At the same time, the edge enhancement features are further utilized in the detection head of the YOLOv5 model to generate the final detection frame. In the head part of the YOLOv5 model, the detection frame and confidence score are generated by integrating the edge enhancement features passed from the backbone part. At this time, during the training process, a new loss function is introduced to consider the balance between the edge enhancement features and the standard target frame loss to further improve the detection performance of the model. This enables the detection head of the YOLOv5 model to more accurately locate the edge position of the object, reduce boundary blur and false detection, and improve the accuracy of detection.

[0077] Finally, the improved YOLOv5 model is applied to the reweighted feature map F', and multiple detection boxes that take into account the edge-added features are generated in the output layer. Through non-maximum suppression (NMS), accurate detection boxes and confidence scores are obtained.

[0078] Among them, the position information of the abnormal package 200 in the corresponding X-ray image is a ten-thousandth coordinate, so the original coordinates of the precise detection frame are converted into a ten-thousandth coordinate:

[0079] (x norm ,y norm )=( );

[0080] Among them, x norm is the horizontal scale coordinate of the abnormal package 200 in the X-ray image, y norm is the vertical scale coordinate of the abnormal package 200 in the X-ray image; x and y are the original coordinates of the precise detection frame, and W and H are the width and height of the X-ray image.

[0081] S2. Obtain the visible light images of the package 200 in real time and cache them, and number the visible light images in the cache according to the time sequence of capturing the visible light images.

[0082] The number of visible light images in the cache is set according to actual needs. However, the number of visible light images in the cache must be at least greater than the number of trays 101 that can be accommodated by the distance between the X-ray machine 1 and the camera 2, usually several times greater. Figure 2 There are three packages 200 between the two, and at this time, the number of visible light images in the cache is 20. In other embodiments, the above number can also be greater.

[0083] S3. When the abnormal package 200 is identified, the first number of the abnormal package 200 in the cache is calculated based on the position information of the abnormal package 200 in the corresponding X-ray image, the relative position of the X-ray machine 1 and the camera 2, the average speed of the high-speed cross belt 100, the system processing time and the center distance between two adjacent pallets 101, and the visible light image corresponding to the first number in the cache is associated with the abnormal package 200.

[0084] In this embodiment, the calculation formula of the first number N in step S3 is:

[0085] ;

[0086] Where N is rounded to an integer; D 摄像机 is the distance from camera 2 to the reference point, D X光机 is the distance from the X-ray machine 1 to the reference point, with the moving direction of the high-speed cross belt 100 as the positive coordinate axis and the side position where the starting point of the high-speed cross belt 100 is located as the reference point; norm L is the horizontal scale coordinate of the abnormal package 200 in the X-ray image, with the moving direction of the high-speed cross belt 100 as the positive direction; X光机 is the actual length of the high-speed cross belt 100 occupied by the display screen of the X-ray machine 1; v is the average speed of the high-speed cross belt 100 in the first time interval; t 分析 is the processing time of the X-ray image; t 校准 It is the calibration time performed by the user on site, and the default value is 0; L 托盘间距 is the center distance between two adjacent trays 101 .

[0087] The above formula needs to be explained as follows:

[0088] D X光机 is the distance from X-ray machine 1 to the reference point, where X-ray machine 1 refers to the side close to camera 2. Similarly, x norm The moving direction of the high-speed cross belt 100 is taken as the positive direction, and its horizontal scale coordinate is also the side of the X-ray machine 1 close to the camera 2. X光机 -D 摄像机 +x norm ×L X光机 That is, the distance required to be transmitted between the abnormal package 200 and the camera 2 is calculated.

[0089] Correspondingly, when the high-speed cross belt 100 drives the package 200 to pass through the X-ray machine 1 first and then the camera 2, D X光机 -D 摄像机 is negative, x norm ×L X光机 is also negative. At this time, D X光机 -D 摄像机 +xnorm ×L X光机 If t is negative, 校准 The corresponding value can be taken to neutralize the above negative number, so that the whole formula is a positive number, and thus N is a positive number. On the contrary, when the high-speed cross belt 100 drives the package 200 to pass through the camera 2 first and then the X-ray machine 1, D X光机 -D 摄像机 is positive, x norm ×L X光机 is also positive. At this time, D X光机 -D 摄像机 +x norm ×L X光机 is positive, then t 校准 It can be 0. However, if the number of visible light images in the cache is small, it can be negative to neutralize the positive number and ensure that the obtained N is less than the number of visible light images in the cache. 校准 It is used to make adjustments according to the moving direction of the high-speed cross belt 100, the relative position of the X-ray machine 1 and the camera 2, and the number of visible light images in the cache.

[0090] Of course, if N is a negative number, you need to wait for the snapshot, for example, if it is -3, you only need to wait for camera 2 to take three snapshots. The number of visible light images in the cache is usually set to a higher value considering this situation. Therefore, generally speaking, t 校准 The default setting is 0, and the specific setting is adjusted by the technicians according to the on-site working conditions.

[0091] Among them, the first time interval is related to the number of visible light images in the cache. In this embodiment, the first time interval is the time elapsed by the number of visible light images in the cache. For example, the number of visible light images in the cache is 20. At this time, the time elapsed for the number of 20 visible light images is 3 seconds, and the first time interval is the average value of all instantaneous speeds within 3 seconds.

[0092] Therefore, compared with the traditional one-to-one association method of the installation identification sequence. Since the camera 2 will only take a snapshot when it receives the trigger of the infrared trigger 102. However, in actual applications, the infrared trigger 102 may be triggered by mistake or not triggered, resulting in missed shots or multiple shots. If it is the existing technology, all subsequent image associations will be wrong, and manual intervention is required. However, in this embodiment, the time point when the abnormal package 200 arrives at the visible light camera 2 is calculated to perform the corresponding cache position calculation. When encountering the above-mentioned triggering problem, at most it affects the corresponding relationship between the number of caches corresponding to the frame in the cache where the triggering problem occurs. For example, if the number of visible light images in the cache is 20, it will affect the association of the next 20 frames, but it will return to normal in the 21st frame. Therefore, this embodiment has the function of automatic error correction, which greatly improves the accuracy of the association between the abnormal package 200 and the visible light image, reduces the probability of manual intervention, and thus improves the efficiency of logistics detection.

[0093] Therefore, the present invention can achieve accurate association between the abnormal package 200 and the visible light image, thereby improving logistics detection efficiency.

[0094] Embodiment 2

[0095] On the basis of the above-mentioned embodiment 1, triggering a bad situation will also affect the image association of the order of the number of visible light images in the cache. In this embodiment, in order to further reduce the impact of such association misalignment, the above-mentioned embodiment 1 is further improved to obtain the embodiment 2 as follows:

[0096] Please refer to Figure 1 to Figure 2 , a method for associating X-ray analysis with visible light images during high-speed transmission, further comprising the steps of:

[0097] S4. Calculate the triggering time interval of the tray 101 passing through the camera 2 by acquiring the instantaneous speed of the high-speed cross belt 100, and adjust the number in the cache according to the triggering time interval.

[0098] In this embodiment, the step S21 of calculating the triggering time interval of the tray 101 passing through the camera 2 by acquiring the instantaneous speed of the high-speed cross belt 100 includes:

[0099] S41, calculating the triggering time interval t of the tray 101 passing the camera 2 平均 for:

[0100] ;

[0101] Where, Q is the number of instantaneous speed measurements in the second time interval; L 托盘间距 is the center distance between two adjacent trays 101; v(n) is the nth instantaneous speed of the high-speed cross belt 100.

[0102] Among them, since the movement speed of the high-speed cross belt 100 may change slightly, as well as the start-stop change problem, in order to accurately calculate the trigger time interval of the pallet 101 passing the camera 2, this embodiment takes into account the spacing and instantaneous speed of the pallet 101, and uses an integral calculation method to determine the time interval.

[0103] Meanwhile, in this embodiment, adjusting the serial number in the cache according to the trigger time interval includes the following:

[0104] S42. When the visible light images in the cache are numbered according to the sequence of their capture time in step S2, the visible light images in the cache are divided into a plurality of capture intervals according to the trigger time interval, each capture interval corresponds to a number, and when multiple visible light images appear in one capture interval, the optimal visible light image is selected for association according to the clarity of the visible light image and the proximity to the trigger time interval.

[0105] In the above method, if a trigger failure of false triggering occurs, since the overall numbering sequence is not affected and the optimal visible light image is selected, it is equivalent to not numbering the falsely triggered photos, so it will not affect the association between any abnormal package 200 and the visible light image. If the trigger failure is not triggered, it is just that the picture corresponding to the number of this capture interval does not exist, which only affects the association between the abnormal package 200 and the visible light image at this moment. Therefore, this embodiment can greatly reduce the impact of association misalignment.

[0106] A simple example is as follows: Assume that the trigger time interval is 0.1 seconds, and the upper and lower limits of the capture interval are 0.05 seconds respectively. If the actual capture interval is 0.095 seconds, 0.113 seconds, 0.098 seconds, and 0.096 seconds. Then there is only one visible image in each of the above capture intervals, and there is no trigger failure. If the actual capture interval is 0.095 seconds, 0.198 seconds, 0.098 seconds, and 0.096 seconds, there is no visible light image in the second capture interval, that is, there is a trigger failure of non-triggering. If the actual capture interval is 0.095 seconds, 0.059 seconds, 0.049 seconds, and 0.098 seconds, there are two visible light images in the second capture interval, that is, there is a trigger failure of false triggering. At this time, the visible light image with a capture interval of 0.049 seconds, which is close to the trigger time interval and has a higher definition, is selected as the visible light image that can be de-associated in this second capture interval.

[0107] S42', or when the first number of the abnormal package 200 in the cache is calculated in step S3, the trigger time interval is used to calibrate the actual capture interval, and the N obtained by the calculation formula is adjusted according to the calibration situation, and the adjusted first number N' is used as the visible light image number in the cache to obtain.

[0108] In the above method, the actual capture interval is calibrated. If a triggering failure is found, the actual capture interval will be abnormal, and thus the association between the abnormal package 200 and the visible light image at the moment of the triggering failure is adjusted, without affecting the association between the abnormal package 200 and the visible light image at other moments. Therefore, this embodiment can greatly reduce the impact of association misalignment.

[0109] A simple example is as follows: Assume that the trigger time interval is 0.1 seconds, and its fluctuation range is plus or minus 0.03 seconds. If the actual snapshot interval is 0.095 seconds, 0.113 seconds, 0.098 seconds, and 0.096 seconds, and N is 4 at this time, then the sum of the four snapshot intervals is 0.402 seconds, which is within the fluctuation range of the trigger time interval * 4 = 0.4 seconds, and N is not adjusted. If the actual snapshot interval is 0.095 seconds, 0.198 seconds, 0.098 seconds, and 0.096 seconds, and N is 4 at this time, then the sum of the four snapshot intervals is 0.484 seconds, which is outside the fluctuation range of the trigger time interval * 4 = 0.4 seconds, and the excess 0.084 seconds is close to 0.1 seconds, so N is reduced by 1, and the adjusted first number N' is 3, that is, there is a trigger failure situation where no triggering occurs. If the actual capture intervals are 0.095 seconds, 0.059 seconds, 0.049 seconds, and 0.098 seconds, and N is 4, the sum of these four capture intervals is 0.301 seconds, which is less than the fluctuation range of the trigger time interval*4=0.4 seconds, and the 0.099 seconds that is less than is close to 0.1 seconds, so N is added by 1, and the adjusted first number N' is 5.

[0110] In summary, this embodiment calculates the trigger time interval of the tray 101 passing through the camera 2 in the current time period to adjust the captured visible light image, so as to ensure that the order of subsequent visible light images will not be disordered when the visible light image is multiple-shot or missed, thereby realizing the automatic error correction function.

[0111] Embodiment 3

[0112] Please refer to Figure 1 to Figure 2 , a system for associating X-ray analysis and visible light images during high-speed transmission, includes an X-ray machine 1, a camera 2 and associated equipment, the X-ray machine 1 and the camera 2 are arranged on a high-speed cross belt 100, and the associated equipment can be arranged in the X-ray machine 1, can be arranged in the camera 2, or can be connected to the X-ray machine 1 and the camera 2 by wireless or wired communication.

[0113] Among them, the X-ray machine 1 is used to implement step S1 in the first embodiment, the camera 2 is used to implement step S2 in the first embodiment, and the associated device is used to implement step S3 in the first embodiment. As for step S4, it depends on whether the technical solution of step S41 or the technical solution of step S42 is adopted. If the former, it is implemented by the camera 2, and if the latter, it is implemented by the associated device.

[0114] Since the system / device described in the above embodiments of the present invention is a system / device used to implement the method of the above embodiments of the present invention, a person skilled in the art can understand the specific structure and deformation of the system / device based on the method described in the above embodiments of the present invention, and thus will not be described in detail here. All systems / devices used in the method of the above embodiments of the present invention belong to the scope of protection of the present invention.

[0115] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, apparatuses or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus) and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions.

[0117] It should be noted that in the claims, any reference numerals placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In the claims enumerating several means, several of these means may be embodied by the same hardware. The use of the words first, second, third, etc., is for convenience of expression only and does not indicate any order. These words may be understood as part of the component name.

[0118] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0119] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments after knowing the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0120] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include these modifications and variations.

Claims

1. A method for associating X-ray analysis with visible light images in high-speed transmission, applied to a high-speed cross belt, wherein the high-speed cross belt transmits packages via a tray and drives the packages through an X-ray machine and a camera, characterized in that: The method comprises the steps of: S1. Acquire an X-ray image of a package in real time, perform abnormal analysis on the package in the X-ray image, and if an abnormal package is identified, obtain location information of the abnormal package in the corresponding X-ray image; S2, acquiring and caching visible light images of the package in real time, and numbering the visible light images in the cache according to the time sequence of capturing the visible light images; S3, when the abnormal package is identified, the first number of the abnormal package in the cache is calculated according to the position information of the abnormal package in the corresponding X-ray image, the relative position of the X-ray machine and the camera, the average speed of the high-speed cross belt, the system processing time and the center distance between two adjacent trays, and the visible light image corresponding to the first number in the cache is associated with the abnormal package; The calculation formula of the first number N in step S3 is: ； Where N is rounded to an integer; D 摄像机 is the distance from the camera to the reference point, D X光机 is the distance from the X-ray machine to the reference point. The two use the moving direction of the high-speed cross belt as the positive coordinate axis and the side position where the starting point of the high-speed cross belt is located as the reference point; norm L is the horizontal scale coordinate of the abnormal package in the X-ray image, with the direction of high-speed cross-band movement as the positive direction; X光机 is the actual length of the high-speed cross-belt occupied by the X-ray machine display screen; v is the average speed of the high-speed cross-belt in the first time interval; t 分析 is the processing time of the X-ray image; t 校准 It is the calibration time performed by the user on site, and the default value is 0; L 托盘间距 The center distance between two adjacent pallets.

2. The method for associating X-ray analysis and visible light images in high-speed transmission according to claim 1, characterized in that: The first time interval is related to the number of visible light images in the cache.

3. The method for associating X-ray analysis and visible light images in high-speed transmission according to claim 1, characterized in that: Also includes the steps: The triggering time interval of the tray passing through the camera is calculated by acquiring the instantaneous speed of the high-speed cross belt, and the number in the cache is adjusted according to the triggering time interval.

4. The method for associating X-ray analysis and visible light images in high-speed transmission according to claim 3, characterized in that: The method of calculating the triggering time interval of the tray passing through the camera by acquiring the instantaneous speed of the high-speed cross belt comprises: Calculate the trigger time interval t when the tray passes the camera 平均 for: ； Where, Q is the number of instantaneous speed measurements in the second time interval; L 托盘间距 is the center distance between two adjacent pallets; v(n) is the nth instantaneous speed of the high-speed cross belt.

5. The method for associating X-ray analysis and visible light images in high-speed transmission according to claim 1, characterized in that: The abnormality analysis of the package in the X-ray image in step S1 includes: the following: The X-ray image is input into an improved YOLOv5 model, and the improved YOLOv5 model performs abnormal analysis on the package in the X-ray image, wherein the improved YOLOv5 model is obtained by introducing edge information into the YOLOv5 model.

6. The method for associating X-ray analysis and visible light images in high-speed transmission according to claim 5, characterized in that: The process of performing abnormal analysis on the package in the X-ray image by the improved YOLOv5 model is as follows: Obtain an input feature map F according to the X-ray image; Use the Laplacian operator to perform edge detection on the input feature map F and extract the edge feature map F edge ; For the edge feature map F edge To average: ； Among them, W edge is the edge feature map F edge The average edge weight of ; C is the number of channels, c is the serial number of each channel; H is the height of the edge feature map, i is the serial number of each pixel in the height direction of the edge feature map; W is the width of the edge feature map, j is the serial number of each pixel in the width direction of the edge feature map; Using the average edge weight W edge Perform global reweighting on the input feature map F: ； Among them, F' is the weighted feature map, is the field of real numbers; The weighted feature map F' is input into the Backbone part of the improved YOLOv5 model, and the edge enhancement features are integrated in the Head part to generate multiple detection boxes, and then the precise detection boxes and confidence scores are obtained through non-maximum suppression. The improved YOLOv5 model introduces a new loss function for balancing the edge enhancement features and the target box loss during training. The precise detection box is the abnormal object in the X-ray image.

7. The method for associating X-ray analysis and visible light images in high-speed transmission according to any one of claims 1 to 6, characterized in that: The X-ray image includes a plurality of packages, and each package appears on a plurality of X-ray images. Then, obtaining the position information of the abnormal package in the corresponding X-ray image in step S1 includes: From multiple X-ray images that identify the same abnormal object, an X-ray image with the highest confidence score is selected as the image to be associated, and the position information of the abnormal package in the image to be associated is obtained.

8. The method for associating X-ray analysis and visible light images in high-speed transmission according to any one of claims 1 to 6, characterized in that: The position information of the abnormal package in the corresponding X-ray image is the coordinates of ten thousandths.

9. A system for correlating X-ray analysis with visible light images during high-speed transmission, characterized in that: include: An X-ray machine is used to obtain an X-ray image of the package in real time, perform abnormal analysis on the package in the X-ray image, and if an abnormal package is identified, obtain the position information of the abnormal package in the corresponding X-ray image; A camera is used to acquire visible light images of the package in real time and cache them, and to number the visible light images in the cache according to the time sequence of capturing the visible light images; an associating device, for calculating, upon identifying the abnormal package, a first number of the abnormal package in a cache according to position information of the abnormal package in a corresponding X-ray image, a relative position of the X-ray machine and a camera, an average speed of a high-speed cross belt, a system processing time, and a center distance between two adjacent trays, and associating a visible light image corresponding to the first number in the cache with the abnormal package; The calculation formula of the first number N is: ； Where N is rounded to an integer; D 摄像机 is the distance from the camera to the reference point, D X光机 is the distance from the X-ray machine to the reference point. The two use the moving direction of the high-speed cross belt as the positive coordinate axis and the side position where the starting point of the high-speed cross belt is located as the reference point; norm L is the horizontal scale coordinate of the abnormal package in the X-ray image, with the direction of high-speed cross-band movement as the positive direction; X光机 is the actual length of the high-speed cross-belt occupied by the X-ray machine display screen; v is the average speed of the high-speed cross-belt in the first time interval; t 分析 is the processing time of the X-ray image; t 校准 It is the calibration time performed by the user on site, and the default value is 0; L 托盘间距 The center distance between two adjacent pallets.

Images