A dynamic calibration method, device, storage medium and equipment
By preprocessing and adjusting the endpoint offset of the manually coarsely calibrated images, the target calibration image is automatically constructed, solving the problems of calibration quality and efficiency, and achieving efficient calibration unaffected by personnel and camera shake.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN FAIRPLAY SPORTS DEV
- Filing Date
- 2023-04-14
- Publication Date
- 2026-05-29
AI Technical Summary
Existing calibration methods rely on the expertise of calibration personnel, resulting in unstable calibration quality, low efficiency, and susceptibility to camera shake.
By acquiring images that have been roughly calibrated manually, performing preprocessing and foreground segmentation, and using the vectors of reference lines to adjust endpoint offsets and repair abnormal endpoints, a target calibration image is automatically constructed.
It achieves calibration quality unaffected by calibration personnel, high calibration efficiency, and is unaffected by slight camera shaking, enabling rapid and automatic calibration.
Smart Images
Figure CN116485911B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of calibration technology, and in particular to a dynamic calibration method, apparatus, storage medium, and device. Background Technology
[0002] Calibration refers to surveying to determine boundary lines. For example, in some sports examination fields, calibration involves marking out scoring areas on an image to facilitate measurement and judgment of which scoring area an object (such as a ball) falls in, thereby determining the score. In other words, after calibration, the actual coordinate position of the object can be determined by its coordinate position on the image, thus determining which scoring area the object falls in and determining the score.
[0003] Currently, existing calibration methods mainly rely on manual calibration. However, this method depends too much on the professionalism of the calibration personnel, making the calibration quality susceptible to the influence of their expertise, and the calibration efficiency is low. In addition, since this calibration method requires calibration personnel to perform the calibration before measurement, if there is slight shaking of the camera after calibration, it is necessary to recalibrate and measure again, which means it is easily affected by external factors. Summary of the Invention
[0004] Based on this, it is necessary to propose a dynamic calibration method, device, storage medium and equipment to address the above problems, so that the calibration quality is not affected by the calibration personnel, the calibration efficiency is high, and the calibration process is not affected by slight camera shaking.
[0005] To achieve the above objectives, the present invention provides a dynamic calibration method in a first aspect, the method comprising:
[0006] Acquire a calibration image, which includes multiple coarse marker points and the captured image;
[0007] The calibration image is preprocessed based on multiple coarse marker points in the calibration image to obtain a vertical top view image. Multiple preset reference lines are placed in the vertical top view image, and the vertical top view image is segmented to obtain a foreground image.
[0008] Each reference line in the foreground image is segmented to obtain multiple line segments corresponding to each reference line;
[0009] After offsetting and adjusting the multiple endpoints of the multiple line segments corresponding to each reference line according to the vector of each reference line and the corresponding foreground boundary line in the foreground image, the multiple endpoints of the multiple line segments corresponding to each reference line are connected in sequence to obtain the boundary line corresponding to each reference line.
[0010] If none of the endpoints of each boundary line are abnormal endpoints, multiple boundary lines are used to form a target calibration image.
[0011] Optionally, the step of offsetting and adjusting multiple endpoints of multiple line segments corresponding to each reference line based on the vector of each reference line and the corresponding foreground boundary in the foreground image includes:
[0012] Based on the vector of each reference line, the multiple endpoints of the corresponding multi-segment line are sequentially offset and adjusted. When an endpoint touches the corresponding foreground boundary line in the foreground image, the offset adjustment of that endpoint stops.
[0013] Specifically, if the endpoint offset is adjusted to the maximum offset adjustment threshold, but the endpoint has not yet touched the corresponding foreground boundary in the foreground image, the endpoint is restored to its position before the offset adjustment.
[0014] Optionally, the method further includes:
[0015] In the case that there are abnormal endpoints among all the endpoints of each boundary line, determine the boundary lines with abnormal endpoints and the boundary lines without abnormal endpoints;
[0016] Identify multiple normal endpoints and at least one abnormal endpoint among all endpoints of each boundary line containing an abnormal endpoint;
[0017] For each boundary line with abnormal endpoints, multiple normal endpoints are fitted to obtain the fitted boundary line corresponding to each boundary line with abnormal endpoints. At least one abnormal endpoint in each boundary line with abnormal endpoints is repaired to the corresponding fitted boundary line to obtain the target boundary line corresponding to each boundary line with abnormal endpoints.
[0018] The target calibration image is formed by the target boundary lines corresponding to all boundary lines with abnormal endpoints and all boundary lines without abnormal endpoints.
[0019] Optionally, determining multiple normal endpoints and at least one abnormal endpoint among all endpoints of each boundary line with an abnormal endpoint includes:
[0020] Based on preset rules and the endpoint coordinates of all endpoints in each boundary line containing abnormal endpoints, determine multiple normal endpoints and at least one abnormal endpoint corresponding to each boundary line containing abnormal endpoints.
[0021] Optionally, the step of repairing at least one anomalous endpoint in each boundary line with anomalous endpoints to the corresponding fitted boundary line includes:
[0022] If the corresponding fitting boundary line is a circular fitting boundary line, then determine the center of the circle of the corresponding fitting boundary line, and take the intersection of the line connecting the center of the circle and each abnormal endpoint in the corresponding boundary line with the corresponding fitting boundary line as the repaired endpoint of each abnormal endpoint in the corresponding boundary line.
[0023] Optionally, the step of repairing at least one anomalous endpoint in each boundary line with anomalous endpoints to the corresponding fitted boundary line includes:
[0024] If the corresponding fitted boundary line is a straight fitted boundary line, then draw a perpendicular line to the corresponding fitted boundary line through each abnormal endpoint in the corresponding boundary line, and take the intersection of the perpendicular line of each abnormal endpoint in the corresponding boundary line and the corresponding fitted boundary line as the repaired endpoint of each abnormal endpoint in the corresponding boundary line.
[0025] Optionally, before segmenting each reference line in the foreground image to obtain multiple line segments corresponding to each reference line, the method further includes:
[0026] Morphological processing is performed on the foreground image.
[0027] To achieve the above objectives, the present invention provides a dynamic calibration device in a second aspect, the device comprising:
[0028] The acquisition module is used to acquire a calibration image, which includes multiple coarse marker points and captured images;
[0029] The processing module is used to preprocess the calibration image based on multiple coarse marker points in the calibration image to obtain a vertical top view image, place multiple preset reference lines into the vertical top view image, and perform foreground segmentation on the vertical top view image to obtain a foreground image.
[0030] The segmentation module is used to segment each reference line in the foreground image to obtain multiple line segments corresponding to each reference line;
[0031] The offset adjustment module is used to offset and adjust the multiple endpoints of the multiple line segments corresponding to each reference line according to the vector of each reference line and the corresponding foreground boundary line in the foreground image, and then connect the multiple endpoints of the multiple line segments corresponding to each reference line in sequence to obtain the boundary line corresponding to each reference line.
[0032] The fitting module is used to construct a target calibration image from multiple boundary lines, provided that none of the endpoints of each boundary line are abnormal endpoints.
[0033] To achieve the above objectives, the present invention provides, in a third aspect, a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method as described in any of the first aspects.
[0034] To achieve the above objectives, the present invention provides a computer device in a fourth aspect, including a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any of the first aspects.
[0035] The present invention provides the following advantages: The method acquires a calibration image (i.e., an image after rough calibration based on a manually captured image). The calibration image includes multiple coarse marker points and the captured image. Then, based on the multiple coarse marker points in the calibration image, it preprocesses the calibration image to obtain a vertical top-view image. Multiple preset reference lines are placed into the vertical top-view image, and foreground segmentation is performed on the vertical top-view image to obtain a foreground image. Next, each reference line in the foreground image is segmented to obtain multiple line segments corresponding to each reference line. Finally, based on the vector of each reference line and the corresponding foreground boundary line in the foreground image, the vector of each reference line is used to obtain multiple line segments corresponding to each reference line. After offsetting and adjusting the endpoints of multiple line segments, the endpoints of the multiple line segments corresponding to each reference line are connected sequentially to obtain the boundary line corresponding to each reference line. Finally, assuming that none of the endpoints of each boundary line are abnormal endpoints, the multiple boundary lines are used to construct the target calibration image. This involves acquiring a manually coarsely calibrated calibration image for automatic calibration. This calibration method involves a manual coarse calibration before measurement to determine the area to be calibrated. Therefore, the calibration quality is not affected by the calibrator, resulting in high efficiency. Furthermore, the calibration process is unaffected by slight camera shake, meaning it is not influenced by external factors. Additionally, this calibration method allows calibration to be performed at the start of measurement. Since the automatic calibration is fast, the calibration time is negligible, allowing for immediate calibration and measurement. Attached Figure Description
[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0037] in:
[0038] Figure 1 This is a schematic diagram of a dynamic calibration method according to an embodiment of this application;
[0039] Figure 2 This is a schematic diagram of images taken in an embodiment of this application;
[0040] Figure 3 This is a schematic diagram of the calibration image in the embodiments of this application;
[0041] Figure 4 This is a schematic diagram of a vertical top view image in an embodiment of this application;
[0042] Figure 5 This is a schematic diagram of the foreground image in an embodiment of this application;
[0043] Figure 6 This is a schematic diagram of the multiple line segments corresponding to each reference line after segmentation of each reference line in the foreground image in the embodiments of this application;
[0044] Figure 7 This is a schematic diagram of the boundary line after the offset adjustment and sequential connection of multiple endpoints of multiple line segments corresponding to one of the example reference lines in this application embodiment;
[0045] Figure 8 This is a schematic diagram illustrating the determination of normal and abnormal endpoints in one of the boundary lines where abnormal endpoints exist, as exemplified in this application embodiment.
[0046] Figure 9 This is a schematic diagram of the fitted boundary line obtained by fitting a normal endpoint in one of the boundary lines with abnormal endpoints in an example of this application embodiment.
[0047] Figure 10 This is a schematic diagram of the target boundary line obtained after repairing the abnormal endpoints of one of the boundary lines with abnormal endpoints in an example of this application to the corresponding fitted boundary line.
[0048] Figure 11 This is a schematic diagram of a dynamic calibration device according to an embodiment of this application;
[0049] Figure 12 This is a diagram showing the internal structure of a computer device in some embodiments. Detailed Implementation
[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0051] Please see Figure 1This is a schematic diagram of a dynamic calibration method according to an embodiment of this application. The method includes:
[0052] Step 110: Obtain the calibration image, which includes multiple coarse marker points and the captured image.
[0053] The captured images can be images from a video being filmed or single images. In this application, since the calibration is dynamic (i.e., calibration is continuously performed based on the images captured in real time), it mainly targets continuous images in the filmed video.
[0054] It should be noted that the calibration image is obtained by manually performing a rough calibration on the image being captured before automatic calibration. That is, by manually marking a few rough points on the captured image, the purpose of which is to determine the areas in the captured image that need to be calibrated. Of course, a rough calibration can also be performed based on machine recognition, programs, etc., to obtain a calibration image. However, this embodiment is not convenient for calibration in different scenarios. That is, it is necessary to re-perform corresponding machine recognition, program programming, etc. according to different calibration scenarios of the captured image. Compared with manual rough calibration, it has the problems of complexity, high cost and high manual workload. Therefore, it is preferable to use manual rough calibration to obtain the calibration image.
[0055] In some embodiments, before step 110, a click operation responsive to the operation button can be set to start the measurement, that is, to perform calibration when the measurement is about to begin. It can be understood that since the automatic calibration is fast, the calibration process time is negligible. Therefore, the calibration can be completed and the measurement can be performed immediately upon receiving the signal to start the measurement.
[0056] It should be further explained that the manual coarse calibration of the image being captured to obtain the calibration image is a preset step performed before starting the measurement, and this coarse calibration only needs to be performed once. It can be understood that when the captured image is an image from a video, since the video is continuously captured and calibrated on the same area that needs to be calibrated, it is only necessary to perform the coarse calibration manually once before starting the measurement, and then automatically calibrate the real-time captured image based on the few coarsely marked points made manually.
[0057] Step 120: Preprocess the calibration image based on multiple coarse marker points in the calibration image to obtain a vertical top view image, place multiple preset reference lines into the vertical top view image, and perform foreground segmentation on the vertical top view image to obtain a foreground image.
[0058] The preprocessing includes transformation and cropping, the purpose of which is to convert the calibration image into a top-down view, and during the transformation process, to crop away unnecessary areas that do not need to be calibrated. In this application, the foreground segmentation methods include, but are not limited to, grayscale thresholding and color extraction. The purpose of foreground segmentation is to segment the foreground boundary in the image (in the context of a badminton match in sports, the foreground boundary is the boundary of the badminton court) to facilitate subsequent calibration. The preset reference lines are pre-set by the operator according to the scene to be calibrated. For example, if the scene to be calibrated is a badminton court in sports, then the preset reference lines are the boundary of the badminton court.
[0059] It should be noted that since the calibration images were taken by setting up the camera on the ground, the calibration images need to be converted into vertical top-down images to make the foreground boundary lines more regular and simpler, which is conducive to subsequent calibration (if not converted, the foreground boundary lines with circular lines will become elliptical, which is not conducive to subsequent calibration).
[0060] In some embodiments, the area to be calibrated can be determined based on multiple coarse markers in the calibration image, then the excess area that does not need to be calibrated is cropped, and the calibration image is converted into a vertical top-view image, i.e., a vertical top-view image, based on the positions of the multiple coarse markers in the calibration image.
[0061] In some embodiments, placing multiple preset reference lines into the vertical top view image can facilitate foreground segmentation of the vertical top view image to obtain foreground boundaries, thereby obtaining a foreground image.
[0062] Step 130: Divide each reference line in the foreground image into segments to obtain multiple line segments corresponding to each reference line.
[0063] In this case, the reference line is formed by connecting the multiple line segments in sequence.
[0064] It should be noted that the reference lines in the foreground image are the preset multiple reference lines. It can be understood that after placing the preset multiple reference lines into the vertical top view image, and then performing foreground segmentation on the vertical top view image, the resulting foreground image will contain the preset multiple reference lines.
[0065] In some embodiments, each reference line in the foreground image can be segmented, that is, each reference line can be subdivided into multiple segments to obtain multiple segments corresponding to each reference line; it is understood that the more segments each reference line is divided into, the better the calibration quality of the final target calibration image.
[0066] Step 140: After offsetting and adjusting the multiple endpoints of the multiple line segments corresponding to each reference line according to the vector of each reference line and the corresponding foreground boundary line in the foreground image, connect the multiple endpoints of the multiple line segments corresponding to each reference line in sequence to obtain the boundary line corresponding to each reference line.
[0067] It should be noted that since the multiple line segments corresponding to each reference line were originally obtained by segmenting the same reference line, the offset adjustment directions of the multiple endpoints of the multiple line segments of the same reference line are the same and are all determined by the vector of the corresponding reference line. The number of foreground boundaries and reference lines in the foreground image are in a one-to-one correspondence. The foreground boundaries are the boundaries obtained when segmenting the foreground. That is, in the badminton match scene in the sports field, the foreground boundaries are the boundaries of the badminton court.
[0068] It should be further explained that after offsetting and adjusting the multiple endpoints of the multiple line segments corresponding to each reference line, there may be no connection between the multiple line segments. Therefore, it is necessary to connect the multiple endpoints of the multiple line segments corresponding to each reference line in sequence to obtain the boundary line corresponding to each reference line.
[0069] Step 150: If none of the endpoints of each boundary line are abnormal endpoints, construct a target calibration image from the multiple boundary lines.
[0070] It should be noted that after offsetting and adjusting the multiple endpoints of the multiple line segments corresponding to each reference line, there may be abnormal endpoint offset adjustments. Therefore, it is necessary to determine whether there are abnormal endpoints among all the endpoints of each boundary line. If there are no abnormal endpoints among all the endpoints of each boundary line, the multiple boundary lines can be directly used to form the target calibration image.
[0071] In this embodiment of the application, the calibration image after manual coarse calibration is obtained, and the calibration image is automatically calibrated. The calibration method is to perform a coarse calibration by hand before the measurement begins to determine the range to be calibrated. Therefore, the calibration quality is not affected by the calibration personnel, the calibration efficiency is high, and the calibration process is not affected by slight camera shaking, that is, it is not affected by external factors.
[0072] In one feasible implementation, step 140 in the above embodiment, which involves offsetting multiple endpoints of multiple line segments corresponding to each reference line based on the vector of each reference line and the corresponding foreground boundary line in the foreground image, includes: sequentially offsetting multiple endpoints of the corresponding multiple line segments based on the vector of each reference line; stopping the offset adjustment of an endpoint when it touches the foreground boundary line in the foreground image; and restoring the endpoint to its position before the offset adjustment when the endpoint offset adjustment reaches the maximum offset adjustment threshold and the endpoint has not yet touched the foreground boundary line in the foreground image.
[0073] The maximum offset adjustment threshold is obtained by the operator based on a large number of experiments or statistics. Of course, it can also be set according to the actual needs of the operator.
[0074] In this embodiment, multiple endpoints of the corresponding multi-segment line are sequentially offset and adjusted according to the vector of each reference line. The position to be offset and adjusted is determined according to the relationship between the endpoints, the foreground boundary line, and the maximum offset adjustment threshold, so that the offset and adjusted endpoints fit the foreground boundary line, thereby improving the calibration quality of the final calibrated target calibration image.
[0075] In one feasible implementation, the method in the above embodiments further includes: when there are abnormal endpoints among all endpoints of each boundary line, determining the boundary lines with abnormal endpoints and the boundary lines without abnormal endpoints; determining multiple normal endpoints and at least one abnormal endpoint among all endpoints of each boundary line with abnormal endpoints; fitting multiple normal endpoints in each boundary line with abnormal endpoints to obtain a fitted boundary line corresponding to each boundary line with abnormal endpoints, and repairing at least one abnormal endpoint in each boundary line with abnormal endpoints to the corresponding fitted boundary line to obtain a target boundary line corresponding to each boundary line with abnormal endpoints; and constructing a target calibration image by combining the target boundary lines corresponding to all boundary lines with abnormal endpoints and all boundary lines without abnormal endpoints.
[0076] It should be noted that after offsetting and adjusting multiple endpoints of multiple line segments corresponding to each reference line, there may be endpoints with abnormal offset adjustments. Therefore, it is necessary to determine whether there are abnormal endpoints among all endpoints of each boundary line. If there are abnormal endpoints among all endpoints of each boundary line, it is necessary to identify multiple normal endpoints and at least one abnormal endpoint among all endpoints of each boundary line with abnormal endpoints in order to distinguish the endpoints with abnormal offset adjustments.
[0077] In some embodiments, after determining multiple normal endpoints and at least one abnormal endpoint among all endpoints of each boundary line with abnormal endpoints, the multiple normal endpoints of each boundary line with abnormal endpoints can be fitted to obtain the fitted boundary line corresponding to each boundary line with abnormal endpoints. Then, at least one abnormal endpoint of each boundary line with abnormal endpoints is repaired to the corresponding fitted boundary line to obtain the target boundary line corresponding to each boundary line with abnormal endpoints. Alternatively, for all endpoints of each boundary line without abnormal endpoints, fitting is not required, and the target boundary line corresponding to all boundary lines with abnormal endpoints and all boundary lines without abnormal endpoints can be used to construct a target calibration image.
[0078] In this embodiment, different processing is applied to boundary lines with and without abnormal endpoints when abnormal endpoints exist among all endpoints of each boundary line. For boundary lines with abnormal endpoints, multiple normal endpoints and at least one abnormal endpoint need to be identified. Then, the normal endpoints are fitted, and the abnormal endpoints are repaired to obtain the target boundary line corresponding to each boundary line with abnormal endpoints. For boundary lines without abnormal endpoints, no fitting operation is required. Finally, the target boundary lines corresponding to all boundary lines with abnormal endpoints and all boundary lines without abnormal endpoints constitute the target calibration image. In other words, by fitting and repairing the boundary lines with abnormal endpoints, the calibration quality of the target calibration image can be effectively improved, resulting in better calibration.
[0079] In one feasible implementation, determining multiple normal endpoints and at least one abnormal endpoint among all endpoints of each boundary line with abnormal endpoints in the above embodiments includes: determining multiple normal endpoints and at least one abnormal endpoint corresponding to each boundary line with abnormal endpoints according to preset rules and the endpoint coordinates of all endpoints in each boundary line with abnormal endpoints.
[0080] It should be noted that the preset rules are set in advance by the operator. For example, in some embodiments, if the endpoint coordinates are in the form of (X, Y), the preset rules include determining whether to compare the X value, Y value, or both of the endpoint coordinates based on the vector of each boundary line with abnormal endpoints. Then, based on the relationship between the difference between the endpoint coordinates (X value and / or Y value) of all endpoints in each boundary line with abnormal endpoints and a preset threshold, multiple normal endpoints and at least one abnormal endpoint corresponding to each boundary line with abnormal endpoints are determined. Of course, this is just an example of this application, and the actual preset rules can be set according to the actual needs of the operator.
[0081] In this embodiment of the application, multiple normal endpoints and at least one abnormal endpoint corresponding to each boundary line are determined according to preset rules and the endpoint coordinates of all endpoints in each boundary line with abnormal endpoints, so as to distinguish the endpoints that are still abnormal after offset adjustment, and avoid the large error between the fitted boundary line obtained by fitting abnormal endpoints and the actual foreground boundary line, which would affect the calibration quality.
[0082] In one feasible implementation, the method in the above embodiments further includes: obtaining a calibration image based on a rough calibration of the captured image by manual means.
[0083] In this embodiment of the application, a calibration image is obtained by roughly calibrating the captured image manually, which facilitates the identification of the areas in the captured image that need to be calibrated, and also facilitates subsequent calibration processing.
[0084] In one feasible implementation, the step of repairing at least one abnormal endpoint in each boundary line with abnormal endpoints in the above embodiment to the corresponding fitted boundary line includes: if the corresponding fitted boundary line is a circular fitted boundary line, then determining the center of the corresponding fitted boundary line, and taking the intersection of the line connecting the center of the circle and each abnormal endpoint in the corresponding boundary line with the corresponding fitted boundary line as the repaired endpoint of each abnormal endpoint in the corresponding boundary line.
[0085] In this application, when the fitted boundary line is a circle, the center of the fitted boundary line is determined, and the intersection of the line connecting the center of the circle and each abnormal endpoint in the corresponding boundary line with the fitted boundary line is taken as the repaired endpoint of each abnormal endpoint in the corresponding boundary line. This achieves the repair of abnormal endpoints, ensuring that the target boundary line after repairing the abnormal endpoints to the fitted boundary line is consistent with the corresponding foreground boundary line, thus ensuring the integrity of the calibration and the quality of the calibration.
[0086] In one feasible implementation, the step of repairing at least one abnormal endpoint in each boundary line with abnormal endpoints in the above embodiment to the corresponding fitted boundary line includes: if the corresponding fitted boundary line is a straight fitted boundary line, then a perpendicular line is drawn through each abnormal endpoint in the corresponding boundary line to the corresponding fitted boundary line, and the intersection of the perpendicular line of each abnormal endpoint in the corresponding boundary line and the corresponding fitted boundary line is taken as the repaired endpoint of each abnormal endpoint in the corresponding boundary line.
[0087] In this application, when the fitted boundary line is a straight line, a perpendicular line is drawn through each abnormal endpoint in the corresponding boundary line to the fitted boundary line. The intersection of the perpendicular line of each abnormal endpoint in the corresponding boundary line and the fitted boundary line is taken as the endpoint after the abnormal endpoint in the corresponding boundary line is repaired. This is to repair the abnormal endpoints and ensure that the target boundary line after the abnormal endpoints are repaired to the fitted boundary line is consistent with the corresponding foreground boundary line, thus ensuring the integrity of the calibration and the quality of the calibration.
[0088] In one feasible implementation, before step 130 in the above embodiment, which involves segmenting each reference line in the foreground image to obtain multiple line segments corresponding to each reference line, the method further includes: performing morphological processing on the foreground image.
[0089] In this embodiment, morphological processing of the foreground image is performed to eliminate noise and connect the broken parts of the foreground boundary in the foreground image, which facilitates subsequent calibration and improves the quality of calibration.
[0090] One specific embodiment of this application will be described in the form of accompanying drawings, taking a badminton match scenario in the sports field as an example, specifically for marking the boundaries of a badminton court:
[0091] Please see Figure 2 The image shown is a schematic diagram of an image taken in an embodiment of this application. The image is of a badminton court, and there may be spectators and other objects nearby.
[0092] Please see Figure 3 This is a schematic diagram of a calibration image in an embodiment of this application. The calibration image includes multiple roughly marked points made manually and a photographed image (the boundary lines of a badminton court).
[0093] Please see Figure 4 This is a schematic diagram of a vertical top view image after placing multiple preset reference lines into the vertical top view image in this embodiment of the application. The multiple preset reference lines in the schematic diagram are the boundary lines of the badminton court corresponding to this embodiment. The calibration image is preprocessed according to multiple coarse markers in the calibration image, that is, the calibration image is converted into a vertical top view image. At the same time as the conversion, the excess areas such as spectators and objects are cropped to obtain a vertical top view image. The multiple preset reference lines are then placed into the vertical top view image to obtain this schematic diagram.
[0094] Please see Figure 5 This is a schematic diagram of the foreground image in an embodiment of this application. After placing multiple preset reference lines into the vertical top view image, the vertical top view image is then segmented to obtain the foreground boundary line (i.e., the boundary line of the badminton court), thereby obtaining the foreground image.
[0095] Please see Figure 6 This is a schematic diagram of the multiple line segments corresponding to each reference line after segmentation of each reference line in the foreground image in this application embodiment. Each reference line in the foreground image is segmented, and each reference line is divided into multiple line segments, thereby obtaining the multiple line segments corresponding to each reference line.
[0096] Please see Figure 7 This is a schematic diagram of the boundary line obtained by offsetting and sequentially connecting multiple endpoints of multiple line segments corresponding to one of the reference lines in an embodiment of this application. As can be seen from the diagram, after offsetting and adjusting multiple endpoints of multiple line segments corresponding to the reference line according to the vector of the reference line and the corresponding foreground boundary line in the foreground image, the multiple endpoints of multiple line segments corresponding to the reference line are sequentially connected to obtain the boundary line corresponding to the reference line. According to this example, after offsetting and adjusting multiple endpoints of multiple line segments corresponding to each reference line and sequentially connecting them, the boundary line corresponding to each reference line can be obtained.
[0097] Please see Figure 8 This is a schematic diagram illustrating the determination of normal and abnormal endpoints in one of the boundary lines with abnormal endpoints in an embodiment of this application. The diagram shows that when abnormal endpoints exist among all endpoints of the boundary line, it is necessary to determine multiple normal endpoints and at least one abnormal endpoint among all endpoints of the boundary line. There are 3 abnormal endpoints and 7 normal endpoints. According to this example, the same operation can be performed on each boundary line with abnormal endpoints to determine multiple normal endpoints and at least one abnormal endpoint among all endpoints of each boundary line with abnormal endpoints. Of course, this operation is not necessary for boundary lines without abnormal endpoints.
[0098] Please see Figure 9 This is a schematic diagram of a fitted boundary line obtained by fitting normal endpoints in one of the boundary lines with abnormal endpoints in an example of this application embodiment. As can be seen from the diagram, when there are abnormal endpoints among all the endpoints of the boundary line, it is necessary to fit the seven normal endpoints of the boundary line to obtain the fitted boundary line corresponding to the boundary line. According to this example, multiple normal endpoints in each boundary line with abnormal endpoints can be fitted to obtain the fitted boundary line corresponding to each boundary line with abnormal endpoints. Of course, this operation is not necessary for boundary lines without abnormal endpoints.
[0099] Please see Figure 10This diagram illustrates the target boundary line obtained by repairing the abnormal endpoints of one of the boundary lines with abnormal endpoints in an example embodiment of this application to the corresponding fitted boundary lines. As can be seen, the target boundary line is obtained by repairing all three abnormal endpoints of the boundary line to the corresponding fitted boundary lines. According to this example, at least one abnormal endpoint in each boundary line with abnormal endpoints can be repaired to the corresponding fitted boundary line to obtain multiple target boundary lines. Of course, this operation is not necessary for boundary lines without abnormal endpoints.
[0100] Finally, the target boundary lines corresponding to all boundary lines with abnormal endpoints and all boundary lines without abnormal endpoints are combined to form a target calibration image, which is the calibration of the badminton court boundary. Of course, if none of the boundary lines have abnormal endpoints, multiple boundary lines can be combined to form a target calibration image.
[0101] In some embodiments, this application also provides a dynamic calibration device.
[0102] Please see Figure 11 This is a schematic diagram of a dynamic calibration device according to an embodiment of this application. The device 1110 includes:
[0103] The acquisition module 1111 is used to acquire a calibration image, which includes multiple coarse marker points and captured images;
[0104] The processing module 1112 is used to preprocess the calibration image based on multiple coarse marker points in the calibration image to obtain a vertical top view image, put multiple preset reference lines into the vertical top view image, and perform foreground segmentation on the vertical top view image to obtain a foreground image.
[0105] The segmentation module 1113 is used to segment each reference line in the foreground image to obtain multiple line segments corresponding to each reference line;
[0106] The offset adjustment module 1114 is used to offset and adjust the multiple endpoints of the multiple line segments corresponding to each reference line according to the vector of each reference line and the corresponding foreground boundary line in the foreground image, and then connect the multiple endpoints of the multiple line segments corresponding to each reference line in sequence to obtain the boundary line corresponding to each reference line.
[0107] The fitting module 1115 is used to construct a target calibration image from multiple boundary lines when none of the endpoints of each boundary line are abnormal endpoints.
[0108] In this embodiment of the application, the relevant contents of the acquisition module 1111, processing module 1112, segmentation module 1113, offset adjustment module 1114 and fitting and constructing module 1115 can be found in the following references. Figure 1The contents of the illustrated embodiments will not be repeated here.
[0109] It should be noted that the device 1110 of this application also includes other modules. It is understood that the method of this application and the device 1110 have a one-to-one correspondence. Therefore, the other modules of the device 1110 of this application are the contents corresponding to the method of this application in the above embodiments.
[0110] In this embodiment of the application, the calibration image after manual coarse calibration is obtained, and the calibration image is automatically calibrated. The calibration method is to perform a coarse calibration by hand before the measurement begins to determine the range to be calibrated. Therefore, the calibration quality is not affected by the calibration personnel, the calibration efficiency is high, and the calibration process is not affected by slight camera shaking, that is, it is not affected by external factors.
[0111] In some embodiments, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform a dynamic calibration method in the above-described method embodiments.
[0112] In some embodiments, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs a dynamic calibration method in the above method embodiments.
[0113] Figure 12 The diagram illustrates the internal structure of a computer device in some embodiments. This computer device may specifically be a terminal, a server, or a gateway. Figure 12 As shown, the computer device includes a processor, memory, and network interface connected via a system bus.
[0114] The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When executed by a processor, this computer program causes the processor to perform the steps in the above method embodiments. The internal memory may also store a computer program, which, when executed by a processor, causes the processor to perform the steps in the above method embodiments. Those skilled in the art will understand that... Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0115] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above.
[0116] Any references to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.
[0117] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0118] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A dynamic calibration method, characterized in that, The method includes: Acquire a calibration image, which is an image after a rough calibration is performed manually on the image being captured before the measurement begins. The calibration image includes multiple rough marker points and the captured image. The calibration image is preprocessed based on multiple coarse marker points in the calibration image to obtain a vertical top view image. Multiple preset reference lines are placed in the vertical top view image, and the vertical top view image is segmented into a foreground image. The multiple preset reference lines are the site boundaries corresponding to the calibration scene. Each reference line in the foreground image is segmented to obtain multiple line segments corresponding to each reference line; After offsetting and adjusting the multiple endpoints of the multiple line segments corresponding to each reference line according to the vector of each reference line and the corresponding foreground boundary line in the foreground image, the multiple endpoints of the multiple line segments corresponding to each reference line are connected in sequence to obtain the boundary line corresponding to each reference line. If none of the endpoints of each boundary line are abnormal endpoints, multiple boundary lines are used to form a target calibration image. The step of offsetting and adjusting multiple endpoints of multiple line segments corresponding to each reference line based on the vector of each reference line and the corresponding foreground boundary line in the foreground image includes: Based on the vector of each reference line, the multiple endpoints of the corresponding multi-segment line are sequentially offset and adjusted. When an endpoint touches the corresponding foreground boundary line in the foreground image, the offset adjustment of that endpoint stops. Specifically, if the endpoint offset is adjusted to the maximum offset adjustment threshold, but the endpoint has not yet touched the corresponding foreground boundary in the foreground image, the endpoint is restored to its position before the offset adjustment.
2. The method according to claim 1, characterized in that, The method further includes: For each boundary line, based on whether there are abnormal endpoints among its endpoints, we determine the boundary lines with abnormal endpoints and the boundary lines without abnormal endpoints respectively. Identify multiple normal endpoints and at least one abnormal endpoint among all endpoints of each boundary line containing an abnormal endpoint; For each boundary line with abnormal endpoints, multiple normal endpoints are fitted to obtain the fitted boundary line corresponding to each boundary line with abnormal endpoints. At least one abnormal endpoint in each boundary line with abnormal endpoints is repaired to the corresponding fitted boundary line to obtain the target boundary line corresponding to each boundary line with abnormal endpoints. The target calibration image is formed by the target boundary lines corresponding to all boundary lines with abnormal endpoints and all boundary lines without abnormal endpoints.
3. The method according to claim 2, characterized in that, The determination of multiple normal endpoints and at least one abnormal endpoint among all endpoints of each boundary line containing an abnormal endpoint includes: Based on preset rules and the endpoint coordinates of all endpoints in each boundary line with abnormal endpoints, determine multiple normal endpoints and at least one abnormal endpoint corresponding to each boundary line with abnormal endpoints. The preset rules include: Based on the vector of each boundary line with anomaly endpoints, determine the X and / or Y values of the endpoint coordinates of each endpoint in each boundary line with anomaly endpoints. Based on the X-values of the endpoint coordinates of all endpoints in each boundary line with an anomalous endpoint, determine the first difference between all endpoints in each boundary line with an anomalous endpoint, and / or based on the Y-values of the endpoint coordinates of all endpoints in each boundary line with an anomalous endpoint, determine the second difference between all endpoints in each boundary line with an anomalous endpoint. Based on the comparison between the first difference and / or the second difference between all endpoints in each boundary line with abnormal endpoints and a preset threshold, multiple normal endpoints and at least one abnormal endpoint are determined for each boundary line with abnormal endpoints.
4. The method according to claim 2, characterized in that, The step of repairing at least one anomalous endpoint in each boundary line with anomalous endpoints to the corresponding fitted boundary line includes: If the corresponding fitting boundary line is a circular fitting boundary line, then determine the center of the circle of the corresponding fitting boundary line, and take the intersection of the line connecting the center of the circle and each abnormal endpoint in the corresponding boundary line with the corresponding fitting boundary line as the repaired endpoint of each abnormal endpoint in the corresponding boundary line.
5. The method according to claim 2, characterized in that, The step of repairing at least one anomalous endpoint in each boundary line with anomalous endpoints to the corresponding fitted boundary line includes: If the corresponding fitted boundary line is a straight fitted boundary line, then draw a perpendicular line to the corresponding fitted boundary line through each abnormal endpoint in the corresponding boundary line, and take the intersection of the perpendicular line of each abnormal endpoint in the corresponding boundary line and the corresponding fitted boundary line as the repaired endpoint of each abnormal endpoint in the corresponding boundary line.
6. The method according to claim 1, characterized in that, Before segmenting each reference line in the foreground image to obtain multiple line segments corresponding to each reference line, the method further includes: Morphological processing is performed on the foreground image.
7. A dynamic calibration device, characterized in that, The device includes: The acquisition module is used to acquire a calibration image, which is an image after a rough calibration is performed manually on the image being captured before the measurement begins. The calibration image includes multiple rough marker points and the captured image. The processing module is used to preprocess the calibration image based on multiple coarse marker points in the calibration image to obtain a vertical top view image, place multiple preset reference lines into the vertical top view image, and perform foreground segmentation on the vertical top view image to obtain a foreground image, wherein the multiple preset reference lines are the site boundaries corresponding to the calibration scene. The segmentation module is used to segment each reference line in the foreground image to obtain multiple line segments corresponding to each reference line; The offset adjustment module is used to offset and adjust the multiple endpoints of the multiple line segments corresponding to each reference line according to the vector of each reference line and the corresponding foreground boundary line in the foreground image, and then connect the multiple endpoints of the multiple line segments corresponding to each reference line in sequence to obtain the boundary line corresponding to each reference line. The fitting module is used to construct a target calibration image from multiple boundary lines, provided that none of the endpoints of each boundary line are abnormal endpoints. The step of offsetting and adjusting multiple endpoints of multiple line segments corresponding to each reference line based on the vector of each reference line and the corresponding foreground boundary line in the foreground image includes: Based on the vector of each reference line, the multiple endpoints of the corresponding multi-segment line are sequentially offset and adjusted. When an endpoint touches the corresponding foreground boundary line in the foreground image, the offset adjustment of that endpoint stops. Specifically, if the endpoint offset is adjusted to the maximum offset adjustment threshold, but the endpoint has not yet touched the corresponding foreground boundary in the foreground image, the endpoint is restored to its position before the offset adjustment.
8. A computer-readable storage medium, characterized in that, The system stores a computer program that, when executed by a processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 6.
9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 6.