Negative sample acquisition method and device, electronic equipment and target detection method
By setting up camera equipment at both ends of a one-way street, negative samples are automatically collected using coordinate mapping and feature information, solving the problem of time-consuming and labor-intensive negative sample collection in existing technologies, and improving the accuracy and efficiency of target detection.
Patent Information
- Application Number
- CN202310515316.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-06
- Publication Date
- 2024-11-08
AI Technical Summary
In existing technologies, collecting negative samples is time-consuming and labor-intensive, making it difficult to efficiently obtain negative samples to improve the accuracy of target detection results, especially when the target object is occluded or at a back or side angle, which can easily lead to errors.
By setting up a first camera and a second camera at opposite ends of a one-way street, the location matching results of the target object are automatically obtained using coordinate mapping relationships. Combined with feature information, false detection, multiple detection, and missed detection sample data are collected to achieve automated negative sample collection.
It improves the efficiency of negative sample collection, reduces manual intervention, enhances the richness and quality of negative sample data, and improves the accuracy of target detection, especially in cases of false detection and missed detection when viewed from behind or to the side.
Smart Images

Figure CN118918407A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of target detection technology, and in particular to a negative sample acquisition method, device, electronic equipment and target detection method. Background Art
[0002] When the target object is blocked or facing away from the target, it is easy to get the target detection result wrong. We can improve the accuracy of the target detection result by collecting negative samples with wrong target detection results and re-calibrating and retraining the target detection.
[0003] In the prior art, various monitoring scenarios are constructed during indicator testing, and negative samples with erroneous detection results are collected based on manual analysis of the indicator test results. However, this method of manually collecting negative samples has the disadvantage of being time-consuming and labor-intensive.
[0004] Therefore, how to better collect negative samples and improve the efficiency of collecting negative samples is a technical problem that needs to be urgently solved by technicians in related fields. Summary of the invention
[0005] The present invention provides a negative sample acquisition method, device, electronic device and target detection method, which are used to solve the technical problems in the prior art of how to better collect negative samples and improve the collection efficiency of negative samples.
[0006] The present invention provides a negative sample acquisition method, comprising:
[0007] Acquire first position information of at least one first target object captured by a first camera device, where the first position information represents position information of the first target object in a coordinate system where the first camera device is located;
[0008] Acquire second position information of at least one second target object captured by a second camera device, wherein the second position information represents position information of the second target object in a coordinate system where the second camera device is located, and the second camera device is arranged relative to the first camera device;
[0009] Based on the first position information, the second position information and the coordinate mapping relationship between the first camera device and the second camera device, a position matching result between the first target object and the second target object is obtained, and target negative sample data is obtained based on the position matching result.
[0010] According to a negative sample acquisition method provided by the present invention, the position matching result between the first target object and the second target object is acquired based on the first position information, the second position information, and the coordinate mapping relationship between the first camera device and the second camera device, including:
[0011] Determine, based on the second position information of the second target object and the coordinate mapping relationship, third position information of the second target object in the coordinate system where the first camera device is located;
[0012] Based on the first position information of the first target object, determining a first coordinate area of the first target object in the coordinate system where the first camera device is located;
[0013] Based on the third position information of the second target object, determining a second coordinate area of the second target object in the coordinate system where the first camera device is located;
[0014] Acquire an overlapping coordinate area of the first coordinate area and the second coordinate area, and determine a first overlapping parameter based on the overlapping coordinate area and the first coordinate area;
[0015] A second overlapping parameter is determined based on the overlapping coordinate area and the second coordinate area, and a position matching result between the first target object and the second target object is determined based on the first overlapping parameter and the second overlapping parameter.
[0016] According to a negative sample acquisition method provided by the present invention, the target negative sample data includes false positive sample data; the position matching result includes a target object group that has been successfully matched; and the step of acquiring the target negative sample data based on the position matching result includes:
[0017] Acquire target feature information corresponding to the first target object and the second target object in the target object group;
[0018] Determine a feature matching result between a first target object and a second target object in the target object group based on the target feature information;
[0019] In a case where it is determined based on the feature matching result that there is a false detection in the target object group, acquiring a first video image corresponding to a first target object in the target object group captured by the first camera device;
[0020] The false detection sample data is determined based on a first video image corresponding to a first target object in the target object group.
[0021] According to a negative sample acquisition method provided by the present invention, the target negative sample data includes multiple inspection sample data; the position matching result includes a first target object that fails to match; and acquiring the target negative sample data based on the position matching result includes:
[0022] Acquire a second video image corresponding to the first target object that failed to match and captured by the first camera device, wherein the first target object that failed to match includes a target object captured by the first camera device and not captured by the second camera device;
[0023] The multi-inspection sample data is determined based on the second video image corresponding to the first target object that fails to match.
[0024] According to a negative sample acquisition method provided by the present invention, the target negative sample data includes missed detection sample data; the position matching result includes a second target object that fails to match; and acquiring the target negative sample data based on the position matching result includes:
[0025] Acquire target spatiotemporal information corresponding to the second target object that failed to match when the second camera device captures the second target object that failed to match, wherein the second target object that failed to match includes a target object that was captured by the second camera device but not by the first camera device;
[0026] Based on the target spatiotemporal information, obtaining a third video image corresponding to the second target object that failed to match and was captured by the first camera device;
[0027] The missed detection sample data is determined based on the third video image corresponding to the second target object that fails to match.
[0028] The present invention also provides a negative sample acquisition device, comprising:
[0029] A first position acquisition module, used to acquire first position information of at least one first target object captured by a first camera device, wherein the first position information represents position information of the first target object in a coordinate system where the first camera device is located;
[0030] A second position acquisition module, used to acquire second position information of at least one second target object captured by a second camera device, wherein the second position information represents position information of the second target object in a coordinate system where the second camera device is located, and the second camera device is arranged relative to the first camera device;
[0031] A sample data acquisition module is used to obtain a position matching result between the first target object and the second target object based on the first position information, the second position information and the coordinate mapping relationship between the first camera device and the second camera device, and to obtain target negative sample data based on the position matching result.
[0032] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the negative sample acquisition methods described above is implemented.
[0033] The present invention also provides a target detection method, comprising:
[0034] Acquire an image to be detected, and input the image to be detected into a pre-trained target detection model to obtain a target detection result corresponding to the image to be detected;
[0035] The target detection model is obtained by training an initial detection model based on target negative sample data, and the target negative sample data is obtained based on any of the negative sample acquisition methods described above.
[0036] The present invention also provides a target detection device, comprising:
[0037] The target detection module is used to acquire an image to be detected and input the image to be detected into a pre-trained target detection model to obtain a target detection result corresponding to the image to be detected; wherein the target detection model is obtained by training an initial detection model based on target negative sample data, and the target negative sample data is obtained based on any of the negative sample acquisition methods described above.
[0038] The present invention also provides a camera device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the target detection method as described above is implemented.
[0039] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the negative sample acquisition method as described in any one of the above or the target detection method as described in any one of the above.
[0040] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the negative sample acquisition method as described in any one of the above or the target detection method as described in any one of the above.
[0041] The present invention also provides a negative sample acquisition system, comprising: a first camera device and a second camera device arranged on a one-way street, wherein: the lens orientation of the first camera device is consistent with the road direction of the one-way street;
[0042] The first camera device is communicatively connected to the second camera device, and is used to execute any one of the negative sample acquisition methods described above.
[0043] The negative sample acquisition method, device, electronic device and target detection method provided by the present invention utilize a first camera device and a second camera device that are relatively arranged to obtain first position information of a first target object captured at a back shooting angle and second position information of a second target object captured at a front shooting angle, thereby obtaining a position matching result between the first target object and the second target object based on the first position information, the second position information and the coordinate mapping relationship between the first camera device and the second camera device, and obtaining target negative sample data based on the position matching result to realize automatic collection of negative sample data. The entire negative sample collection process is automatically run by a computer without manual intervention, thereby avoiding the time-consuming and labor-intensive defects of manually collecting negative samples in the prior art, and further effectively improving the collection efficiency of negative samples, thereby solving the technical problems of how to better collect negative samples and improve the collection efficiency of negative samples in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0045] Figure 1 This is one of the flowcharts of the negative sample acquisition method provided by an embodiment of the present invention;
[0046] Figure 2 is a schematic diagram of installation positions of a first camera device and a second camera device on a one-way street in an embodiment of the present invention;
[0047] Figure 3 This is the second flowchart of the negative sample acquisition method provided by the embodiment of the present invention;
[0048] Figure 4 This is the third flow chart of the negative sample acquisition method provided by the embodiment of the present invention;
[0049] Figure 5 This is a fourth flow chart of the negative sample acquisition method provided in an embodiment of the present invention;
[0050] Figure 6 This is a fifth flow chart of the negative sample acquisition method provided in an embodiment of the present invention;
[0051] Figure 7 is a schematic structural diagram of a negative sample acquisition device provided by an embodiment of the present invention;
[0052] Figure 8It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0054] Combine the following Figure 1-Figure 6 The negative sample acquisition method provided by the present invention is described. Figure 1 As shown, the present invention provides a negative sample acquisition method, comprising:
[0055] Step 101: Acquire first position information of at least one first target object captured by a first camera device, where the first position information represents position information of the first target object in a coordinate system where the first camera device is located.
[0056] The first target object refers to the target object captured by the first camera device at the back shooting angle. Optionally, the first camera device is a network camera (IP Camera, IPC for short).
[0057] Step 102, obtaining second position information of at least one second target object captured by a second camera device, wherein the second position information represents position information of the second target object in a coordinate system where the second camera device is located, and the second camera device is arranged relative to the first camera device.
[0058] The second target object refers to the target object captured by the second camera device under the front shooting angle. Optionally, the second camera device is a network camera (IP Camera, abbreviated as IPC).
[0059] In one embodiment, the first camera device and the second camera device are arranged opposite to each other on a one-way street, and the lens orientation of the first camera device is consistent with the road direction of the one-way street.
[0060] Figure 2 Schematic diagram of the installation positions of the first camera device and the second camera device on a one-way street in an embodiment of the present invention. Figure 2As shown, since the first camera device and the second camera device are arranged oppositely at the two ends of the one-way street, and the lens direction of the first camera device is consistent with the road direction of the one-way street, the lens direction of the second camera device is opposite to the road direction of the one-way street. It can be understood that the first camera device is installed at one end of the one-way street with its back facing, and the second camera device is installed at the other end of the one-way street with its front facing, and the lens directions of the first camera device and the second camera device are opposite or opposite, where "opposite" means face to face.
[0061] Further, Figure 2 The first target object in the image represents the target object captured by the first camera device on the one-way street. When the first target object walks along the one-way street, the first camera device can capture the image of the first target object with its back or sideways. Figure 2 The second target object in the image represents the target object captured by the second camera device on the one-way street. When the second target object walks along the one-way street, the second camera device can capture an image of the second target object walking toward it from the front. It can be seen that the shooting angles of the first camera device and the second camera device are different.
[0062] It should be noted that, by arranging the first camera device and the second camera device opposite to each other at the two ends of the one-way street, the first camera device and the second camera device can completely monitor the target object on the one-way street.
[0063] Step 103 , based on the first position information, the second position information and the coordinate mapping relationship between the first camera device and the second camera device, obtain a position matching result between the first target object and the second target object, and obtain target negative sample data based on the position matching result.
[0064] Furthermore, by calibrating the camera images of the first camera device and the second camera device to construct a coordinate mapping relationship between the first camera device and the second camera device, since the specific calibration process and calculation process are mature technical solutions, this application will not repeat them.
[0065] In the above steps 101 to 103, the first position information of the first target object captured under the back shooting angle and the second position information of the second target object captured under the front shooting angle are obtained by utilizing the relatively arranged first camera device and the second camera device, thereby obtaining the position matching result between the first target object and the second target object based on the first position information, the second position information and the coordinate mapping relationship between the first camera device and the second camera device, and obtaining the target negative sample data based on the position matching result to realize automatic collection of negative sample data. The entire negative sample collection process is automatically run by the computer without human intervention, thereby avoiding the time-consuming and labor-intensive defects of manually collecting negative samples in the prior art, and further effectively improving the collection efficiency of negative samples, thereby solving the technical problem of how to better collect negative samples and improve the collection efficiency of negative samples in the prior art.
[0066] In one embodiment, the position information of the first target object and the second target object is converted to the same coordinate system based on the coordinate mapping relationship, and the position matching result between the first target object and the second target object is determined based on the position information of the first target object and the second target object in the same coordinate system.
[0067] Furthermore, the position matching result is used to determine whether the first target object and the second target object are the same target object captured by different camera devices. The position matching result includes at least one of the target object group that successfully matches, the first target object that failed to match, and the second target object that failed to match.
[0068] Among them, the "successfully matched target object group" includes the first target object and the second target object whose positions are successfully matched, which can also be understood as the first target object and the second target object being the same target object captured by different camera devices. For the "first target object that failed to match", the position matching results between the first target object and each second target object are all failed matches. For the "second target object that failed to match", the position matching results between the second target object and each first target object are all failed matches.
[0069] In the above embodiment, the position information of the second target object is converted into the coordinate system of the first camera device by utilizing the coordinate mapping relationship between the first camera device and the second camera device, so as to unify the shooting angles of the first target object and the second target object, thereby judging whether the first target object and the second target object are the same target object based on the position information under the same shooting angle, thereby obtaining the position matching result of the first target object and the second target object, and collecting different types of negative sample data based on different position matching results, thereby improving the type richness of the negative sample data, and further realizing how to better collect negative samples and improve the quality of negative samples.
[0070] In one embodiment, the detection rate and attribute recognition rate of the second camera device for the target object at the front angle are greater than a preset probability threshold, and the communication delay between the first camera device and the second camera device is less than or equal to the preset delay threshold. For example, the detection rate and attribute recognition rate of the second camera device for the target object at the front angle can reach 99% to 100%. The communication delay between the first camera device and the second camera device is less than or equal to 1 millisecond (ms).
[0071] In one embodiment, Figure 3 As shown, based on the first position information, the second position information and the coordinate mapping relationship between the first camera device and the second camera device, the step of obtaining the position matching result between the first target object and the second target object specifically includes:
[0072] Step 201: Determine third position information of the second target object in the coordinate system where the first camera device is located based on the second position information and the coordinate mapping relationship of the second target object.
[0073] Further, based on the coordinate mapping relationship between the first camera device and the second camera device, the second position information of the second target object in the coordinate system of the second camera device is converted into the third position information of the second target object in the coordinate system of the first camera device. The second position information includes the position coordinates of the second target object in the second camera device, and the third position information includes the position coordinates of the second target object in the first camera device.
[0074] Step 202: Determine a first coordinate area of the first target object in a coordinate system where the first camera device is located based on first position information of the first target object.
[0075] Step 203: Determine a second coordinate area of the second target object in the coordinate system where the first camera device is located based on the third position information of the second target object.
[0076] Step 204 : Acquire an overlapping coordinate area of the first coordinate area and the second coordinate area, and determine a first overlapping parameter based on the overlapping coordinate area and the first coordinate area.
[0077] Further, the first area of the overlapping coordinate area and the second area of the first coordinate area are obtained. The first area of the overlapping coordinate area is divided by the second area of the first coordinate area to obtain a first overlap parameter. The first overlap parameter represents the area ratio of the overlapping coordinate area to the first coordinate area.
[0078] Step 205 : determining a second overlap parameter based on the overlapped coordinate area and the second coordinate area, and determining a position matching result between the first target object and the second target object based on the first overlap parameter and the second overlap parameter.
[0079] Further, the third area of the second coordinate area is obtained. The first area of the overlapping coordinate area is divided by the third area of the second coordinate area to obtain a second overlapping parameter. The second overlapping parameter represents the area ratio of the overlapping coordinate area to the second coordinate area.
[0080] Further, it is determined whether the first overlap parameter and the second overlap parameter are greater than a preset overlap threshold, and when at least one of the first overlap parameter and the second overlap parameter is greater than the preset overlap threshold, it is determined that the position matching result of the first target object and the second target object is a successful position matching. The preset overlap threshold can be set to 0.8 to 1.
[0081] Alternatively, when the first overlap parameter is greater than a preset overlap threshold and the second overlap parameter is greater than the preset overlap threshold, it is determined that the position matching result of the first target object and the second target object is a successful position matching.
[0082] In one embodiment, Figure 4 As shown, the target negative sample data includes false positive sample data; the position matching result includes a target object group that has been successfully matched; and the steps of obtaining the target negative sample data based on the position matching result specifically include:
[0083] Step 301: Obtain target feature information corresponding to a first target object and a second target object in a target object group.
[0084] Furthermore, the target feature information includes category information and attribute information. The category information of the target object includes a motor vehicle or a pedestrian. The attribute information includes the gender and color of the target object.
[0085] Step 302: Determine a feature matching result between a first target object and a second target object in the target object group based on the target feature information.
[0086] Furthermore, the first target feature information of the first target object in the target object group is matched with the second target feature information of the second target object to obtain a feature matching result between the first target object and the second target object in the target object group.
[0087] Step 303: when it is determined based on the feature matching result that there is a false detection in the target object group, a first video image corresponding to the first target object in the target object group captured by the first camera device is obtained.
[0088] Furthermore, when the feature matching result is a feature matching failure, it is determined that there is a false detection in the target object group corresponding to the feature matching result.
[0089] It should be noted that, since the first target object and the second target object in the target object group are determined to be the same target object through position matching, but the conclusion is drawn that they are not the same target object during feature matching. This is because the first camera device captures the back image features of the first target object, while the second camera device captures the front image features of the second target object. Misjudgment is prone to occur when matching the back image features and the front image features of the same target object. Therefore, it can be determined that there is misdetection in the target object group.
[0090] In addition, since the first camera device and the second camera device have a special position setting relationship, it is relatively accurate to judge whether two target objects are the same based on the position information collected by the first camera device and the second camera device. Therefore, the position matching result can be used as a false detection judgment standard for the feature matching result to quickly identify the target object group with false detection, and then obtain false detection sample data based on the target object group with false detection.
[0091] Step 304: determine false detection sample data based on the first video image corresponding to the first target object in the target object group.
[0092] In one embodiment, the above step 304 includes: determining the first video image corresponding to the first target object in the target object group as the false detection sample data.
[0093] Alternatively, obtain fourth position information of the first target object in the target object group in the coordinate system where the first camera device is located; determine the first image area corresponding to the first target object in the target object group in the first video image based on the fourth position information, and determine the first image area or the first video image with the first image area marked as false detection sample data.
[0094] It should be noted that, since the first camera device captures the image features of the target object's back, and false detection is prone to occur when the target object is with its back or sideways, the negative sample data is the image data of the first target object captured by the first camera device from the back shooting angle.
[0095] In the above steps 301 to 304, by utilizing the special position setting relationship between the first camera device and the second camera device, the position matching result of the target object is used as the false detection judgment standard of the feature matching result, so as to quickly identify the target object group with false detection, thereby obtaining false detection sample data based on the target object group with false detection, thereby improving the type richness of negative sample data, and further realizing how to better collect negative samples and improve the quality of negative samples.
[0096] In one embodiment, Figure 5 As shown, the target negative sample data includes multiple inspection sample data; the position matching result includes the first target object that failed to match; and the step of obtaining the target negative sample data based on the position matching result specifically includes:
[0097] Step 401 : acquiring a second video image corresponding to a first target object that fails to match and is captured by a first camera device. The first target object that fails to match includes a target object that is captured by the first camera device but not by the second camera device.
[0098] The second position area is determined based on the first position information of the first target object that failed to match. It should be noted that since the "first target object that failed to match" refers to a target object that is only captured by the first camera device but not by the second camera device, the "first target object that failed to match" can also be referred to as a target object that is detected by the first camera device more than once.
[0099] Step 402: determine multiple inspection sample data based on the second video image corresponding to the first target object that fails to match.
[0100] In one embodiment, the above step 402 includes: determining the second video image corresponding to the first target object that fails to match as the multi-inspection sample data.
[0101] Alternatively, obtain the fifth position information of the first target object that failed to match in the coordinate system of the first camera device; based on the fifth position information, obtain the second image area corresponding to the first target object that failed to match in the second video image, and determine the second image area or the second video image with the second image area marked as multi-inspection sample data.
[0102] In the above steps 401 to 402, by acquiring the second video image corresponding to the first target object that failed to match and captured by the first camera device, and obtaining multi-inspection sample data based on the second video image corresponding to the first target object that failed to match, the type richness of the negative sample data is improved, and how to better collect negative samples and improve the quality of negative samples is further realized.
[0103] In one embodiment, Figure 6As shown, the target negative sample data includes missed sample data; the position matching result includes the second target object that failed to match; and the step of obtaining the target negative sample data based on the position matching result specifically includes:
[0104] Step 501, obtaining target spatiotemporal information corresponding to a second target object that fails to match when captured by a second camera device, wherein the second target object that fails to match includes a target object captured by the second camera device but not by the first camera device.
[0105] The third position area is determined based on the third position information of the second target object that failed to match. It should be noted that since the "second target object that failed to match" refers to a target object that is only captured by the second camera device but not by the first camera device, the "second target object that failed to match" can also be called a target object that is missed by the first camera device.
[0106] Step 502: Based on the target spatiotemporal information, a third video image corresponding to the second target object that failed to match and was captured by the first camera device is obtained.
[0107] The target spatiotemporal information includes the target capture moment corresponding to when the second camera device captures the second target object that fails to match, and / or the target position information of the second target object that fails to match in the coordinate system where the second camera device is located.
[0108] In one embodiment, the above-mentioned step 502 includes: when there is a network delay between the first camera device and the second camera device, obtaining a third video image captured by the first camera device at the target capture moment, wherein the third video image includes an image area corresponding to the second target object that failed to match and an image area corresponding to the surrounding environment.
[0109] It should be noted that, in the case where there is a network delay between the first camera device and the second camera device, it can be ensured that the third video image obtained is a video image containing the second target object that failed to match based on the target capture moment.
[0110] Alternatively, in the case where there is no network delay between the first camera device and the second camera device, the sixth position information of the second target object that failed to match in the coordinate system of the first camera device is determined based on the target position information, and a third video image corresponding to the sixth position information captured by the first camera device is obtained. The third video image only contains the image area corresponding to the second target object that failed to match.
[0111] It should be noted that when there is no network delay between the first camera device and the second camera device, the third video image corresponding to the sixth position information can be directly captured from the current frame video image captured by the first camera device, thereby directly obtaining the third video image that only contains the image area corresponding to the second target object where the match failed.
[0112] Step 503: Determine missed detection sample data based on the third video image corresponding to the second target object that fails to match.
[0113] In one embodiment, when the third video image includes an image area corresponding to the second target object that failed to match and an image area corresponding to the surrounding environment, the above step 503 includes: determining the third video image corresponding to the second target object that failed to match as missed detection sample data.
[0114] Alternatively, obtain the sixth position information of the second target object that failed to match in the coordinate system of the first camera device; based on the sixth position information, obtain the third image area of the second target object that failed to match in the third video image, and determine the third image area or the third video image with the third image area marked as missed detection sample data.
[0115] In the above steps 501 to 503, the third video image corresponding to the second target object that failed to match and was captured by the first camera device is obtained based on the target spatiotemporal information corresponding to the second target object that failed to match and was captured by the second camera device, and the missed detection sample data is obtained based on the third video image corresponding to the second target object that failed to match, thereby improving the type richness of the negative sample data and further realizing how to better collect negative samples and improve the quality of negative samples.
[0116] In summary, the negative sample acquisition method provided by the present invention has at least the following advantages:
[0117] (1) By using a first camera device and a second camera device relatively arranged on a one-way street to perform a simple scene construction, negative sample data of target objects with their backs turned or their sides turned, etc., can be automatically collected during the negative sample detection process. Then, target detection training can be performed based on these negative sample data to avoid false detection, missed detection, and multiple detection of target objects with their backs turned or their sides turned, etc., thereby improving the accuracy of target detection results.
[0118] (2) By using the first camera device and the second camera device installed at different angles to capture the same target object, the capture results at different shooting angles are obtained, and the capture results at different shooting angles are compared to realize the automatic acquisition of negative sample data of the target object with its back or sideways, thereby improving the efficiency of negative sample collection. This method is also applicable to the automatic acquisition of falsely detected negative samples in situations such as poor exposure and occlusion.
[0119] The negative sample acquisition device provided by the present invention is described below. The negative sample acquisition device described below and the negative sample acquisition method described above can be referenced to each other.
[0120] like Figure 7 As shown, the present invention provides a negative sample acquisition device, and the negative sample acquisition device 100 includes:
[0121] The first position acquisition module 1001 is used to acquire first position information of at least one first target object captured by a first camera device, where the first position information represents position information of the first target object in a coordinate system where the first camera device is located.
[0122] The second position acquisition module 1002 is used to obtain second position information of at least one second target object captured by the second camera device, where the second position information represents the position information of the second target object in the coordinate system of the second camera device, and the second camera device is arranged relative to the first camera device.
[0123] The sample data acquisition module 1003 is used to obtain the position matching result between the first target object and the second target object based on the first position information, the second position information and the coordinate mapping relationship between the first camera device and the second camera device, and obtain target negative sample data based on the position matching result.
[0124] In one embodiment, the sample data acquisition module 1003 includes a location matching unit.
[0125] A position matching unit is used to determine the third position information of the second target object in the coordinate system of the first camera device based on the second position information of the second target object and the coordinate mapping relationship; determine the first coordinate area of the first target object in the coordinate system of the first camera device based on the first position information of the first target object; determine the second coordinate area of the second target object in the coordinate system of the first camera device based on the third position information of the second target object; obtain the overlapping coordinate area of the first coordinate area and the second coordinate area, and determine the first overlapping parameter based on the overlapping coordinate area and the first coordinate area; determine the second overlapping parameter based on the overlapping coordinate area and the second coordinate area, and determine the position matching result between the first target object and the second target object based on the first overlapping parameter and the second overlapping parameter.
[0126] In one embodiment, the target negative sample data includes false positive sample data; and the position matching result includes a group of target objects that are successfully matched.
[0127] The sample data acquisition module 1003 also includes a sample acquisition unit, which is used to obtain target feature information corresponding to the first target object and the second target object in the target object group; determine the feature matching result between the first target object and the second target object in the target object group based on the target feature information; when it is determined based on the feature matching result that there is a false detection in the target object group, obtain the first video image corresponding to the first target object in the target object group captured by the first camera device; and determine the false detection sample data based on the first video image corresponding to the first target object in the target object group.
[0128] In one embodiment, the target negative sample data includes multiple inspection sample data; and the position matching result includes the first target object for which the matching fails.
[0129] The sample acquisition unit is also used to acquire a second video image corresponding to the first target object that failed to match and was captured by the first camera device. The first target object that failed to match includes a target object that was captured by the first camera device and not captured by the second camera device; based on the second video image corresponding to the first target object that failed to match, multiple inspection sample data is determined.
[0130] In one embodiment, the target negative sample data includes missed detection sample data; and the position matching result includes a second target object for which matching fails.
[0131] The sample acquisition unit is also used to obtain the target time-space information corresponding to the second target object that failed to match when captured by the second camera device, the second target object that failed to match includes the target object that was captured by the second camera device and not captured by the first camera device; based on the target time-space information, obtain the third video image corresponding to the second target object that failed to match and captured by the first camera device; based on the third video image corresponding to the second target object that failed to match, determine the missed sample data.
[0132] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the negative sample acquisition method provided by the above methods, and the method includes:
[0133] Acquire first position information of at least one first target object captured by a first camera device, the first position information indicating position information of the first target object in a coordinate system where the first camera device is located; acquire second position information of at least one second target object captured by a second camera device, the second position information indicating position information of the second target object in a coordinate system where the second camera device is located, the second camera device being arranged relative to the first camera device; acquire a position matching result between the first target object and the second target object based on the first position information, the second position information and a coordinate mapping relationship between the first camera device and the second camera device, and acquire target negative sample data based on the position matching result.
[0134] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.
[0135] On the other hand, the present invention also provides a target detection method, comprising: acquiring an image to be detected, and inputting the image to be detected into a pre-trained target detection model to obtain a target detection result corresponding to the image to be detected.
[0136] Among them, the target detection model is obtained by training the initial detection model based on the target negative sample data, and the target negative sample data is obtained based on the negative sample acquisition method provided by the above methods.
[0137] In another aspect, the present invention further provides a target detection device, comprising:
[0138] The target detection module is used to acquire the image to be detected and input the image to be detected into a pre-trained target detection model to obtain the target detection result corresponding to the image to be detected; wherein the target detection model is obtained by training the initial detection model based on the target negative sample data, and the target negative sample data is obtained based on the negative sample acquisition method provided by the above methods.
[0139] On the other hand, the present invention also provides a camera device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the target detection method provided by the above methods is implemented.
[0140] In another aspect, the present invention further provides a computer program product, which includes a computer program. The computer program can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the negative sample acquisition method provided by the above methods or the target detection method provided by the above methods, wherein:
[0141] The negative sample acquisition method includes: acquiring first position information of at least one first target object captured by a first camera device, the first position information indicating position information of the first target object in a coordinate system where the first camera device is located; acquiring second position information of at least one second target object captured by a second camera device, the second position information indicating position information of the second target object in a coordinate system where the second camera device is located, the second camera device being arranged relative to the first camera device; acquiring a position matching result between the first target object and the second target object based on the first position information, the second position information and a coordinate mapping relationship between the first camera device and the second camera device, and acquiring target negative sample data based on the position matching result.
[0142] The target detection method includes: obtaining an image to be detected, and inputting the image to be detected into a pre-trained target detection model to obtain a target detection result corresponding to the image to be detected; wherein the target detection model is obtained by training an initial detection model based on target negative sample data, and the target negative sample data is obtained based on the negative sample acquisition method provided by the above methods.
[0143] In another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the negative sample acquisition method provided by the above methods or the target detection method provided by the above methods, wherein:
[0144] The negative sample acquisition method includes: acquiring first position information of at least one first target object captured by a first camera device, the first position information indicating position information of the first target object in a coordinate system where the first camera device is located; acquiring second position information of at least one second target object captured by a second camera device, the second position information indicating position information of the second target object in a coordinate system where the second camera device is located, the second camera device being arranged relative to the first camera device; acquiring a position matching result between the first target object and the second target object based on the first position information, the second position information and a coordinate mapping relationship between the first camera device and the second camera device, and acquiring target negative sample data based on the position matching result.
[0145] The target detection method includes: obtaining an image to be detected, and inputting the image to be detected into a pre-trained target detection model to obtain a target detection result corresponding to the image to be detected; wherein the target detection model is obtained by training an initial detection model based on target negative sample data, and the target negative sample data is obtained based on the negative sample acquisition method provided by the above methods.
[0146] On the other hand, the present invention also provides a negative sample acquisition system, comprising: a first camera device and a second camera device relatively arranged on a one-way street, wherein: the lens orientation of the first camera device is consistent with the road direction of the one-way street; the first camera device and the second camera device are communicatively connected to execute the negative sample acquisition methods provided by the above methods.
[0147] Furthermore, the second camera device is used to transmit the captured second position information of at least one second target object to the first camera device.
[0148] Furthermore, the first camera device is also used to execute the target detection method provided by the above methods.
[0149] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.
[0150] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.
[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A negative sample acquisition method, characterized in that: include: Acquire first position information of at least one first target object captured by a first camera device, where the first position information represents position information of the first target object in a coordinate system where the first camera device is located; Acquire second position information of at least one second target object captured by a second camera device, wherein the second position information represents position information of the second target object in a coordinate system where the second camera device is located, and the second camera device is arranged relative to the first camera device; Based on the first position information, the second position information and the coordinate mapping relationship between the first camera device and the second camera device, a position matching result between the first target object and the second target object is obtained, and target negative sample data is obtained based on the position matching result.
2. The negative sample acquisition method according to claim 1, characterized in that: The obtaining a position matching result between the first target object and the second target object based on the first position information, the second position information, and a coordinate mapping relationship between the first camera device and the second camera device includes: Determine, based on the second position information of the second target object and the coordinate mapping relationship, third position information of the second target object in the coordinate system where the first camera device is located; Based on the first position information of the first target object, determining a first coordinate area of the first target object in the coordinate system where the first camera device is located; Based on the third position information of the second target object, determining a second coordinate area of the second target object in the coordinate system where the first camera device is located; Acquire an overlapping coordinate area of the first coordinate area and the second coordinate area, and determine a first overlapping parameter based on the overlapping coordinate area and the first coordinate area; A second overlap parameter is determined based on the overlap coordinate area and the second coordinate area, and a position matching result between the first target object and the second target object is determined based on the first overlap parameter and the second overlap parameter.
3. The negative sample acquisition method according to claim 1 or 2, characterized in that: The target negative sample data includes false positive sample data; the position matching result includes a target object group that is successfully matched; and obtaining the target negative sample data based on the position matching result includes: Acquire target feature information corresponding to the first target object and the second target object in the target object group; Determine a feature matching result between a first target object and a second target object in the target object group based on the target feature information; In a case where it is determined based on the feature matching result that there is a false detection in the target object group, acquiring a first video image corresponding to a first target object in the target object group captured by the first camera device; The false detection sample data is determined based on a first video image corresponding to a first target object in the target object group.
4. The negative sample acquisition method according to claim 1 or 2, characterized in that: The target negative sample data includes multiple inspection sample data; the position matching result includes the first target object that failed to match; and obtaining the target negative sample data based on the position matching result includes: Acquire a second video image corresponding to the first target object that failed to match and captured by the first camera device, wherein the first target object that failed to match includes a target object captured by the first camera device and not captured by the second camera device; The multi-inspection sample data is determined based on the second video image corresponding to the first target object that fails to match.
5. The negative sample acquisition method according to claim 1 or 2, characterized in that: The target negative sample data includes missed sample data; the position matching result includes a second target object that fails to match; and obtaining the target negative sample data based on the position matching result includes: Acquire target spatiotemporal information corresponding to the second target object that failed to match when the second camera device captures the second target object that failed to match, wherein the second target object that failed to match includes a target object that was captured by the second camera device but not by the first camera device; Based on the target spatiotemporal information, obtaining a third video image corresponding to the second target object that failed to match and was captured by the first camera device; The missed detection sample data is determined based on the third video image corresponding to the second target object that fails to match.
6. A negative sample acquisition device, characterized in that: include: A first position acquisition module, used to acquire first position information of at least one first target object captured by a first camera device, wherein the first position information represents position information of the first target object in a coordinate system where the first camera device is located; A second position acquisition module, used to acquire second position information of at least one second target object captured by a second camera device, wherein the second position information represents position information of the second target object in a coordinate system where the second camera device is located, and the second camera device is arranged relative to the first camera device; A sample data acquisition module is used to obtain a position matching result between the first target object and the second target object based on the first position information, the second position information and the coordinate mapping relationship between the first camera device and the second camera device, and to obtain target negative sample data based on the position matching result.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the negative sample acquisition method according to any one of claims 1 to 5 is implemented.
8. A target detection method, characterized in that: include: Acquire an image to be detected, and input the image to be detected into a pre-trained target detection model to obtain a target detection result corresponding to the image to be detected; The target detection model is obtained by training an initial detection model based on target negative sample data, and the target negative sample data is obtained based on the negative sample acquisition method according to any one of claims 1 to 5.
9. A camera device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the target detection method as claimed in claim 8 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the negative sample acquisition method according to any one of claims 1 to 5 or the target detection method according to claim 8 is implemented.
Citation Information
Patent Citations
Target detection tracking method and device, equipment and storage medium
CN110428449A
Camera linkage snapshot method and device and computer storage medium
CN110738150A
Pedestrian re-identification method and device, equipment and medium
CN111881757A
Difficult sample image screening method and device for video target detection model
CN112347818A
Image processing method and device, storage medium and equipment
CN115471731A
