Image-based actual scene prediction method, system, device and medium
By obtaining the target endpoint pixel coordinates and aircraft information in the image system, converting them to the line of sight and target coordinate systems, and calculating the actual size of the target, the problem of low prediction accuracy of aircraft in the dynamic environment in the prior art is solved, and efficient and accurate are achieved target scenario prediction.
Patent Information
- Application Number
- CN202411404398.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2025-01-14
AI Technical Summary
The prior art is difficult to effectively predict the target scene size when flying at the end of the aircraft, especially in dynamic environments, when rolling, the stereoscopic geometry algorithm is complex and the computing resources are consumed, the prediction accuracy is not high, making it difficult to effectively predict the target scene size.
By obtaining the target endpoint pixel coordinates and aircraft information in the image system, converting them to the line of sight coordinate system and the target coordinate system, calculating the actual size of the target, simplifying the algorithm, and reducing the computing resource requirements.
It improves prediction accuracy, reduces computing resource requirements, is suitable for ships, warehouses, aircraft and other scenarios, and enhances the applicability and efficiency of prediction.
Smart Images
Figure CN119314027A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing and navigation guidance and control technology, and specifically to an image-based actual scene prediction method, system, device and medium. Background Art
[0002] In the prior art, when the final stage of an aircraft's flight does not involve rolling, the target scene size in a three-dimensional geometric object can be used for prediction. If rolling is involved, the three-dimensional geometric algorithm is relatively complex and often relies on complex algorithms and a large amount of computing resources. In addition, in a dynamic environment, the parameters change rapidly and the prediction accuracy is not high.
[0003] Therefore, a method is needed to improve prediction accuracy, reduce the demand for computing resources, and increase computing speed by comprehensively analyzing multiple data. Summary of the invention
[0004] In view of the above-mentioned defects or deficiencies in the prior art, the present application aims to provide an image-based actual scene prediction method, system, device and medium.
[0005] In a first aspect, the present application provides an image-based actual scene prediction method, comprising the following steps: S100. Obtain the coordinates of the two target endpoints in the image system, recorded as the image system endpoint pixel coordinates, wherein the image system is used to characterize the position of the target in the visual image, the origin of which is the upper left corner vertex of the visual image, the y-axis of which is along the longitudinal field of view of the visual image, with the positive direction downward, and the z-axis of which is along the transverse field of view of the visual image, with the positive direction rightward; S200. Acquire an aircraft information set, the aircraft information set including at least an aircraft field of view angle, an aircraft attitude angle, a preset angle, an image rotation angle, and a target distance, wherein the target distance is the distance between the aircraft and the target center point; S300. Obtain the coordinates of the target endpoint in the sight line coordinate system according to the endpoint pixel coordinates of the image system, and record them as the endpoint coordinates of the sight line coordinate system; the origin of the sight line coordinate system is the center of mass of the aircraft, the x-axis is consistent with the sight line of the aircraft target, the y-axis is perpendicular to the x-axis in the plumb plane where the x-axis is located, and the z-axis, the x-axis and the y-axis comply with the right-hand rule; S400. Convert the endpoint coordinates of the sight system to the target coordinate system to obtain the endpoint coordinates of the target coordinate system, wherein the origin of the target coordinate system is the launch point of the aircraft, the y-axis is consistent with the normal of the local reference ellipsoid and points upward, the x-axis is perpendicular to the y-axis and points to the North Pole, and the z-axis, the x-axis and the y-axis follow the right-hand rule; S500. Calculate the actual size of the target based on the coordinates of the two target system endpoints.
[0006] According to the technical solution provided in the embodiment of the present application, step S400 includes the following steps: S410. Convert the endpoint coordinates of the sight system to the aircraft coordinate system to obtain the endpoint coordinates of the flight system; S420. Convert the endpoint coordinates of the flight system to the target coordinate system to obtain the endpoint coordinates of the target system.
[0007] According to the technical solution provided in the embodiment of the present application, step S300 includes the following steps: S310. Convert the endpoint pixel coordinates of the image system to the line of sight coordinate system to obtain the endpoint pixel coordinates of the line of sight system; S320. Obtain the line of sight endpoint coordinates according to the line of sight endpoint pixel coordinates.
[0008] According to the technical solution provided in the embodiment of the present application, step S320 includes the following steps: S321. According to the sight angle of the aircraft and the target distance, the sight system endpoint pixel coordinates are converted into length coordinates to obtain the sight system y-axis length coordinates and the sight system z-axis length coordinates; S322. Obtain the line of sight x-axis length coordinate according to the line of sight y-axis length coordinate and the line of sight z-axis length coordinate; S323. Obtain the endpoint coordinates of the line of sight system according to the y-axis length coordinate of the line of sight system, the z-axis length coordinate of the line of sight system, and the x-axis length coordinate of the line of sight system.
[0009] According to the technical solution provided in the embodiment of the present application, step S322 includes the following steps: S3221. Preset the x-axis length coordinate of the target endpoint in the line of sight coordinate system, recorded as the preset line of sight x-axis length coordinate; S3222. Constructing a preset sight system endpoint coordinate according to the sight system y-axis length coordinate, the sight system z-axis length coordinate, and the preset sight system x-axis length coordinate; S3223. Convert the preset sight system endpoint coordinates to the target coordinate system to obtain the preset target endpoint length coordinates, wherein the preset target endpoint length coordinates include the preset target system x-axis length coordinates, the preset target system y-axis length coordinates, and the preset target system z-axis length coordinates, all of which are related to the preset sight system x-axis length coordinates; S3224. Reversely infer the value of the preset line of sight system x-axis length coordinate based on the preset target endpoint length coordinate, which is the line of sight system x-axis length coordinate.
[0010] According to the technical solution provided in the embodiment of the present application, the value of the y-axis coordinate of the preset target system is 0 under the target coordinate system, and the value of the x-axis length coordinate of the preset line of sight system is calculated.
[0011] According to the technical solution provided in the embodiment of the present application, the actual size of the target is obtained by using the Euclidean formula based on the coordinates of the two target system endpoints.
[0012] In a second aspect, the present application provides an image-based actual scene prediction system for implementing the above-mentioned image-based actual scene prediction method, the system comprising: A first acquisition module, wherein the first acquisition module is configured to acquire the coordinates of two target endpoints in an image system, which are recorded as image system endpoint pixel coordinates, wherein the image system is used to characterize the position of the target in the visual image, the origin of which is the upper left corner vertex of the visual image, the y-axis of which is along the longitudinal field of view of the visual image, and is positive downward, and the z-axis of which is along the transverse field of view of the visual image, and is positive rightward; A second acquisition module, wherein the second acquisition module is configured to acquire an aircraft information set, wherein the aircraft information set at least includes an aircraft field of view angle, an aircraft attitude angle, a preset angle, an image rotation angle, and a target distance, wherein the target distance is a distance between the aircraft and a target center point; A first calculation module, wherein the first calculation module is configured to obtain the coordinates of the target endpoint in the sight line coordinate system according to the pixel coordinates of the endpoint in the image system, which are recorded as the endpoint coordinates of the sight line system; the origin of the sight line coordinate system is the center of mass of the aircraft, the x-axis is consistent with the sight line of the aircraft target, the y-axis is perpendicular to the x-axis in the plumb plane where the x-axis is located, and the z-axis, the x-axis and the y-axis comply with the right-hand rule; a second calculation module, the second calculation module being configured to convert the endpoint coordinates of the sight system into a target coordinate system to obtain the endpoint coordinates of the target system, wherein the origin of the target coordinate system is the launch point of the aircraft, the y-axis is consistent with the normal of the local reference ellipsoid and points upward, the x-axis is perpendicular to the y-axis and points to the North Pole, and the z-axis, the x-axis and the y-axis follow the right-hand rule; A third calculation module is configured to calculate the actual size of the target based on the coordinates of the two target system endpoints.
[0013] In a third aspect, the present application provides a terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image-based actual scene prediction method as described above when executing the computer program.
[0014] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the image-based actual scene prediction method as described above are implemented.
[0015] In summary, the present application proposes an image-based actual scene prediction method, which obtains the image system endpoint pixel coordinates and aircraft information set of two target endpoints in the image system, obtains the line of sight system endpoint coordinates of the target endpoint in the line of sight coordinate system according to the image system endpoint pixel coordinates, and then converts them to the target coordinate system to obtain the target system endpoint coordinates, and finally calculates the actual size of the target according to the two target system endpoint coordinates. The present application can know the size of the target in the actual scene only through the image data and related command telemetry data packets collected by the aircraft, without the need for complex algorithms and a large amount of computing resources, and can be used for ships, warehouses, airplanes, etc., and has strong applicability. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 A flowchart of an actual scene prediction method based on an image provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of the terminal device provided for this application.
[0018] The text annotations in the figure represent: 700. Computer system; 701. CPU; 702. ROM; 703. RAM; 704. Bus; 705. I / O interface; 706. Input part; 707. Output part; 708. Storage part; 709. Communication part; 710. Drive; 711. Removable media. DETAILED DESCRIPTION
[0019] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.
[0020] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0021] Example 1 As mentioned in the background technology, the present application proposes an image-based actual scene prediction method, comprising the following steps: S100. Obtain the coordinates of the two target endpoints in the image system, recorded as the image system endpoint pixel coordinates, wherein the image coordinate system is used to represent the position of the target placed on the ground in the visual image, the origin of which is the upper left corner vertex of the visual image, the y-axis is along the longitudinal field of view of the visual image, and is positive downward, and the z-axis is along the transverse field of view of the visual image, and is positive rightward; The visual image can be obtained by a visual camera on the aircraft. It is a prior art to obtain the pixel coordinates of the endpoints of the image system, which will not be described in detail here. The image system has only two dimensions. S200. Acquire an aircraft information set, the aircraft information set including at least an aircraft field of view angle and a target distance, wherein the target distance is the distance between the aircraft and the target center point; The field of view angle of the aircraft is an inherent property of the aircraft. The aircraft is provided with a distance measuring device, through which the target distance can be obtained; S300. Obtain the coordinates of the target endpoint in the sight line coordinate system according to the endpoint pixel coordinates of the image system, and record them as the endpoint coordinates of the sight line coordinate system; the origin of the sight line coordinate system is the center of mass of the aircraft, the x-axis is consistent with the sight line of the aircraft target, the y-axis is perpendicular to the x-axis in the plumb plane where the x-axis is located, and the z-axis, the x-axis and the y-axis comply with the right-hand rule; The sight line coordinate system is recorded as o p - x s y s z s , when the target is in front of the aircraft, y s The axis is positive when it points upward. o p - x s y s z s Constitute a right-handed rectangular coordinate system; the image coordinate system is the line of sight coordinate system o p - y s z s The plane is perpendicular to x s The projection in the axis direction, the o-yz plane in the image coordinate system and the line of sight coordinate system x s vertical.
[0022] S400. Convert the endpoint coordinates of the sight system to the target coordinate system to obtain the endpoint coordinates of the target coordinate system, wherein the origin of the target coordinate system is the launch point of the aircraft, the y-axis is consistent with the normal of the local reference ellipsoid and points upward, the x-axis is perpendicular to the y-axis and points to the North Pole, and the z-axis, the x-axis and the y-axis follow the right-hand rule; The steps include: S410. Convert the endpoint coordinates of the sight system to the aircraft coordinate system to obtain the endpoint coordinates of the flight system; wherein the aircraft coordinate system is recorded as , whose origin o p is the center of mass of the aircraft, The axis coincides with the longitudinal axis of the aircraft, and the direction pointing toward the head is positive; The axis is located in the longitudinal symmetry plane of the aircraft. The axis is vertical, pointing upward is positive; Axis perpendicular to The plane and direction are determined by the right-hand rectangular coordinate system.
[0023] The aircraft information further includes a preset angle, and the preset angle includes: Heading preset angle The line of sight coordinate system x s Axis in the aircraft coordinate system x p o p y p Projection in the plane x p The angle of , and its positive and negative values conform to the right-hand screw rule.
[0024] Pitch preset angle : is the line of sight coordinate system x s Axis and Vehicle Coordinate System x p o p y p The angle of the plane, , and its positive and negative values conform to the right-hand screw rule.
[0025] Roll preset angle : is the line of sight coordinate system z s Axis and Vehicle Coordinate System x p o p z p The angle of the plane, , and its positive and negative values conform to the right-hand screw rule.
[0026] The first direction cosine matrix converted from the line of sight coordinate system to the aircraft coordinate system is As shown below:
[0027] The product of the flight system endpoint coordinates and the first direction cosine matrix is the sight system endpoint coordinates, thereby obtaining the flight system endpoint coordinates.
[0028] S420. Convert the endpoint coordinates of the flight system to the target coordinate system to obtain the endpoint coordinates of the target system; The aircraft information set also includes an aircraft attitude angle, which includes a pitch angle φ , yaw angle ψ and roll angle γ : Pitch Angle φ : is the aircraft coordinate system x p Axis in target plane x T oy T The projection and x The angle of the axis, , and its positive and negative values conform to the right-hand screw rule.
[0029] Yaw angle ψ : is the aircraft coordinate system x p Axis and target plane x T oy T The angle of , and its positive and negative values conform to the right-hand screw rule.
[0030] Roll Angle γ : is the aircraft coordinate system y p Axis and target plane x T oy T The angle of , and its positive and negative values conform to the right-hand screw rule.
[0031] The second direction cosine matrix from the aircraft coordinate system to the target coordinate system As shown below:
[0032] The product of the target system endpoint coordinates and the second direction cosine is the flight system endpoint coordinates, and then the target system endpoint coordinates are obtained.
[0033] S500. Calculate the actual size of the target based on the coordinates of the two target system endpoints.
[0034] Wherein, the actual size of the target is obtained by using the Euclidean formula according to the coordinates of the two target system endpoints; for example, the coordinates of the two target system endpoints are respectively ( x 1, y 1), ( x 2, y 2), then the actual size of the target l for: .
[0035] In summary, this application can know the size of the target in the actual scene only through the image data and related command telemetry data packets collected by the aircraft, without the need for complex algorithms and a large amount of computing resources. It can be used for ships, warehouses, airplanes, etc., and has strong applicability.
[0036] In a preferred embodiment, step S300 includes the following steps: S310. Convert the endpoint pixel coordinates of the image system to the line of sight coordinate system to obtain the endpoint pixel coordinates of the line of sight system; If the image endpoint pixel coordinates of point W are (y, z), and the visual image plane is A×B pixels, then the y-axis line-of-sight pixel coordinates of point W in the line-of-sight coordinate system are y s and the z-axis line of sight pixel coordinate z s They are calculated by the following formulas: Formula (1) S320. Obtaining the line of sight endpoint coordinates according to the line of sight endpoint pixel coordinates; comprising the following steps: S321. According to the sight angle of the aircraft and the target distance, the sight system endpoint pixel coordinates are converted into length coordinates to obtain the sight system y-axis length coordinates and the sight system z-axis length coordinates; The conversion into length coordinates is a prior art and will not be described in detail here.
[0037] S322. Obtaining the line of sight x-axis length coordinate according to the line of sight y-axis length coordinate and the line of sight z-axis length coordinate; comprising the following steps: S3221. Preset the x-axis length coordinate of the target endpoint in the line of sight coordinate system, recorded as the preset line of sight x-axis length coordinate; S3222. Constructing a preset sight system endpoint coordinate according to the sight system y-axis length coordinate, the sight system z-axis length coordinate, and the preset sight system x-axis length coordinate; Among them, since the visual image is a two-dimensional image, it is converted into a three-dimensional image in the sight line coordinate system, lacking an x-axis. Therefore, the preset sight line system x-axis length coordinate is xs , the pixel coordinates of the endpoints of the preset sight system are ( x s , y s , z s ).
[0038] S3223. Convert the preset sight system endpoint coordinates to the target coordinate system to obtain the preset target endpoint length coordinates, wherein the preset target endpoint length coordinates include the preset target system x-axis length coordinates, the preset target system y-axis length coordinates, and the preset target system z-axis length coordinates, all of which are related to the preset sight system x-axis length coordinates; S3224. Reversely infer the value of the preset sight system x-axis length coordinate according to the preset target endpoint length coordinate, which is the sight system x-axis length coordinate. This includes the following steps: S32241. According to the target coordinate system, the value of the y-axis coordinate of the preset target system is 0, and the value of the x-axis length coordinate of the preset sight system is calculated; Among them, since the origin of the target coordinate system is the launch point, that is, the origin is on the ground, the target coordinate system x T oz T The plane is the earth, y T Pointing to the sky, while the actual target is on the ground, the y-axis value is 0. The preset target system x-axis length coordinate, the preset target system y-axis length coordinate, and the preset target system z-axis length coordinate are all related to the preset line of sight system x-axis length coordinate; the preset target system y-axis length coordinate is included x s By solving the linear equation, we can get x s The value of is the x-axis length coordinate of the preset line of sight system.
[0039] S323. Obtain the endpoint coordinates of the sight system according to the y-axis length coordinate of the sight system, the z-axis length coordinate of the sight system and the x-axis length coordinate of the sight system; The preset line of sight system x-axis length coordinate, the line of sight system y-axis length coordinate, and the line of sight system z-axis length coordinate obtained by reverse deduction are combined to obtain the line of sight system endpoint coordinate.
[0040] Example 2 Based on Example 1, the present application proposes an image-based actual scene prediction system for implementing the above-mentioned image-based actual scene prediction method, the system comprising: A first acquisition module, wherein the first acquisition module is configured to acquire the coordinates of two target endpoints in an image system, which are recorded as image system endpoint pixel coordinates, wherein the image system is used to characterize the position of the target in the visual image, the origin of which is the upper left corner vertex of the visual image, the y-axis of which is along the longitudinal field of view of the visual image, and is positive downward, and the z-axis of which is along the transverse field of view of the visual image, and is positive rightward; A second acquisition module, wherein the second acquisition module is configured to acquire an aircraft information set, wherein the aircraft information set at least includes an aircraft field of view angle, an aircraft attitude angle, a preset angle, an image rotation angle, and a target distance, wherein the target distance is a distance between the aircraft and a target center point; A first calculation module, wherein the first calculation module is configured to obtain the coordinates of the target endpoint in the sight line coordinate system according to the pixel coordinates of the endpoint in the image system, which are recorded as the endpoint coordinates of the sight line system; the origin of the sight line coordinate system is the center of mass of the aircraft, the x-axis is consistent with the sight line of the aircraft target, the y-axis is perpendicular to the x-axis in the plumb plane where the x-axis is located, and the z-axis, the x-axis and the y-axis comply with the right-hand rule; a second calculation module, the second calculation module being configured to convert the endpoint coordinates of the sight system into a target coordinate system to obtain the endpoint coordinates of the target system, wherein the origin of the target coordinate system is the launch point of the aircraft, the y-axis is consistent with the normal of the local reference ellipsoid and points upward, the x-axis is perpendicular to the y-axis and points to the North Pole, and the z-axis, the x-axis and the y-axis follow the right-hand rule; A third calculation module is configured to calculate the actual size of the target based on the coordinates of the two target system endpoints.
[0041] Example 3 like Figure 2As shown, the computer system 700 of the terminal device includes a CPU 701, which can perform various appropriate actions and processes according to the program stored in the ROM 702 or the program loaded from the storage part 708 to the RAM 703. In the RAM 703, various programs and data required for system operation are also stored. The CPU 701, the ROM 702 and the RAM 703 are connected to each other through the bus 704. The I / O interface 705 is also connected to the bus 704. The following components are connected to the I / O interface 705: an input part 706 including a keyboard, a mouse, etc.; an output part 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage part 708 including a hard disk, etc.; and a communication part 709 including a network interface card such as a LAN card, a modem, etc. The communication part 709 performs communication processing via a network such as the Internet. The drive is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, is mounted on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.
[0042] In particular, according to the embodiments of the present application, the above reference process Figure 1 The described process can be implemented as a computer software program. For example, embodiment 3 of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by CPU 701, the above-mentioned functions defined in the present computer system 700 are executed.
[0043] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0044] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments 3 of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0045] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the image-based actual scene prediction method as described in the above embodiment.
[0046] For example, the electronic device may implement Figure 1 As shown in: S100. Obtain the coordinates of the two target endpoints in the image system, recorded as the image system endpoint pixel coordinates, wherein the image system is used to characterize the position of the target in the visual image, its origin is the upper left corner vertex of the visual image, its y-axis is along the longitudinal field of view direction of the visual image, downward is positive, and its z-axis is along the transverse field of view direction of the visual image, right is positive; S200. Obtain an aircraft information set, the aircraft information set at least includes an aircraft field of view angle, an aircraft attitude angle, a preset angle, an image rotation angle, and a target distance, wherein the target distance is the distance between the aircraft and the target center point; S300. Obtain according to the image system endpoint pixel coordinates The coordinates of the target endpoint in the line of sight coordinate system are recorded as the line of sight system endpoint coordinates; the origin of the line of sight coordinate system is the center of mass of the aircraft, the x-axis is consistent with the line of sight of the aircraft target, the y-axis is perpendicular to the x-axis in the plumb plane where the x-axis is located, and the z-axis follows the right-hand rule with the x-axis and the y-axis; S400. Convert the line of sight system endpoint coordinates to the target coordinate system to obtain the target system endpoint coordinates, wherein the origin of the target coordinate system is the aircraft launch point, the y-axis is consistent with the normal of the local reference ellipsoid and points upward, the x-axis is perpendicular to the y-axis and points to the North Pole, and the z-axis follows the right-hand rule with the x-axis and the y-axis; S500. Calculate the actual size of the target based on the two target system endpoint coordinates.
[0047] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments disclosed in the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.
[0048] In addition, although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.
[0049] Through the description of the above implementations, those skilled in the art can easily understand that the example implementations described here can be implemented by software, or by combining software with necessary hardware.
[0050] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the present application.
Claims
1. A method for predicting actual scenes based on images, characterized in that: The steps include: S100. Obtain the coordinates of the two target endpoints in the image system, recorded as the image system endpoint pixel coordinates, wherein the image system is used to characterize the position of the target in the visual image, the origin of which is the upper left corner vertex of the visual image, the y-axis of which is along the longitudinal field of view of the visual image, with the positive direction downward, and the z-axis of which is along the transverse field of view of the visual image, with the positive direction rightward; S200. Acquire an aircraft information set, the aircraft information set including at least an aircraft field of view angle, an aircraft attitude angle, a preset angle, an image rotation angle, and a target distance, wherein the target distance is the distance between the aircraft and the target center point; S300. Obtain the coordinates of the target endpoint in the sight line coordinate system according to the endpoint pixel coordinates of the image system, and record them as the endpoint coordinates of the sight line coordinate system; the origin of the sight line coordinate system is the center of mass of the aircraft, the x-axis is consistent with the sight line of the aircraft target, the y-axis is perpendicular to the x-axis in the plumb plane where the x-axis is located, and the z-axis, the x-axis and the y-axis comply with the right-hand rule; S400. Convert the endpoint coordinates of the sight system to the target coordinate system to obtain the endpoint coordinates of the target coordinate system, wherein the origin of the target coordinate system is the launch point of the aircraft, the y-axis is consistent with the normal of the local reference ellipsoid and points upward, the x-axis is perpendicular to the y-axis and points to the North Pole, and the z-axis, the x-axis and the y-axis follow the right-hand rule; S500. Calculate the actual size of the target based on the coordinates of the two target system endpoints.
2. The image-based actual scene prediction method according to claim 1, characterized in that: Step S400 includes the following steps: S410. Convert the endpoint coordinates of the sight system to the aircraft coordinate system to obtain the endpoint coordinates of the flight system; S420. Convert the endpoint coordinates of the flight system to the target coordinate system to obtain the endpoint coordinates of the target system.
3. The image-based actual scene prediction method according to claim 1, characterized in that: Step S300 includes the following steps: S310. Convert the endpoint pixel coordinates of the image system to the line of sight coordinate system to obtain the endpoint pixel coordinates of the line of sight system; S320. Obtain the line of sight endpoint coordinates according to the line of sight endpoint pixel coordinates.
4. The image-based actual scene prediction method according to claim 3, characterized in that: Step S320 includes the following steps: S321. According to the sight angle of the aircraft and the target distance, the sight system endpoint pixel coordinates are converted into length coordinates to obtain the sight system y-axis length coordinates and the sight system z-axis length coordinates; S322. Obtain the line of sight x-axis length coordinate according to the line of sight y-axis length coordinate and the line of sight z-axis length coordinate; S323. Obtain the endpoint coordinates of the line of sight system according to the y-axis length coordinate of the line of sight system, the z-axis length coordinate of the line of sight system, and the x-axis length coordinate of the line of sight system.
5. The image-based actual scene prediction method according to claim 4, characterized in that: Step S322 includes the following steps: S3221. Preset the x-axis length coordinate of the target endpoint in the line of sight coordinate system, recorded as the preset line of sight x-axis length coordinate; S3222. Constructing a preset sight system endpoint coordinate according to the sight system y-axis length coordinate, the sight system z-axis length coordinate, and the preset sight system x-axis length coordinate; S3223. Convert the preset sight system endpoint coordinates to the target coordinate system to obtain the preset target endpoint length coordinates, wherein the preset target endpoint length coordinates include the preset target system x-axis length coordinates, the preset target system y-axis length coordinates, and the preset target system z-axis length coordinates, all of which are related to the preset sight system x-axis length coordinates; S3224. Reversely infer the value of the preset line of sight system x-axis length coordinate based on the preset target endpoint length coordinate, which is the line of sight system x-axis length coordinate.
6. The image-based actual scene prediction method according to claim 5, characterized in that: According to the target coordinate system, the value of the y-axis coordinate of the preset target system is 0, and the value of the x-axis length coordinate of the preset sight system is calculated.
7. The image-based actual scene prediction method according to claim 1, characterized in that: The actual size of the target is obtained by using the Euclidean formula based on the coordinates of the two target system endpoints.
8. An image-based actual scene prediction system, used to implement the image-based actual scene prediction method according to any one of claims 1 to 7, characterized in that: The system comprises: A first acquisition module, wherein the first acquisition module is configured to acquire the coordinates of two target endpoints in an image system, recorded as image system endpoint pixel coordinates, wherein the image system is used to characterize the position of the target in the visual image, the origin of which is the upper left corner vertex of the visual image, the y-axis of which is along the longitudinal field of view of the visual image, with downward being positive, and the z-axis of which is along the transverse field of view of the visual image, with rightward being positive; A second acquisition module, wherein the second acquisition module is configured to acquire an aircraft information set, wherein the aircraft information set at least includes an aircraft field of view angle, an aircraft attitude angle, a preset angle, an image rotation angle, and a target distance, wherein the target distance is a distance between the aircraft and a target center point; A first calculation module, wherein the first calculation module is configured to obtain the coordinates of the target endpoint in the sight line coordinate system according to the pixel coordinates of the endpoint in the image system, which are recorded as the endpoint coordinates of the sight line system; the origin of the sight line coordinate system is the center of mass of the aircraft, the x-axis is consistent with the sight line of the aircraft target, the y-axis is perpendicular to the x-axis in the plumb plane where the x-axis is located, and the z-axis, the x-axis and the y-axis comply with the right-hand rule; a second calculation module, the second calculation module being configured to convert the endpoint coordinates of the sight system into a target coordinate system to obtain the endpoint coordinates of the target system, wherein the origin of the target coordinate system is the launch point of the aircraft, the y-axis is consistent with the normal of the local reference ellipsoid and points upward, the x-axis is perpendicular to the y-axis and points to the North Pole, and the z-axis, the x-axis and the y-axis follow the right-hand rule; A third calculation module is configured to calculate the actual size of the target based on the coordinates of the two target system endpoints.
9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the image-based actual scene prediction method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image-based actual scene prediction method according to any one of claims 1 to 7 are implemented.